For me at least, I believe part of the reason is the circular visual layout of the menu, which makes it hard to scan. With traditional context menus, I usually develop muscle memory that helps me aim to the desired option. With pie menus this is not needed, but scanning the available options feels weird, as there's no clear left-to-right reading sequence.
I would love a "square pie" menu that combined the improvements of pie menus (click-and-drag to select, infinite size) with a square, table-like layout surrounding the cursor above, below and to the sides of the initial mouse click point.
Something like this (where <¬ is the mouse cursor):
.___.___.___
| A | B | C |
|___|___|___|
| D | <¬ | E |
|___|___|___|
| F |
|__________|
| G |
|__________|