You can use the visual mode for selecting text, then operate on it with the usual vim business.

There are commands built in for a lot of actions, and if there is a right click option that the extension doesn't provide, you can always write a script to extend it. Vimperator is nicely extensible.

I'm not sure if Vim scrolling with hjkl resolves the issue you saw with the arrow keys- I'm sure that's site dependent, and you'll have to try it yourself.

Good luck!