Well, without being a neurologist, I assume at least the part of the brain that connects directly to the optical nerves (which is however not the visual cortex) has more or less an exact representation of what your eyes are seeing. So you'd probably need to stimulate that instead...
IANAN either, so I'm checking out [0]. The visual system is really, really complicated. For example, see the uncertainty [1] regarding the differences between vision to support action (e.g. catching a ball) and vision for perception (e.g. recognising a texture). There is no single obvious place to stick a wire from a VR device.