I do it. It helps to always have the VM on in the background, so logging into it is as easy as firing up cygwin and ssh'ing in. Also, you should set up samba, so you can edit files in your text editor of choice remotely.
Why not just work directly in the VM? I mainly use emacs on OSX now, so I would expect it to feel natural.
I only have an i5 and 6 GB of Ram, so running directly in the VM isn't that good on performance. I just run a linux server as a VM, not a full desktop. It works well for me.