If I wanted to paste something in a file remotely, I'd open it in my local editor (which can ssh into remote machines). If I needed to transfer files back and forth, I'd use an FTP client (Filezilla works cross-platform).
If so, please point me to it.
But that's sort of the point I'm trying to make. If the Ubuntu team wants to get Windows devs to try out their thing, they need to make stuff like this completely painless. To the point of building a "Windows Kit" that you simply install at your end and it hooks you up with a complete set of tools you need to interact with your remote server.
I think you're missing the point that admining a Linux server is different. The complete set of tools you need to interact with a remote Linux server is: an SSH client.
As others have stated, spending 15 minutes learning about the linux commandline will save you a ton of time, effort, and pain.
It's just a shame that the tools to do it remotely aren't up to scratch.