If you're making libraries, console applications or doing modern websites it's pretty easy to get your build working without Visual Studio.
One - ELEPHANT SIZE - problem is that for some reason Microsoft only ship the Windows SDKs as part of Visual Studio. Meaning that you'll need to install the IDE at least once to get a build running (you can copy the SDKs around after that).
Microsofties - I know that a lot of dev-div people will be reading - can you shed some light on this situation? What's the reasoning behind not shipping the SDKs separately. I've never had this problem with Java..