The sticking point is the rest of the interface. There's the BlueInput drivers for Windows Mobile, that lets you hook up a bluetooth mouse and keyboard. You could then use RDP to connect to a machine elsewhere and have it host your IDE and compiler. Almost certainly not the "right" way to do it but it's a start.
Multi-touch interfaces turn out to be hard with video glasses, at least on current phones -- you don't see where your finger is. Of course you could ignore the video glasses and just look at the phone screen, but then you have a small screen.
May be worth thinking through what kinds of development are best suited for a person walking around. Are you planning to camp out at a coffee shop and work for six hours straight, are you planning to make quick changes to an existing app "on the fly" as you think about them, or something else?