In fact I can't just randomly go walking wherever I want. I can't walk through your backyard, or just stand and hang out there. I'll get arrested for trespassing. Right now the problem is that these rules haven't been defined, but most likely just like cars and walking there will be places where you can and can't go.

I highly suspect that the space above a private property up to the commercial line will be considered part of that's person's space. After that it will be like most everything else, there will be places you can and can't fly, just like there are places you can drive a car, take a walk, and so on.