I think both can easily be addressed by having flight corridors - much akin to virtual roads in the sky. It does take away some of the freedom / versatility / ability to go as the crow flies; but would ensure a degree of safety / sanity.
EDIT: the important damage to control would be that of citizens on the ground. I'm not suggesting drones should prioritize their own safety.
Restricted zones have some interesting effects. DJI, a Chinese multirotor manufacturer, recently introduced 'no-fly zones' around airports for their ultra-consumer-level uav, the phantom 2. An unfortunate side effect, in the UK at least, is you are now unable to fly your phantom 2 in areas where it is perfectly legal to do so. I believe this limitation also effects where you can place waypoints on their more advanced autopilots.
They also prevent you flying over or near Tienanmen square...