On a dad level, well. Protective emotions can be intense.
The solution is to raise the height for normal aircraft. That makes room for drones. We could give the drones from 500 to 1900 feet, putting normal aircraft at 2000 feet and above.
There are also plenty of good reasons to be under 2,000 ft. The most general is that all training and approach for VFR ops at airports is done at 1,000 ft. by default.
Airspace and altitudes are pretty complex, for a more involved example: A VFR flight departing North from Boeing field in Seattle HAS to stay below 1800 ft but above the highest parts of the city at ~750 ft. Because the air-space above 1800 belongs to jets on instrument approach to Sea-tac. In other words, to get out of that airfield you have to fly underneath the 747s. Seattle is just one example, there's weird airspace like this that balances the needs of different users just about everywhere.
In any case, raising floor altitude for non-RC/unmanned aircraft to 2k feet would pose fairly substantial challenges as, even ignoring the fact that airports require approaches below 2k feet and are often near populated areas, lots of commercial applications for light aircraft require low altitudes. Survey, photography, news gathering, law enforcement, sightseeing, etc. These are many of the same applications drones are posed for... there doesn't really seem to be any way to simply separate these types of traffic entirely.