The situation you describe happens pretty rarely--Uber has done over five billion trips in seven years, and typically drivers are fairly intent on finding their rider quickly and delivering a five-star experience.
In the future, perhaps various improvements to the product can mitigate the situation you describe. For example, Uber uses accelerometer data from the driver phone to improve ride safety. https://eng.uber.com/telematics/
If you have any thoughts on further improving the Uber experience, feel free to reach out.