Are you able to tell right from left without thinking?
I also can't really rotate something. When I spent a few hours with a driving simulator game to see if it was at all possible one big issue I had was not being able to figure out which side was the right side of the road, relative to the new perspective after a turn. So I turned straight into traffic more than once.
But there's GPS apps now so aside from driving and math it's not that limiting. If they ever fix the public transit here I'd probably be fine...