Pick two points in your town: say your home and your office. Imagine a stranger had to get from one to the other, and they know how to drive. The map if for all the information that stranger does not have.
Without map you also need time. In some cases, a lot of it. Not all cities have perfect grid structure and there are one-way streets that can mess you up a lot. In the city where I live it would be really time consuming, because there's a huge pedestrian-only area in the center and many the streets around it are one way, or turning is prohibited at some crossroads so you cannot suddenly decide that you made a mistake and you want to make a U-turn and go back.
Without maps you need time or luck.
OK, how does a first time visitor with a rental car get a sense of direction and distance without a map or driving directions from a local?
How do you reliably solve the beginner's mind problem for a self-driving computer?
Without a baseline input, the car will have to rely on the passengers to do back-seat driving which defeats the point of driverless.
This is not common at all in Europe. Sure, we have a general idea of the direction ("it's with of Paris") but we would not use that for anything else. Especially the building case was weird to me, I woukd have said "the side chose to the entrance" rather than "SE"