This is one of my biggest pet peeves with modern recommendation systems. It's like somebody decided "no, the user can't click a 'show-me-more-like-this' button, it has to be automatic" and now I have to treat every website like I'm doing reinforcement training with a cat.
But at least with AI there's something marginally close to an excuse, because you need a ton of data to make it work. With maps it's just plain and simple bad UX design, I can't think of any reason why locations shouldn't be sorted by last visited instead of last clicked.
> Google has a user history website
From a usability standpoint, if it's not accessible from the same same interface where your results are displayed, then it doesn't exist.