I would expect the opposite with a basic LRU cache before the fetch to OSM
That's fair unfortunately it's not what happened :-(
I'm in a city well under 50,000 people not in the Americas nor Europe. The site gave a message that it was retrieving the data from OSM, then rendered the map faster than the browser would render a png. Very impressive.