Agreed. But I was pleasantly surprised to find the map is built out of a HTML table and extremely simple js, and longer I looked at it the more I liked it.
I'd go so far as to say it's the back button that needs to change, rather than the site. By that I mean, it's often rather useful to have a stack-based history of your explorations around a map (like this, or a deep dive into Wikipedia, or...), but it'd also (obviously) be nice to pop the whole stack for this site and jump back to HN in a single step.
After all, why are browser history and bookmark navigation so linear when the way we use the web is not? It's like being forced to use turtle graphics or turn-based navigation to connect places on a geographic map. The bookmark manager in browsers is super primitive and hasn't really evolved in 20 years.
Likewise, think about when you have a plethora of open tabs; I've had as many as 400 at times, spread across 7 or 8 windows, with heavy insite overlap (ie I might have 10 tabs pointing at different books or products, multiple tabs going to a specific news outlet, multiple tabs pointing to different Wiki and Git pages etc.). Yet my open tabs are arranged in highly linear fashion along the top of each browser window, and there's no simple way to pull back from looking at the individual pages to looking at the map of my page universe.
tl;dr We use the web in a nonlinear way, like jumping around a dynamic tree, but the browser limits us to an ant-like perspective where leaves are privileged over the tree.