The biggest things for me are probably the excessive amount of resources used, the long load times, how hard it is to get things "right" (like not breaking the back button) and links taking awhile to load because we have to load your stupid SPA (if they work at all).
SPAs did win out big in one area though. On mobile. Except on mobile they're called "apps". As much as many HNers like to chafe against native mobile development as they yearn for the open standards of the Web, native apps have basically "won". Users love em.
I guess that leaves desktop.
I do wish more people asked "does my Web presence need to be an SPA?" or even "will it benefit significantly from being an SPA?" because I feel like the answer more often than not is "no".