In my experience, optimizing for perceived latency (time between user request sent and site becoming usable) inevitably leads to server side rendering, in geographically distributed availability zones. If you know this in advance, then you can choose to use tools that support code reuse across the client/server divide. This way, SSR is mostly a one time cost, and it can be implemented at the very beginning of the project, while everything is still nice and simple. It does impose additional constraints during UI development, but many of those are good practices anyway (proper use of HTML elements, prefer native form elements to fully custom inputs, use URL and history to manage state, etc).

It has been somewhat humbling to watch users on low speed connections use the app before the JS finishes loading; it shows everyone the true value (or lack thereof, for the passive content consumers who make up ~98% of the visitors) of the JS portion of the app, since they see it with and without JS. Sure, they can't drag a box on my fancy visualization, but they can see a snapshot of it, and use the peripheral controls as links to navigate to the desired application state. Then, as they are reading something that is useful to them, the ~2mb JS blob slips in and quietly brings the application to life.

Also, I love the way the 3rd party JS blobs that management mandated, are so obviously slower than the rest of the page. This gives me good ammunition for the arguments where I demand an HTTP API, not a JS blob, from our partners. They hate that, because then they can't stuff their own trackers into our clients. Fork 'em.