While it's certainly true that the first page may load slower, and you'll load a few scripts as well, you never need to reload those again. Frameworks like Angular encourage you to use a "service" mindset that capitalizes on this property.
The longer you use a single page app, the fewer round trips you will have. If you ask me, your communication should only be for raw materials (scripts, templates) that you won't need to validate or request again during the current session, and raw data (json). This is more loosely coupled, more cacheable at all the different levels, and more scalable in large part due to the decoupling.
Once the initial view loads, I totally agree that you should intelligently precache all your resources and data asynchronously in the background to usher in the era of near zero-latency user interactions. Preferably, you do this in an order based off of historical behavior/navigation profiling to best use that time/bandwidth you have before the next click.
I get the impression articles similar to this one that there was once a similar mindset surrounding mainframes and dumb terminals. The future is decentralized, web included.