You're never going to be able to overcome physics so you aren't going to somehow make a ten megabyte JavaScript monstrosity load instantly over a spotty 3G connection in the middle of nowhere.
What you can do is give some worst case connection the best chance you can to successfully load your page.
Instead of building it all in the DOM with JavaScript send the client the damn HTML. Send some baseline CSS in a style tag. Use CSS patterns instead of big header images or backgrounds. Use box/text shadows to make elements stand out without images. Bundle as much as possible (CSS sprites, JS bundles, etc) to cut down on external resource requests. Minify and compress the shit out of everything. Include a viewport meta tag.
It's not that difficult to have a good looking and functional page that's literally a single HTML page. A single compressed resource that has everything it needs in a single load.
We're actually in an era where the average web browser is extremely capable and isn't some version of IE. Even super low end Android phones have a very capable browser installed by default. You've got HTML5 and CSS3 along with at least ES6 everywhere. The same is true on the desktop in most areas.
There's also just so much shit you don't need in most pages. Far too many people treat a web page like it's some full blown mini OS. There's also a seeming generation of web devs that have no idea what browsers do and think they need to include the kitchen sink to display some text or load resources. You definitely don't need to override scroll events.
You can never make a universally fast page because you don't control the user's environment. But you can make pages and fast and as light as possible so they're not wasting anyone's time or battery. You can also focus on uncached loads because honestly that's going to be a majority of visits.