I really don't think this is an optimal solution. This makes every page load slowly, instead of just the first page. Waiting to do work doesn't make the work any quicker.
It isn't purely about speed, it is about perception, a very slow first page load is way more annoying that a couple half or quarter second loads distributed over a long interaction, after the js is cached all is nearly instantaneous, but you don't have to wait a while to start using the site in your first visit.
If you want you can also wait for the first page to download completely and them import the remaining js while the user is on the first page, but I didn't tried to see how well it works.