Also a big fan of Nuxt. The way it takes care of potentially tricky tasks like SSR and code splitting is a joy. The only thing I've had difficulties with is getting the Lighthouse speed rating above 50.
Admittedly it was more of a personal goal but it's definitely possible and I'm pretty sure nuxt isn't the reason for you not getting it over 50.
Ironically, the only pages where I can't get a 100 score is when I'm using google maps due to a "Does not use passive listeners to improve scrolling performance" rule.