You can have your NextJS/React app server-rendered, and then users visiting a page will get the page. Clicking links within the application will use Javascript routing to load the part of the application that needs to be displayed (and prevent the default navigation from clicking said links). I think in practice it's more likely for users to fall back to default navigation which then relies on full server rendering when they have Javascript disabled, than it is because they've clicked before the front-end router has started working.
Also, before this thread I didn't realize React itself could be server-rendered (I thought that was the distinguishing point of Nextjs). Is it common to use non-next React for SSR?