Your intuition is right, there's a pretty standard algorithm to solve this called "floating origin".

Essentially you translate the world back to origin when the player gets too far away.