How does something like this work in general? My naive idea about how to make such a thing essentially seems like a virtual machine, which is quite a bit of effort.
In principle, you can use any compiler of your choice to fake this: Just create a dynamic library for each line that has been entered into the REPL that you immediately load and call into.
Now, 'all' you need to do is cut out the middleman and directly output properly relocated code to memory. There are compilers that can do so. Clang/LLVM is one one of them. TinyCC is another.
For a less confusing explanation, see https://en.wikipedia.org/wiki/Position-independent_code
(Also: no, it's not really implemented by using one offset register for each .so file in the process.)
Another way would be to operate at block- instead of line-level and just not allow local variable to be shared between blocks. Ie
int i = 5 ↵
printf("%i", i) ↵
would fail, but { ↵
int i = 5; ↵
printf("%i", i); ↵
} ↵
would not.