I think the system you propose would require a fat server to properly serialize the necessary contents back to the user's shell. In effect you'd be writing a replacement for an ssh server with deep integration into whatever shell the user is running locally. The alternative, constantly dumping the shell state and interpreting it, would be easy to get wrong, and desyncs of local and remote state could have a severe impact on the commands you run.
It's not technically impossible, just very impractical. I think it also goes against the philosophy of "do one thing and do it right" because of all the moving parts.