tmux kind of is that daemon. The agent keeps running on the box and I attach from my laptop or phone. What I was missing was everything around it, like paste and files.