I think you may want to clear the environment (e.g., of `SSH_AUTH_SOCK`) as well as isolate in a PID namespace as well. I also reflexively `--as-pid-1 --die-with-parent`.

    bwrap --dev-bind / / --clearenv --tmpfs ~ --unshare-pid --as-pid-1 --die-with-parent ssh terminal.shop

(The `bwrap` manpage says “you are unlikely to use it directly from the commandline,” yet I use it like this all the time. If you do, too, then we should be friends!)