Ahhh that would explain why it didn't feel consistent for me, since I use ProxyCommand to hop through another box most of the time. I usually need to do that key sequence twice. But I guess now I know I just need the tilde twice.
I think the ProxyJump config option will help you.
ProxyJump (or the -J option as a shortcut) plus TCP port forwarding (-L/-R) is ridiculously powerful. You can chain jump hosts with commas, threading a port forward deep into (or out of) multiple server hops with a single command.