Is there a way I can rig it so every machine I ssh into has already had 'set -o vi' run for my session, without having to change anything on machine I'm logging into?
All of the single-character shell options documented in the description of the set builtin command, including -o, can be used as options when the shell is invoked.
So set RemoteCommand to something like /bin/bash -o vi