Programs that use readline can be configured to use vi mode via inputrc. For the few that aren't you can always use rlwrap. Overall there's no downside.