Autojump is the answer here. Totally a gamechanger. If you like its semantics enough, check out fasd [1] , which is a superset of autojump that applies its cache in many other places. Absolutely awesome.

[1] https://github.com/clvv/fasd