I just use fzf-marks. Then I can fuzzy cd the only directories that ever matter to me.