I tried, but I missed a lot of thinks like being able to click a random line instead of down down down down. Or selections. Commands are good but I pretty much just use find and replace in vim.
You don't have to click down multiple times. If you want to go somewhere specific there are too many ways to list tbh but they're all faster than arrow keys or mouse clicks. I'm a big fan of the EasyMotion https://github.com/easymotion/vim-easymotion approach myself but the built-in search functionality + tag navigation is also good.