Rather than trying to remember all those commands for moving around in your command, I recommend turning on vim mode for the command line, and remapping ESC to "jk".

bindkey -v bindkey -M viins 'jk' vi-cmd-mode

Then you can edit your command line the same way you would edit a line in vim.