I have emacs running as a systemd service, so I never notice its real start time.
> I don't actually want Vim to grow to approach Emacs in customization.
It already has. The difference is that it doesn't come with very many packages, and packages aren't all written in elisp.
Neovim will end up with a similar amount of packages eventually. A lot of those packages will be written in Lua, making them relatively fast and lightweight.
(Neo)Vim doesn't lend itself to total customizability, since its foundation is dependent on its default keybindings.