It is easy to go overboard with oh my zsh for sure, but startup time, as long as it's less than ~5 seconds, is not likely to be a big problem for most people. On slower machines the real killer is the time it takes for the next prompt to appear after each command. There are mitigations, but I've definitely had machines where it is almost unbearable.