I use prezto (https://github.com/sorin-ionescu/prezto -- initially a fork of oh-my-zsh).
I'm in the process of trimming down prezto to exactly what I need, since it also has too much fluff. It's a great resource, I'd recommend it to anyone wanting a slimmer and easier to grok zsh configuration.