I've come to believe that the second part of this statement really is the primary reason why Lisp drives some people absolutely nuts. The parens are really just a distraction and "easy" justification for not liking the language.
It makes sense, though. When you have to read both "outside in" and "bottom up", a lot of your acquired reading abilities (left to right, top to bottom) are in tension with the code you're trying to understand.
At least in Clojure, the ->> family of operators ("threading" macros) are doing a good job helping to alleviate this. Maybe they're available in other Lisps, I don't know.
EDIT: found a reference to these macros in Emacs Lisp: http://emacswiki.org/emacs/ThreadMacroFromClojure