Interesting, that could well be it. Languages I pick up very easily (in rough order): C, C++, PHP, C#, Java, Scala, Perl, Python, Ruby, Javascript, Lisp. Haskell is definitely one that gives me trouble as well. It's not the functional aspects: I've no trouble writing recursive descent parsers and the like in C++. I can't stand the parentheses, but I can understand and reason with Lisp.

I think in Haskell's case, it's just doing too much with too little. Like how they can implement quicksort in three short lines of code, and to me it's just so much to unroll in my head at once; but it's hard to be sure for obvious reasons.