I find Chomsky frustrating because he comes up with these incredibly elegant theoretical machinery--and then is totally disinterested in empirically testing then.
For example, the "Language Acquisition Device" absolutely HAS to be the brain, but a lot of the discussion of it--and Universal Grammar--are totally untethered by psychology and neuroscience.
For a while, it seemed like he regarded recursion as a key aspect of human language, versus animal "communications." Fitch, Hauser, and Chomsky had a 2002 article in Science that seemed to argue as much, and they were excited to show that humans, but not monkeys, can learn center-embedded recursion patterns[1]. However, Chomsky seems to have subtly backed away from that and now says that recursion important but might not manifest itself in all human languages (or something along those lines).