In my mind, though, the difference between annoying and insufferable is whether or not you're willing to admit to being incorrect. And Jonathan Blow has gone on record to talk about his failures, flaws, and when he was definitely wrong. So while I can sympathize with people who don't enjoy him on like a personal level. I think it's objectively unfair to sink to "insufferable."
Also, I personally get a LOT of mileage just avoiding people and conversations about things that annoy me. Of course, I suppose your mileage may vary.
I've listened to him far more than probably any other game designer or programmer at this point, probably well over 100 hours worth. Attended a couple of his talks in person at GDC too, many years ago (before Braid was released, he showed a brief demo of it during one of them). Didn't even know who he was, just had interesting sounding game design talks.
He then posted something like "<Company(Google?)>, where they reject you for not knowing <X> algorithm, even though they use and depend on your open source software".
I think Blow said something along the lines of "well the dumbest comp sci 101 student would know that algo, so I'm not sure what kind of coder you are?"
This is vaguely from memory ~10 years ago but is what I remember about Jon Blow too.