Incidentally I went looking for what pure TeX actually looks like as most people say TeX when they really mean LaTeX. Here's Knuth's TAOCP Volume I errata:
http://www-cs-faculty.stanford.edu/~uno/err1.textxt
Talk about Perl being line noise... I wonder if Knuth has ended up being the only person to actually write pure TeX.
My experience of using LaTeX compared to WYSIWYG is that it's easier to focus on the text in the editor instead of fighting with the UI interface but it then requires some chasing around package bugs and compile errors. In total it's clearly a net positive for me.
Writing pure TeX would seem to push the tradeoff quite a bit, having so much distracting stuff around the text that requires a totally different mindset than the text being written. It was probably designed for a world where typesetting was done as a final step before publication, potentially by someone else.
And no, Knuth is definitely not the only one using plain TeX. Yes, it is a niche.
Like a lot of things, if you're handed a finished product, it looks like random noise. If you could watch it being written, it pretty much makes perfect sense. It's a hideous syntax, and sometimes I still wake up in a cold sweat with long strings of '#' signs running taunting me from the corner of my eyes. But I can't say it was all that daunting to write, although I suspect my TeX code was highly non-idiomatic.