Well, the example on the site show it can be pretty simple (I never had use for it, so I never really tried it ), and formatting could be easely added by editing a classic text file...

So the machine that would convert latex to PDF doesn't have to be the one the text files are written with.

TeX (and LaTex) is pretty old, and designed to run on machines much less powerful than what we have today. Back then, I've used LaTeX on MSDOS PCs with 640 kB of memory. And the original TeX was written in 1978 for a PDP-10.

Many distributions are indeed big and clunky because you get lots of packages and fonts all in one, and there's no good way to selectively install.

Or do you mean verbosity when writing the markup? Actual text doesn't require a lot of markup, and you can make things even simpler by defining your own commands.