The bulk of it is stuff you don't need. Most folks probably don't need anything more than the format for the journal they're submitting to or a couple standard formats for books or articles. But your article format and mine might be different, either by aesthetic preference or technical requirement (maybe I'm publishing something on ancient Greek, and you're publishing a paper on some number theory topic).
However, by using a common language/environment we get things like CTAN which help to spread the features we added or allow us to discover features we want so we don't have to reinvent the wheel (though it still happens). It's also probably so large because it's been around so long. LaTeX dates to 1984, with TeX being released in 1978. So we have 31 and 37 years of community contributions to these tools.
For anyone who appreciates Typography, there are few things out there that can produce aesthetic results as pleasing as LaTeX.
It does have its downsides though, no question, including a steep learning curve, unintuitive error msgs etc.
Org-mode tables are pretty cool, bordering on being a spreadsheet... It's worth watching a demo, even if you never use it.
At the 2014 TeX conference one speaker gave out a MacTeX distributions (that's based on TeX Live) on a 8GB USB stick. Personally I wonder whether putting a lot of effort into creating stripped-down versions is worthwhile.