Gulp was much much worse. Nothing worked at all out of the box and months later I still hadn't found the patience to get everything working right. Documentation was next to non-existent and there were points where I wanted to rip my hair out trying to work around "vinyl source streams", whether I needed a buffer or not, how to wrap a regular stream, how to get live reload to work, figure out why they had "blacklisted" popular modules that just made it easy to plug things in. At least it was fast but I never did get it working for some of the things I wanted to do and fell back on grunt sometimes.
Webpack is somewhat better I suppose but at this point I'm just prepared to be disillusioned every time I try some new javascript package which has maybe nulled the pain to some extent.
It isn't even that doing this stuff would be that difficult it's just that the docs are usually wrong, missing, incompatible, or outdated. No one bothers to fix them because by the time they do the JS community has just chugged along to the next "framework" leaving a trail of broken and abandoned projects in its wake.
I don't give a damn about bloat and modularity. Most of the time I'm just looking for a quick solution. If I have to download a big package so be it. Do everyone a favor and package things together that are compatible and work together so I don't have to spend hours tracking down the proper module for doing something like compiling sass and then another hour figuring out how to configure it and insert it into the build tool.
If I wanted to work on build tools then I would work on build tools but I get so tired of doing this stuff each time I need to set something up. In theory it sounds nice for there to be a million ways of getting something done but in practice I think it's just easier for someone to set a precedent for doing it one way and then have the community work on incremental improvements to that one way of doing things. This eliminates a load of confusion, incompatibility, and excess work.
All this might be an unpopular opinion but when I'm working on a project I like to direct my time towards the project, not the supporting development tools.