No, I vastly prefer 10x faster build times and merely 2% file sizes over anything that's on order of magnitude slower. It's not even a competition.
2% sounds small. And it is, if your traffic is small. It's not small when you have millions of users.
But for an established product with substantial traffic, swapping out a js minifier for one that achieves even single digit % compression improvement seems worthwhile to me - if the only downside is adding a few extra seconds to build time.