But all these old monolithic frameworks/libraries, that try to solve everything and there citchen sink, are so 2009-ish.
They have its place in the enterprise world. And for performance optimisation you want a more modular approach.
Personally I'm with you, I enjoy a small, isolated, component based approach where each component follows the unix philosophy. I think this lets you write the most performant and resuable code. Also, I think when following this style, it limits the scope of your dependences, which is another big plus. For instance, you might need a library for crypto, but you won't need one for doing your authentication system, and you won't have massive modules for handling auth that would come from a framework getting compiled into your production code.
But when you go this route you do have to make a lot of "right" decisions that are made for you when using a framework. Also, I think a lot has to do with deadlines and developer speed, it initially takes a little longer to write good standalone components that are reusable, and just grabbing some framework will help get shit done in the short term, but might have the same detriments as technical debt in the long run.
Something like React bothers me far less as it smaller, less imposing, and solves real problems I've seen with other frameworks (for example by pushing clear/consistent/simple communication paths).
Except React is a library. The slogan [1] clearly says "A JavaScript library for building user interfaces".