With a framework you not only have to learn how build things in the framework (which takes a lot of time to become truly proficient) I also have to build what I am already supposed to be building.
I find personally that frameworks
* Simple things hard. Hard things simple.
* Anything outside of the happy flow they present to you in the docs is a major PITA.
* I frequently find myself scrabbling around for information when I need to override how the framework works in a particular scenario.
* They nullfy much of your pre-existing knowledge e.g. I am described at a lot of places as a JavaScript guy in the team, because I get JavaScript. When I use Angular or React, most that is irrelevant. It doesn't really help me.
If I am just using well established patterns + my existing knowledge of the programming language and environment. I can normally put something together myself that does just what I need it to do and I use a library for things like date time calculations or Ajax requests or something else that is mundane.