That's always the trotted-out response, most often recited by people who've read someone else saying it but not had the experience to form an opinion for themselves.
You absolutely don't end up recreating a massive, bloated, slow framework that tries to be as generic as possible to try to be everything to everyone. What you end up creating is a relatively tiny, fast, piece of code that is specific to that site's needs. The difference in size, speed and simplicity tends to be absolutely enormous.
Once you start adding dynamic forms for managing multiple items of different kinds, dashboards, error states, i18n, etc, you either end up with a homegrown framework, or with a lot of spaghetti.
Almost every web application being built today, framework or no framework, is a variation of a theme. We have no issue recreating those ad nauseum, but we draw the line at recreating a framework?