But if you are already experienced in a framework like rails, django or laravel, then bashing out a side project should be significantly faster with the framework then without. There are so many things you dont have to re-invent when using a framework like routing, installing an ORM, rendering templates etc.
I honestly haven't read any convincing arguments against using a framework in this thread other than "I don't want to spend time learning it" but if you pick a popular and mature framework like rails or django then learning that will benefit you loads in the long term.
Also keep in mind you could make the same argument against learning a programming language like C and say "I just prefer writing assembly code becuase I know exactly what bits go into which registers".