Agile is a lot like Communism: there's a lot of people who swear it works great but everywhere it seems to have been tried it hasn't gone well... And the apologists just insist "those places didn't understand it and do it right".
I also worked at a funded start up that implemented agile from the beginning, worked pretty great.
Do you really want to go back to the waterfall and V model style of development? Those are basically guaranteed failures in a fast moving industry like software development. If anything your comparison should be applied to the waterfall/V model because it is essentially central planning.
If you are doing things like building MVPs, iterated/incremental development with frequent changes and deployments to production then you are doing agile development.
Maybe it is not hyper formalized like Scrum but it is agile nevertheless.
Versus communism where people (e.g. Benjamin Tucker) were predicting its primary failure mode decades before it ever was instantiated at any scale.
This hinders freedom to experiment with new techniques and methods within development teams, but it doesn't stop it. A "trick" is to provide all the artifacts that they want as if you followed their process to the letter, but still do things the way you want so long as it gets the job done. The problem with that is that you have no evidence you did things differently than the defined process and so they'll continue to believe the defined process is perfectly fine, if not excellent. Then some exec will decide to write a book about it and become a consultant selling the (broken) defined process (I assume this is how SAFe came to be, an ironically very rigid "agile" process).
People are the problem.
What I mostly see it is people doing it to themselves. Someone thinks he has to be guardian of the process and refuses to let the experiment run.
The result: if you're not using safe you're lead is going to be removed. Zero consistency between project tooling.
It's like the worst of both worlds.