I'm not saying the big trendy thick client JS frameworks are the wrong tool for every job. But in my experience they do seem to be the wrong tool for an alarming number of jobs they are presently being used for. And I'm not just spitting rhetoric here. I'm more than happy to put my money where my mouth is. If you want to know specifically when I think a big thick client JS framework might be the right tool for the job and when it probably isn't, this article does a good job of outlining that: https://www.sitepoint.com/javascript-dependency-backlash-myt...
As for the tradeoff associated with developer productivity vs. performance, there is some validity to that. However, I think people overestimate the developer productivity gains the frameworks offer. You only get productivity gains if your whole team is already fully trained on the framework. And in addition to the performance tradeoffs, you also risk your codebase having a shorter lifespan because the framework may make breaking changes to its API some day (e.g. Angular 1 -> Angular 2), whereas a vanilla JS implementation that relies on smaller, single purpose JS libraries will age considerably better.