Backbone is probably several times less restrictive and controlling than something like, say, Ember or Angular. I've only used Angular, so let me comment on that. I think Angular definitely offers some powerful features that justify its structure. Two-way data binding, a system for reusable components (directives), a structure for organizing animations, a system for managing data from RESTful APIs, a full testing scaffold/system, etc. It's not uncommon for jQuery and even Backbone apps that are rewritten in Angular to cut their codebase >2x.
I've heard that Ember is even more restrictive/opinionated than Angular, and perhaps it is more powerful as a result - I don't know. I've been satisfied with Angular.
That being said, there are tradeoffs for using frameworks. For example, you should only do DOM manipulation inside Angular directives. Honestly I think this is a positive, due to various powerful structural features of directives (reusability, modularity, customizability), but for some this can be a negative.
Moreover, because these frameworks are just contained in JavaScript files, you can selectively apply these frameworks, where necessary, to different parts of your web sites.
Ultimately, I think frameworks like Angular or Ember are as restrictive and as powerful as frameworks like Rails and Django. Backbone would be like Flask or Sinatra.