Yep -- when you're dealing with large numbers of DOM elements, you absolutely don't want to have an individual View for each element. At the other extreme, you don't want to have a single View for your entire application. Find a balance that makes sense, and represents a logical chunk of UI.
For a bit more, see the FAQ, particularly the bottom of: http://backbonejs.org/#FAQ-tim-toady