As this article points out, we're past the point where we need frameworks for small/medium projects. Animation is now handled by CSS3. JavaScript has been updated in all major browsers and now supports bind/each/map/filter right out of the box.
If you're starting now, you can build effectively in JavaScript alone. That said, you may still benefit from a framework if you're doing a very large, long-term project.