Until you want to automatically unregister event handlers when the component is removed
Until you want to compose them and select child and parent components without traversing the DOM
Until you want to pass options to the components in various ways, and have default values for them
Until you want to store component state and re-render part of the component whenever something in the state changes (since storing state in the DOM and reading it from there is bad)
Until you want to load javascript files on demand to construct and manage components instead of loading everything in one huge js file up front for a big website
Until you want to have a unified CSS convention so you can style all these components
That's when you need a well tested framework.