This somewhat annoys me. You can definitely do it.
Sure, it's easier, more comfortable, safer and quicker to use something like React. But you can still do some modularization that scales reasonably well without it.
I've been working on an app and spent a lot of time looking into frameworks to handle its complexity but ended up sticking to vanilla JS to keep more control, especially for detail optimizations.