Why are you asking? Learning how to use webpack (for example) can be difficult depending on your background, but once you set everything up it's pretty solid. Plus you could use boilerplates like create-react-app if you're not willing to invest too much time into figuring it out.
I'm trying to decrease cognitive load in development.
Every single thing I need to learn takes time and introduces complexity.
I try to the greatest extent possible to program the underlying technology. This does not mean I don't use frameworks/libraries - I use lots of them - but I always try to decide if I'm better to learn the underlying tech than the framework/library - if the library is a better way to go then I use it. I'm just very pragmatic about whether or not I need to use that framework/library cause I cannot afford to expend time on learning new stuff if I can afford to.
Full stack development means building everything from the OS to the cloud to the database, the back end web server and front end. I already pay a huge cognitive price to know all that and task switching between parts of such systems incurs further brain cost.
I already know ReactJS very well but I'm just thinking maybe I can reduce the cognitive load further by not using it.
Well, I hope you are not building a full RDBMS everytime you get a new project though.
So, I'm an old timer. I've been doing software development professionally for 25 years. For me, new development is stimulating; maintenance - not so much.
Andrew, here is what I would suggest - create and finish a small project and code it in ReactJS. Then code the same one using only raw javascript. With the experience, you'll have your answer whether the cognitive load is worth it.