If you're involved in the JS scene, learning React will give you a big advantage. At which point you can decide whether you like it or not.
As for mixing JS and HTML, I went to a talk at last year's React Europe that beautifully explained why this isn't a concern. I had the same reservations at first, but I was convinced then. Suffice to say, give it a shot and see how you feel about it after (besides, jsx is optional).
Start here: http://jamesknelson.com/learn-raw-react-no-jsx-flux-es6-webp...
But you're not mixing js and html. JSX is not html, it's a DSL to express a tree-like structure. In that sense, it's like mixing js and...dom creation logic.
The reasons I'm aware of to avoid mixing html and js fall into roughly two categories - separation of concerns(don't mix business rules, state management, control flow etc with presentation) and concerns of encoding(building strings of html makes for poor development, debugging, brittle code, and so on).
Since React is just a view layer, it does not impose an entangling of concerns. You can do everything within React components, or separate things on your own or via some other framework.
Meanwhile, the gains from manipulating a DOM or a Virtual DOM are immediately obvious. Even the most popular libraries like jQuery advocate using a more structured approach than simply concatenating strings(e.g. jQuery's whole DOM creation and manipulation lib - although it does make tradeoffs and still allows plenty of html through). The (Virtual) DOM tree naturally allows for composition, compared to the complexities of composing strings of html together.
The paradigm shift from React isn't so much React itself, but the ecosystem it (subtly, or not so subtly) pushes alongside with it. Immutable state and pure rendering functions force you to think through the states and edge cases of your application that would otherwise remain as subtle bugs. Components with a clean life cycle(as opposed to angular 1 directives) allow for easy composition. The virtual dom allows you to divorce your framework from html and the browser, and transplant it to other devices.
By the way, the separation of markup and style was a fight for a semantic web. Web apps, which are React's primary use-case, generally sidestep that concern. The theoretical benefits of separating markup and style are still the same today, but the expectations placed on a web site have increased tremendously, and you must juggle those benefits with others.
I think the benefits of this were undeniable but not quite for the reasons everyone thought. At that point html was such a mess that imposing ANY structure on it would have been an improvement. CSS saved us from nested image-table-slicing hell.
Separating markup from style is one way to structure a complex document but it's not the only way. However it's better than 'no way'. But don't cargo-cult it.
In the past I tried many times to use some spare time to learn Angular, but it always felt too overengineered to me (also because it does much more than React) and I always stopped at the beginning. With React, just a few hours were enough to appreciate the concepts and get started.
Then of course you can add Redux and that will take more time, but you can almost just as well use React components with any state manager you like.