The app itself is similar to a module of a much larger product the company sells, but that product is far from being a money-maker for the company (may just break even). People are certainly not buying the whole product because of that module, it's a nice thing to have. There are several other companies that do sell stand-alone apps similar to mine though, there is a market for it.
The other concern was whether starting now the work based on the MIT open-sourced version of my app creates a clean slate for me, since the previous work was done in part during work hours on their PCs. Open-sourcing the app through my employer was the only way not to lose it completely in case I switch companies, and have a chance to try a startup based on it.
Some advice I've got was that forking the OSS version should be done by a friend (not in the company) and I should work on it in the background and officially join later, after I quit - that also feels sketchy.