As far as why doesn't a new tractor manufacturer start up, they have to compete with a few things: first, the market for old tractors and parts (if they are repairable basically forever and the amount of land used by agriculture isn't changing, this is not a growth market). Worse, it's much more difficult to design a small engine for sale that is both reliable and will pass current environmental regulations. This is a major barrier to entry that didn't exist decades ago.
So if you're a startup in the ag space, it makes eminently more sense to make small parts for someone else's platform, which is why you see companies doing things like deploying GPS, cameras, lidar, datalogging systems. There, the market opportunity is "all tractors could benefit from this", the competition is initially "other small companies", the exit strategy is "get purchased by a large manufacturer who would rather buy than develop the tech themselves", and the regulatory environment is much more friendly.
A workaround may be selling it as a kit. If it's a kit and not intended for road use you could probably sell whatever you want, similar to selling radio kits that are illegal to assemble.
This doesn't mean it runs dirty, it just didn't pass inspection.
I'm not really a fan of skirting all EPA regulations, but passing them is a huge ordeal and probably not something a startup could do without lots of VC that would tell them to just emulate John Deere. It also covers exactly one engine setup.
I would start reading here:
https://www.epa.gov/regulations-emissions-vehicles-and-engin...
All of the small family owned operators simply went out of business, and the large corporate drillers have little interest in small rural markets.
Do all the major farm equipment manufacturers design and build their own engines?