I cannot for the life of me remember where I read this, but I heard that marketing is what labeled clover as bad. Clover was supposedly a fine plant to have in your yard. Herbicides couldn't (or was too expensive to) be formulated to kill clover without also killing other desirable grasses. The fix was to market clover as an undesirable weed instead.
You came very close to my understanding - herbicides that left clover and desirable grass alone were not developed - the best performing weed killers also killed clover, so clover was defined by marketing as a weed to fix that particular bug of the herbicides.
I used to really dislike it specifically because of the white flowers and how they attracted bees, but have since made an almost 180 on it. As Isamu said, it has great nitrogen-fixing benefits and I'm now trying to encourage the bees.
As a kid clover was always part of a healthy lawn. I was always told it was there to fix nitrogen. This was before people soaked lawns in weed killer though. As a kid the lawn was just naturally healthy. I don't recall us doing much other than mowing and watering it. It did get stung by bees on my feet repeatedly as a kid which was the main downside to clover but the bees were probably healthier then as well.
They attract bees. Bees scare people who are not used to it..also kids. Bee stings..bee allergies. Etc.
Seems unlikely a reason since flowers and herbs, two things people specifically plant, are even more popular with bees.