Horses were bread for that work as well, there was a direct 1-to-1 correlation between the two. Not so with humans.
This is handwringing over the decline of paper or pencils after computers came about when paper didn't magically come about by itself. Horses were not wild animals in North America at any point in modern history and barely in other continents. They were purely bred for industry, travel, and hobby. When industry and travel no longer became a thing we stopped producing them: a clear 1-to-1 correlation.
There's already been reductions in human populations without AI from leisure, abundance, culture, etc, - a very very different causality than horses.
You make a very compelling argument. It made me think from another angle. Consider that there's evolutionary pressure on humans to want to procreate. So in a certain sense, we are conceived to serve the gene and to survive we have to labor.
And when you're done doing whatever it is that is, do you go home to your home, your stable, if you like, to sleep and eat?
Are you truly so different from a horse in the scheme of things just because you spend a portion of your free time staring at a mobile phone?