>why would people all over the world adopt it against their own self interest

There was no easy going back. Once agricultural societies had settled there would be little if any free land to hunt/gather on. Also, much of the traditional knowledge would be lost in a few generations. Plus, peasants were often kept on their land by force.

Everything has tradeoffs and unforeseen effects and social structure is a slow moving ship. Food security is pretty obviously compelling, and creates a stability that allows a society to scale and grow more wealthy and powerful. The loss in autonomy and flexibility is part of the cost. Individuals see things different ways, but the only vote they get is within a social context that has its own momentum. What wins is not necessarily the society that the individual feels happiest in, but the one that is most evolutionarily fit over many generations and conflicts.