I am no expert but another linked story (http://discovermagazine.com/1987/may/02-the-worst-mistake-in...) explains it better why Agriculture had overall deteriorating effect on human health:

1. Early farmers had less access to variety of diet compared to hunter-gatherers. It is hard to disagree with this one. For a lot of Asia, Rice is still the main food and rice on its own contain little more than carbohydrates.

2. Risk of starvation if crops failed.

3. Encouraged crowded societies and deterioration in position of Women. Agriculture forced women to have children often and that in turn impacted their health. I don't know much about Europe but in India Govt. still runs advertisement encouraging Women to have kids at a gap of 3 years or more. I think it is widely held that healthier Mother in general means healthier baby.