While Frost is beautifully right here, of course,
there is a mild irony in that many of the rocks in
New England walls were brought there by repeated glaciation,
so cold both made the walls possible, and, as frost-heave,
both surfaces buried rocks, and breaks down dry walls
made with the generally rounded glacial rocks.
A secondary cause for spilling the upper boulders is the behavior of herds of dairy cattle, with each cow reaching over the wall to reach that one extra blade of tasty grass, with eventual wall collapse.