During times when Roman security was good - the economy could develop complex systems of trade, which developed coinage, mathematics, architecture, and all sorts of specialists and artisans.
When security was no longer reliable due to the influx of raiders, piracy, and hoards of barbarians, all of those complex activities ceased.
The most complex aspects of society are always the first to collapse when security disappears.
...and eventually a "fend for yourself" attitude permeates society - which becomes a feedback loop, and it's no longer the externalities that are destroying security - but internal actors.
We were lucky to have a safe and stable society for so long that we are shocked to see mass looting. Once it's gone though, it is takes generations to rebuild that.