Scientific Bug Hunting in the Cloud: An Unexpected CEO Adventure

I don't think it's too disrespectful to summarize this article with a TL:DR.

They were using RCS to do version control of some unknown files in their language. This caused a bunch of IO to happen intermittently when they were doing relatively simple operations.

This looks like the most common cause of "wow it sure is hard to figure out what causes this occasional problem": "we didn't know all of the stuff our code was doing."