This isn't a typical academic research paper but is from a University of Chicago researcher, and I think gives a great overview of failure modes of complex systems in general (software included): https://how.complexsystems.fail .

As someone who writes a lot of complex/evolving data analysis software that needs to work correctly, I find some of the considerations listed in the above to be immensely helpful.