I imagine the engineers who wrote the code for the Boeing 737 MAX MCAS or the Tesla engineers who accidentally wrote log files to the internal flash thought the same. Maybe another team member could show the author why these tests are important...
I totally get tests as a way to prevent known incidents from happening again. But tests as a way of preventing unknown issues seems dubious to me.
True. At my previous job, I used a new C compiler [1] that revealed a bug due to undefined behavior [2]. It was a two-line fix, but my manager at the time refused to accept the fix unless I 1) made a test that showed the error, and 2) write tests to show other instances of this type of bug. The test would have been useless since I used an unsanctioned C compiler, and (due to the nature of the bug) I'm not sure if static analysis would have found the bug anyway.
[1] Test driving the latest Ubuntu Linux installation for development.
[2] The code worked as expected on the business-supported C compilers (it's undefined behavior, anything can happen). Even valgrind didn't catch the error (we were using that at the time) because of the way the code was generated.
There are endless permutations that are possible in this modern ideal that are all correct for some or all testing purposes, during all test runs, and in every system the average project participant will ever see up close.
They are wrong in production, vastly outnumber correct production configurations, and confuse any discussions about the attempt to port things to production as they are more canonical to the group than production itself.
That's not to say there aren't ways to do this style of multiple test types correctly, only that I think it was done with more care at each level instead of less as layered redundancy needs particular properties or you get holes that all line up, configuration for tests that make it to production, etc.