Glossing over 10 years of tens of thousands of people's work, things like Titan Rain (1, 2) led to a lot of thinking about monitoring your production environment with things like the istio sidecar system.
(1) https://en.wikipedia.org/wiki/Netwitness
(2) https://en.wikipedia.org/wiki/Shawn_Carpenter