Twitter must have test environments for developers and automatic tests.
Do they just not have a full staging environment? That would be understandable. It's impossible to create a production-like test environment for a product like Twitter. You're better off just doing canary releases or feature flags with slow rollouts.
What makes you think that?
I mean, I know it costs money and needs people who understand how the product works and how customers behave - but why on earth would it be impossible?
For something with many millions of users, it makes more sense to do canary releases to internal users, then to a few thousand, and then scale up from there.
And if can't afford a 1:1 scale replica, build a 1/10th scale replica and have your senior engineers use their expertise to ensure it's representative enough for test purposes.
Unstable connections. Traffic spikes that can be highly localized or nearly system wide. System maintenance, planned and reactive. Poorly behaved clients. Malicious traffic. Many more similar factors...and yes, each of these you could throw engineers at them and get solutions eventually. You might also end up with a testing engineering team that dwarfs the product engineering team.
Also, Twitter doesn't make money. $5B revenue vs $5.5B opex in 2021. They can't really afford to run the actual system, much less a fully representative one. Even a 1/10th replica, which would be unable to find some classes of problems no matter how many engineers you throw at, would further break the bank. As would the engineering staff to support it.
A 1:10 replica is fine, but also astronomically expensive. What's better about it than 1:100? Or 1:10000?
Once you give up on 1:1 (which is reasonable), you're either testing in production or not really testing at all. You'll have many lines of code that will only apply to the 1:1 scale.
Sure, you won’t be simulating a whole class of problems that can arise at scale, but you can still test the more localized features in a safe environment.
For large scale sites you need a parallel copy of the internet which doesn't exist. For example, we once caused a production incident (for us and other people) by simulating a major business event to our test stack and saturated links between <Cloud Provider> and <CDN Provider>. For future tests that was always an edge case that we couldn't fully test. Rinse and repeat those kinds of problems with every vender, service, and traffic source.
"Every company has a test environment. Some companies are disciplined enough to have a production environment separate from the test environment."
/s