How often do identities or access settings change? Not often, I guess. So it seems (to me) that they are accessing almost static data at a very high speed, which is not really a surprise. Am I missing something?
If you're using caches, you're deliberately sending old data to the replicas. I can't think of why you'd do that. I guess you could have replicas of your caches, but I'm referring specifically to data store replicas.
A cache stores the output of an operation to avoid needing to perform the same operation again. You deliberately put data into a cache to avoid needing to create that data a second time—it won't magically appear there.
But perhaps the permission updates are slow. Like, you change a permission, wait 5 seconds, and the system tells you that they are updated across all servers.
The difference between mostly static and completely static is where the interesting bit is. The two categories beget completely different solutions, especially when serving stale data is not acceptable, as in the case we're looking at.
I suppose this comment falls in the same category as "never underestimate the bandwidth of a truck full of tapes".
Yes, the numbers are huge. Whether it is a technical achievement is another question entirely.
This is a globally distributed service operating at massive throughput and ultra low latency scaling requirements where if it fails millions of customers and a majority of online commerce as a whole comes to a halt. What you’re referring to is just the very tip of the iceberg.
The hard problems stem from how the system deals with failures and how the system propagates writes across the replicas while meeting latency and consistency SLAs. On top of that the system needs to be built in a way that it can be maintained by many developers each working on a small piece of the system without knowing the ins and outs of the system as a whole. In addition, when the system fails debugging and mitigation needs to be able to be parallelized across many developers so that availability SLAs can be maintained. You can read about this in “Designing Data-Intensive Applications” by Martin Kleppman where he discusses the complexity involved in building distributed systems.
Used to work on 20M+ user identity service that didn’t see volume anything like this, but when utilization went up usually it was just a matter of adding more replica databases and auth servers.
The load isn't the only issue either you also have availability SLAs you need to meet that force you to design the system to tolerate faults at that scale or else half the internet gets taken down. Your system has to be broken up at a granularity where many engineers can work in parallel to diagnose and mitigate an issue when your service does get taken down to minimize downtime. “Scale breaks everything” as they say.
That's the point: nobody said permission updates have to happen within a second. They are performed infrequently, so the system can afford to make them slow.
https://aws.amazon.com/iam/faqs/
P90 values provided to us in 2017 (large enterprise considering AWS) was under 1 second.
Thought experiment: if it suddenly took 10 seconds to respond to the calls, the number of responses still equals the number of requests, so the request velocity mentioned in the headline remains the same.