Moreover, "removing OOBGC reduced average response times by about 25%". CPU utilisation is a pretty terrible metric on its own; once you've paid for a CPU, you might as well run it at 100%!
But as a PM I have a different question. Which is the impact of the response time of Github on the user experience? 25% reduction on an average of 2 sec improves greatly the user experience, on a 200ms for a service that has basically no competition is just a marginal improvement, on a 50 ms average is not noticeable.
How is it distributed across multiple operations? Improving the latency of loading a diff is probably more important than the latency of approving a PR. In one case I'm trying to work, in the second I'm done with the work.
Now, my experience with GitHub is that it is already reasonably fast unless I'm doing something stupid. So the CPU utilization is a good metrics because 10% less CPU means 10% less server I need to pay and that goes to their bottom line (I don't know their economics to decide if it is substantial or not). The 25% reduction in latency is just icing on the cake...
Someone told me that the team at his previous job with a "large cloud provider" based in Seattle was told not to go above 30% CPU usage, at which point they'd buy more hardware. I'm sure not all teams there do things that way, but, coming from Google, 30% (95th percentile over a week) is very low.
consider a deployment (deployment app) update it will first create a new container and than kill the old one, which means depending how much memory you request at container creation is the least minimum you would have on spare.
if you schedule java apps with at least 1 Gi heap you would at least need 1Gi spare capacity and that is just with one pod/container (it will be worse if you need to do blue/green deploys, since you need the same capacity as your live cluster)
Even then, deployment updates don't necessarily need to surge above their replica count. You can also configure them to terminate X replicas at a time before bringing up new ones. At Google, all teams have Borg quotas, so it's not unusual to max those up by running as many replicas as possible. During updates, Borg does not allow an user to temporarily oversubscribe their quota (unless you're changing replica count and replica footprint at the same time, but that's another fun story), so it will always take down Y tasks first.