Apple was fine, but this poor old woman was at the genius bar trying to get some accounts connected. The genius at the Apple store was trying to explain the outage - and it just was NOT clicking for the woman. He must have said 4 variations of "Yes your accounts SHOULD connect, but Microsoft is having a major outage right now so they do not."
TSA is down too.
Do you really think that when these events happen, every single computer is patched at the SAME time ?
Does that sound intelligent ?
Does this happen in other industries ? Does the electric company shutdown every single transformer/station/big machine i dont know the name and perform servicing at the SAME time ?
Of course not, if they do they shouldnt.
If you had to reboot all of a distributed system to prevent propagation of something that keeps jumping around? Maybe.
Apple has literally no idea what they are doing with Linux and cloud.
They also use AWS and run servers in China using whatever cloud provider the Party uses
Like you’d expect a trillion dollar global tech enterprise to be, they run a ton of stuff geared toward whatever meets their business objectives
Yeah.
Such tactics mean a single compromised engineer laptop means a compromise of production, and thus a data breach is all but inevitable.
I would disqualify anyone seriously suggesting Salt, Puppet, Ansible or otherwise advocating for live-mutating production servers in a sysadmin interview today.
Custom operating systems are not scary. Make a UKI so your entire OS is just a kernel, init, and a target daemon payload. Treat servers like firmware. The less code in memory the better.
That said, I am working on an general purpose hardened server OS to be released soon publicly covered in my last talk https://lance.dev/talks/bsm2026
> Such tactics mean a single compromised engineer laptop means a compromise of production, and thus a data breach is all but inevitable.
I hope you don't believe that an immutable image cannot be compromised. The commits done to the build system come from the same engineer laptops and, in a world where CVE are published ao frequently, said compromised image could be deployed largely in only a few minutes hours.
Also there is something to be said with lack of debugging tools. While I agree that reducing the surface of attack is the way to go, I am pretty sure the reason it takes so long for some of the big infra outages we have experienced from various cloud vendors comes from the inability of engineers to log on some systems, observe and potentially fix temporarily by hand to stop a race condition. There is often a race condition and serpent eating its own tail situation involved and logs might not even reach the monitoring infra anymore.
Going back to manage pets is not the way to go but I am not entirely certain removing ability for a subset of engineers to log on systems is exactly the way to go.
Obviously, but I will trust a unikernel with just kernel/init and an unprivileged sandboxed daemon to have dramatically less attack surface than a redhat system. Reducing attack surface is the job.
> The commits done to the build system come from the same engineer laptops and, in a world where CVE are published ao frequently, said compromised image could be deployed largely in only a few minutes hours.
That is what signed commits and signed reviews are for, and a CI that mandates both. I mentioned quorum control and this is the method I always use. I founded the only Linux distribution that does this.
> Also there is something to be said with lack of debugging tools.
In some cases one may wish to expose a custom, unprivileged, and sandboxed ssh daemon that provides basic logging and read only diagnostic tools to admins. This is usually fine. Hardening and observability can co exist. No need for a real ssh and huge traditional desktop linux shell interface attack surface.
But if I want to do major debugging with dangerous tools in prod, which should almost never be required, it should have to be with consent of another team member to add a -new- image to the fleet with those debug tools, temporarily.
This would only be required when the only way to debug is with production secrets, but the default should always be to reproduce on a duplicate system with dummy secrets.
They don't.
Apple is making the same stupid mistakes Binance did. Binance was just more profitable to hack, for now.
That's not the entirely of their cloud exposure, however. A ton of their stuff runs in AWS, I wouldn't be surprised if they had things on other cloud providers as well.