Global Microsoft Outage
cnn.com
cnn.com
Along with the daily Github outages recently, it's making me do an audit of my dependence on Microsoft.
Of course I haven't received a single email because every important email is sent via Microsoft but I still feel superior because I'm not affected
FreeBSD(ZFS+jails old but gold) + postfix + dovecot.
And yes uptime (with short outages(updates etc)) is not that important for a mail-server, since the sender re-transfers in intervals (mine is 5min the 8 then 15 then 60 then 180 then something up to 2days)
So the uptime for the service is 100% (maybe 99.999) over the last 8 years.
and
https://mecsa.jrc.ec.europa.eu/ (offline atm Azure??)
True, i have some hetzner (not vm's but metal), and Vultr (just vm's), i just had one bad IP until now (i just changed it instead of trying to get it removed from a blacklist)
It's not, it's easy. We've been doing this stuff for years.
Doing it at billion user scale is hard. You are not google.
root@mail ~ uptime
11:57:30 up 2114 days, 4:36, 2 users, load average: 0.20, 0.26, 0.61Sending out is harder but that 1% when I send something I need it delivered.
Hosting e-mail for friends and family is harder and mostly what I would call stupid.
I guess software is still not solved, I'll let Dario know
Because "personal outlook" at https://outlook.live.com/mail has been working this whole time.
Because I know people here are devs but you guys dont know IT lmao
365 that is
Peronal outlook has been fine
I'd guess that almost every CEO affected asks themselves whether they should be so reliant on Microsoft.
1.5% is basically random intra-day fluctuations.
I hope this sort of stuff pushes us to a distributed model rather than a centralised mono-provider one.
So again rationally this shouldn't affect the stock price much.
However repeated disruptive outages every few weeks/months (à la GitHub) is a different story, and there's also the invisible effect of businesses maybe being less likely to choose Microsoft in the future
A week long outage though , yeah at that point the decisions becomes a nobrainer and not even network effects will prevent the migration.
Switching to what though? It's not like we didn't have outages before SaaS. It just happened at different times throughout the year in stead of all having an outage at the same time.
I feel like the big difference right now is that management can't call the EoD for progress updates every 10 minutes, they just have to assume that someone at Microsoft is working on fixing the issue.
So yes that's another argument to not do anything and stick with Microsoft unless things become terrible
[1] ...but LLMs have the potential to change that for the worse
Rather smug with my separate SMTP/IMAP provider now. And hourly offline backups of the mailboxes.
IMHO a bigger concern is that Russia hacked Microsoft in 2022 and China hacked Microsoft in 2023. Even that wasn't enough to scare companies off MS.
Basically, MS customers are locked in, the cost of migrating to anything else is huge, and there's a "no-one gets fired for buying MS" herd mentality, so MS has little to lose no matter how hard their product fails, just like Cloudstrike.
There's also the question of what alternative would really be better, or at least more reliable. My impression is that Google's infrastructure and engineering are genuinely better, but I'm recently ex-Google so take that with a grain of salt.
We've had more downtime from all our SaaS and cloud services in the last year than the 20 years before when self hosting.
Now it feels like something is always down.
Email is not a particularly easy thing to reliably host and secure for a sizable organisation, especially if its not your main focus.
If I lose a whole day, we lose all the productivity for that day and delay countless upstream and downstream efforts by a day.
The distribution of those minutes matters.
I suppose "unplanned outage minutes" would be a better metric. I still expect most companies to be worse on that metric. Most companies are just bad at running services reliably, especially when it's not their core business.
The fact that the government can get taken down by a central server is mental.
Is it some sort of spying thing? Incompetence?
It’s Microsoft we’re talking about so I feel like the latter is a given, and the former is a reasonable assumption.
Jokes aside, it does seem that somehow these hyperscaler cloud providers who tout the reliability of their multiple zones and global distribution sure do seem to always have some central dependency or single point of failure. AWS for example seems to have a global outage once every year or two at this point, or like us-east-1 problems always seem to affect other zones too. You’d think with their scale these clouds would be completely decentralized and redundant but apparently not.
With a linux fleet can run via SSH/ansible and you can patch thousands of machines programmatically in minutes and control the deploy at a fine level. With Microsoft/Windows it often involves/d logging in to the one AD server where everything runs, back it up via 3rd party software, apply the patches and hope it doesn't break, and if it does, call MS support, because rolling back is really your last resort (I feel it's become better since 10y ago, but I don't have hands-on experience anymore).
Our company runs on-prem Exchange since 2019, and... well... it's a PITA to work with (Legacy Outlook/New Outlook), we downgraded from M365 and it was really a step down in terms of features and reliability. OTOH I guess it's keeping us afloat today when M365 is having a really bad day.
You can run your own email easily and cheaply, but if you wan to run with a team big enough and experienced enough to recover quickly if there's a problem it gets expensive fast.
Google Workspace isn't going to be any different.
LLM generated code and/or configs?
Malice or gross incompetence isn't really required to break it, that's its natural state. You need high levels of competence to get it to work
The timing is suspicious but could indeed be a coincidence.
No need for conspiracy here.
Yesterday Outlook. Today FAA ground stop. Tomorrow ... well we'll see.
one of the all time tech gossip blog posts
Second-order effects of this are immense.
Curious how such a worldwide issue can manifest, considering Microsoft usually has some pretty good geo availability mirroring?
It's also got two stories on the BBC Home Page, including one which is live reporting.
Last Error: 451 4.7.0 Temporary server error. Please try again later. PRX3 NextHop: substrate-int.office.com. Last IP: [MI3PEPF00008491.eurprd03.prod.outlook.com 2026-11-13T10:15:37.721Z
Last Error: 421 4.3.2 The maximum number of concurrent server connections has exceeded a limit, closing transmission channel [MI3PEPF00008487.eurprd02.prod.outlook.com 2026-11-13T10:42:55.252Z ]
We're also seeing messages about alternate failover failing.
As long as it works its good for them.
Much of the productivity gain from this AI boom came from shuffling off the expensive emphasis on robustness and reliability, and this downtime is just one of the externalities of this increase in developer productivity.
As evidenced by Crowdstrike stock quadrupling since it took down half the internet in 2024, vanishingly few people actually care about downtime like this.
Meanwhile, there is decades-long familiarity with Microsoft's product offerings which makes for generational vendor lock in.
I think this is just a new normal. Genuinely speaking, what percentage of businesses will de-Microsoft in the next year over this? I don't think it would even be 2%.
You can reduce the blast radius of outages at the cost of increasing the frequency a lot.
worked on a service that plays a role here. I spent all my time working on infrastructure availability (ON CALL). only to be told during rewards that keeping livesite available is not impact and 'expected'
fucking idiots.
How feasible would it be to quickly stand up an SMTP server and point your MX to it and get some mail while the outage is ongoing?
- No emails, calendar, file sharing, etc. - Claude Code broke as well because I cannot login back. - Multiple services I use with 365 are also broken.
We will also discuss and see how to failure proof our infrastructure against Microsoft outages. We made the mistake to base our VPN 2factor on Email. So now no VPN for anyone.
Truly next-level corporate speak.
> ...
> Newsweek contacted Microsoft for comment via email.
I hope that was biting sarcasm so that those of us who are getting up with restored emails can have something small to smile about.
Thank you!
first - were the updates fully tested in a legitimate test environment? (robust mirror with proper configuration and data) or were they "modeled" on a digital twin?
second - did any canary deployment precede the full rollout which seems to have a performance/capacity issue that borks (highly technical term) servers under load?
third - regardless of the answers for the first two, given their release quality history, should someone be suggesting "hey, this could be an impactful change, we should roll it during a quiet period like 2am Sunday" to mitigate customer impact and provide a longer restoration window option?
or has Microsoft simply asked OpenAI to manage it all...?
The future demands small, stable core services, and mail service should be a small core service commodity that has been defined and solved (protocol wise) a long time ago already - we only need to make it an accessible commodity for all without the big tech entanglement.
Two decades ago I was at a startup that moved from a Linux email solution to a Microsoft one. Why? For the CEO's blackberry and calendar integration.
There are loads of FOSS email solutions and providers. There are fewer "email+calendar" providers, and there are even fewer "email+calendar+desktop logon SSO" providers.