Microsoft's fix for global outage isn't going 'as expected'
Microsoft says attempts to fix global outage not going 'as expected'
A worldwide Microsoft outage has knocked out Outlook, Teams and Copilot for more than ten hours, hitting users in 74 countries and disrupting government systems from Singapore to Japan. Microsoft blames "elevated load" on its infrastructure and says its fixes are only producing partial, unsustainable recovery. The company rules out a cyber-attack, though one expert warns that if it isn't one, it points to "incredible incompetence."
Microsoft is a massive part of the internet backbone and reminds us how critical email still is.
- kirtivr
FYI, if your business uses Microsoft 365, you are eligible to having 50% of your monthly fee waived off as per MS online services SLA.
It's been 18 hours since the start of the outage. If service isn't restored in the next 18 hours (36hrs+ outage), you can get the entire fee waived.
Microsoft gives a 99.9% monthly uptime guarantee, which means maximum allowed downtime is 7hrs.
- Eji1700
I’m extremely curious what actual veterans could even come up with to explain the situation as it currently appears.
I’ve seen the chats about cascading failures and it’s impossible to not expect AI is involved somehow (likely brain drain and what not), but what are the options?
It sure looks like “fired everyone who knew what they were doing” but I feel like at this point you’re breaking open the check book and calling them, and even if they did that I’m assuming this shouldn’t be a “normal” event?
- proc0
Looking forward to the postmortem on this one and how much AI contributed to the problem and troubleshooting.
I've been saying AI hasn't been tested properly. All the showcases I see aren't really pushing it far enough and they're always things you could in theory do by searching answers online. The real value is in all the problems that cannot be solved by online search.
- christhecaribou
This is one for the history books. The status updates seem like AI nonsense, it doesn’t seem like they’ve even root-caused what’s driving this.
- dwedge
I know this is only tenuously related, but after a week of them making changes (starting with DKIM) and blocking third party mailservers (and this week I got my first "your IP is blocked" but when you use their request to remove they say it isn't), I hope it goes down long enough to make more people migrate away