Microsoft 365 and Outlook Still Down After 12 Hours as Servers Fail Health Checks
Microsoft Outlook/365 still down after 12 hours
Microsoft has spent over 12 hours fighting an outage that knocked out Exchange Online and Microsoft 365 connectivity across multiple regions. Servers failed automated health checks and dropped offline, straining the rest of the infrastructure. Microsoft paused all deployments, restarted affected servers, and validated an alternate health-check method, but returned servers keep going unresponsive once they take traffic. A configuration change meant to improve serviceability did not work as expected.
The servers we returned to service can become unresponsive again once they receive traffic; we're analyzing diagnostic data to determine what's triggering the behavior.
- nickdothutton
Microsoft is unable to run a reliable hosted email service, given infinite resources. What people don't realise is that the former is in part a consequence of the latter. Without a forcing function, complexity multiplies, silent fragility accumulates, end-to-end understanding dwindles. There is no incentive to simplify.
- magnio
Apparently this single outage means Office 365 uptime this year drops to 3 nines (2 nines for this month). You can claim SLA credit worth 50% service fee by applying before December 31.
- vld_chk
This is arguably one of the biggest outages in modern history. AWS region level outage was more visible, but email is a backbone of the modern world.
BBC is running live coverage of it at the level of government and ministers, feels surreal
- BLKNSLVR
How many Microsoft employees are working on this? It's concerning that one of the biggest, most valuable companies on the planet, that runs services internationally, can have one of its primary services have gone down for this long.
6 hours ago they seemed to not exactly understand what the problem was, and that was 6 hours into the outage.
This is concerning.
- daitangio
Is for this reason that my email is hosted on a cheap VPS:
https://gioorgi.com/2020/mail-server-on-docker/
https://github.com/docker-mailserver/docker-mailserver
It is very easy to set up (just a bit of time on average every 6 month to follow upgrades) and work out of the box.
Europe outlook users are experiencing problems too.
- pitabas106
I have always felt it's less about email redundancy and more about having a backup plan. If outlook goes down tomorrow, does everyone know how to keep work and client communication moving? that is something worth thinking about.
- enopod_
Good morning and happy Friday the 13th everyone
- eddythompson80
Having friends who work at Microsoft, this is not really surprising at all. The company has always had deep structural problems. The AI change has really just exposed it and is aggressively accelerating the inevitable implosion. It's truly in a free fall that is planned and orchestrated perfectly by their upper management. Their CEO and VPs are convinced you don't need engineering know-hows any more because of AI, and they got all the management layers to sincerely agree with them. Not in a malicious or subverting way, but in a "you can't get a man to understand something his salary depends on him not understanding it" way.
Microsoft has a moat, but their moat is having billions of dollars to spend on data centers. It's a failure everywhere else. It'll continue to rake in billions in revenue because people need compute now, but it's rotten at the top.
- MisterMunchkin
This is going to cause absolute chaos. Basically every government organisation and business in the UK runs on outlook. The whole day is going to be a write off. It’s like a bank holiday!
- karlkloss
Why would anyone introduce a single point of failure out of their control into their company?
We're doing anything to avoid single points of failure in our products, but our IT is doing the opposite.