wasl.tech

Microsoft 365 Outage Explained: The Hidden Bug That Took Down Outlook for 24 Hours

What Caused the Microsoft 365 Outage

Picture the scene: it is Thursday afternoon, and in offices across the world the same thing is happening at the same moment. Outlook spins. The inbox refuses to load. A meeting invite will not send, and the little circle next to the Wi-Fi icon keeps turning, as if the internet itself has forgotten how to work. For millions of people, that was the start of the Microsoft 365 outage — and for almost a full day, nobody knew why.

Now Microsoft has told us. In a preliminary incident report published on November 15, the company lays out exactly what went wrong — and the answer is stranger than a simple server crash. It was not one failure, but three, arriving together. A bug that had been hiding in Microsoft’s own systems since July, a sudden surge of traffic, and a chunk of missing server capacity. Any one of them alone would have been fine. Together, they took down the front door to Microsoft 365.

The incident began on Thursday, November 12, at 4:20 PM UTC, and Microsoft did not declare it over until Friday, November 13, at 4:10 PM UTC — nearly 24 hours later. Exchange Online was hit hardest, with most users unable to reach their mailboxes through Outlook on desktop, the web or mobile, as well as through Exchange ActiveSync and messaging APIs. Around 130,000 people reported Outlook problems on Downdetector at the peak, and the disruption spread across 74 countries. Parts of Teams, SharePoint, OneDrive, Copilot and Defender that rely on Exchange were affected too.

The explanation came from Charles Lamanna, Microsoft’s executive vice president of Copilot, Agents and Platform, who posted the findings on X, and from the preliminary Post-Incident Report for incident EX1550543, which Office-Watch has published in full. Microsoft had already apologised on November 13 and ruled out a cyberattack. Now the detail is here — and it reads like a slow-motion accident that nobody saw coming.

A Bug Hiding in Plain Sight Since July

The first factor was a latent defect — a sleeper bug. In July, Microsoft rolled out a Windows fix built specifically for its networking gateway servers. These are the “front door” machines that connections to Exchange Online and many other Microsoft 365 services pass through. The fix deployed safely over several weeks and then ran worldwide for more than three months without a single problem.

But the fix carried a hidden flaw: lock contention inside an internal Windows process. Under certain load conditions, parts of the system would end up waiting on each other, CPU and memory would be exhausted, and the server would simply stop responding. The flaw was worst on newer machines with more processor cores — exactly the kind of hardware Microsoft runs in production.

Microsoft’s own lab testing never caught it, because the labs used weaker hardware with fewer cores and a different mix of traffic. The bug only woke up on the big production machines, under real-world load. The defect — and the fix that caused it — apply only to the gateway servers, not to broader Windows deployments, so your own PC was never at risk. That is small comfort when the gateway in question is the one your email depends on.

Three Problems, One Bad Day

The second factor was a surge in load. About fifteen hours before the outage, at 1:07 AM UTC on Thursday, Teams traffic automatically failed over from Microsoft’s alternative networking gateway onto the primary one — the same gateway that also serves Exchange Online. It was a routine, automatic move, part of normal resilience operations. But it piled extra load onto the gateway and changed its traffic patterns.

The third factor was nearly 14 percent of the gateway’s server capacity sitting out of service, knocked out by an unrelated issue the day before. On its own, that would not have mattered — Microsoft says there was enough headroom for the service to cope. But combined with the extra Teams load, it pushed the gateway towards the hidden bug from July.

Then the dominoes fell. When servers started locking up, Microsoft’s traffic systems did what they are designed to do: they moved users to servers that were still healthy. Those servers then hit the same bug and locked up too. In under two hours, more than 80 percent of the gateway servers had stopped responding. The bulkheads that are supposed to stop a failure spreading from one region to another did not hold — a point Microsoft says it has not yet explained.

Why Recovery Took So Long

Getting the service back was harder than fixing a normal outage. Every time engineers restarted a server, the waiting traffic rushed back in and overwhelmed it again. The usual recovery tools — rollbacks, flighting and feature flags — were useless, because the faulty change was months old and long past the point where it could simply be undone.

Once engineers confirmed the defect, they built an update that returned the code to its earlier working state. From 7:20 AM UTC on Friday, they took servers out of service in large groups — more than half a region at a time — applied the fix, restarted them, and brought them back slowly. Webmail was back to normal worldwide by 2:17 PM UTC, and Microsoft did not declare the incident over until 4:10 PM UTC.

The good news for users is that email was not lost. Microsoft gave incoming messages extra time to be delivered instead of bouncing them back as undeliverable. Outgoing mail in desktop Outlook sat in the Outbox until the connection returned, though some messages sent from phone apps may have failed. If you sent something important on Thursday or Friday and never heard back, it is worth checking with the recipient.

For a fuller picture of what happened during those 24 hours, see our report from the day itself: Microsoft 365 Outage Knocks Out Outlook, Teams and Copilot for Millions.

What Microsoft Is Doing Next

The preliminary report is only the first word. Microsoft says a final Post-Incident Review will follow within five business days of the incident being resolved — expected by November 20 — with more detail on the long-term fixes. The company has already said it will explain why the regional bulkheads failed, which is the part of the story that still has no answer.

There is also an uncomfortable lesson in the report. Some services, like Teams, OneDrive and SharePoint, can “fail out” of the faulty gateway and keep running. Exchange Online cannot — for scale reasons, Microsoft says. That means the world’s biggest business email service has a single front door, and when that door jams, email stops for nearly everyone.

The timing adds salt to the wound. The outage struck just two days after Microsoft’s November Patch Tuesday, which delivered more than a thousand fixes, including an Outlook zero-day already under attack. Microsoft has not linked the two events — and the bug at the heart of this outage came from a gateway-specific fix, not a Windows update your PC would ever receive.

What Happens Next

For most users, life has already returned to normal. Services were fully restored on Saturday, and Microsoft’s service health page showed all products operational. But the questions will linger: how did a bug survive three months of global deployment and full lab testing, and what changes when the final report lands later this week?

The outage also lands in a week when Microsoft has been pushing hard on enterprise AI — including a new agent platform built with IBM. For now, though, the outage stands as a reminder of how much of modern work runs through a single doorway. One hidden bug, one routine failover, one unlucky capacity shortage — and the inbox of the world went dark. Microsoft has told us what happened. The final report will tell us whether it can happen again.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top