When your provider goes down
Last updated:
A provider status page is the public dashboard a cloud, hosting, code or payments provider maintains to declare its own health: one overall indicator, a list of components, and a timeline of incidents written by the provider itself while those incidents are still unfolding. Reading one well is an operator skill, because the headline is a summary and the truth lives underneath it — in which component is affected, in which region, at what declared impact, and in how far the incident has moved through the provider’s own workflow. This article teaches that reading, and ends with what to do in the first ten minutes when the provider in question is one you depend on. Start on the Provider Status Board, which collects the major providers’ pages in one snapshot, and keep the question from is it down, or is it just you? in mind throughout: the two tools answer opposite halves of the same doubt.
Two different things: the indicator and the incident state
Every mature status page carries two separate signals that readers constantly conflate. The first is the overall indicator: a single summarised state for the whole service, usually one of none, minor, major, critical or maintenance in the machine-readable feed behind the page — the model documented in the Statuspage developer documentation, which most large providers either use directly or imitate closely. The indicator is deliberately coarse. It exists so a busy reader — or a machine, like our status board — can answer “is something wrong over there?” in one second, and it is computed from the worst-affected component, the provider’s own judgement, or both.
The second signal is the incident itself, and it has its own lifecycle that the indicator flattens away. An incident is opened with a name, a declared impact and a status drawn from a workflow: investigating (we see symptoms and do not yet know the cause), identified (we know the cause and are working on it), monitoring (a fix is out and we are watching it hold) and resolved (we believe it is over, subject to the postmortem). These states are information the indicator cannot carry. A critical incident that has been in monitoring for two hours is usually ending; a minor incident that entered investigating four minutes ago may be about to become anything at all.
Read both, in that order, and read the updates between them. The timeline of short updates under an incident is where the provider tells you the things the headline never will: which subsystem failed, which workaround exists, whether data is affected or only availability, and whether the next update is promised in twenty minutes or two hours. An incident whose updates stop arriving is itself a signal — either the team is overwhelmed, which correlates with a longer outage, or the incident has quietly been folded into a bigger one. Operators who screenshot the headline and stop reading routinely act on the least informative sentence on the page.
Why “all systems operational” can still mean you are down
The indicator summarises the whole service, and modern services are not one thing. A provider can be honestly green globally while the region you deploy in, the one product surface you call, the legacy API version you are pinned to, or the control plane you need for a deploy is failing for every customer who touches it. Component lists exist precisely because of this: a page like GitHub’s status page breaks the service into Git operations, API requests, Actions, Pages and more, and any one of them can be degraded while the overall banner stays reassuring — or the reverse, with the banner red for a component you have never used.
There are three quieter gaps as well. First, declaration lag: a provider declares an incident when it has confirmed one, not when the first error occurred, so the first minutes of many outages show a green page and a burning social feed; the page is not lying, it is just behind. Second, thresholds: a provider may treat a two-percent error rate on one tier as normal background and never declare anything, while that two percent is concentrated on exactly your traffic pattern. Third, scope: status pages describe the provider’s side of the wire. A DNS failure, a routing problem or a blocked path between your users and a healthy provider produces a total outage for you under a green indicator — which is why the companion question, is it down or is it just you, has to be answered from independent vantage points, not from the provider’s own page.
The working rule is to treat green as “no declared incident”, never as “proven healthy”. When your own monitoring says users are failing and the provider says operational, believe your monitoring about your users and use the status page for a different job: checking whether a declaration appears later, which retroactively confirms the cause, and whether the component you depend on has a history of incidents shaped like the one you are living through.
Subscribe to components, not to pages
Most operators subscribe to a status page the way they subscribe to a newsletter: one subscription, every incident, forever — and then, predictably, they start ignoring the mail, because a global feed for a large provider produces a steady drip of incidents for components they do not use, in regions they do not serve. Alert fatigue is not a character flaw; it is the correct response to a badly scoped signal. The fix is mechanical and available on almost every mature status page: subscribe at component level. The same Statuspage model that defines the indicators also lets subscribers filter by component, and providers running their own systems, like Cloudflare’s status page, offer the same choice through their own subscription flows.
Scope each subscription to what would actually change your behaviour at 3 a.m. The components your production traffic calls belong on the loud channel — the one that pages, messages or at least badges something you will see within minutes. Components you use only for builds, analytics or internal tooling belong on a quiet channel you review in working hours. Components you do not use belong nowhere at all: every notification you correctly ignore makes the next one easier to ignore too, including the one that matters. Revisit the list whenever the stack changes, because subscriptions rot in both directions — new dependencies nobody subscribed to, and dead ones still paging a channel nobody watches.
Prefer machine-readable feeds where you can. Email is the slowest and most ignorable channel a status page offers; webhooks and feeds can drive your own dashboard, open an internal ticket, or flip your support team’s prepared holding page before the first user complaint arrives. Even a lightweight version pays: the snapshot on our Provider Status Board is built from exactly these public feeds, which is proof that the data is there to be consumed programmatically rather than admired in a browser tab you have to remember to open.
Build your own dependency inventory
None of the advice above works without a list, and most teams discover during an outage that they do not have one. A dependency inventory is simply the written answer to a question that sounds trivial and is not: which external services, if they stopped tonight, would stop us — and which would merely embarrass us? Build it in calm weather. Walk your architecture surface by surface — hosting, DNS, code hosting and CI, the database and storage providers, the payment processor, the email and SMS senders, the identity provider, the error tracker, the analytics tag, the fonts and scripts your pages load from someone else’s CDN — and write down, for each, the provider, the component you actually consume, where its status page lives, and what your users experience when it fails.
Rank each entry by the shape of its failure, not by how much you like the vendor. Some dependencies are hard: if they stop, you stop, and the only work available is communication and waiting. Some are soft: the site runs without them, degraded — no analytics, no chat widget, no personalisation — and for those, the inventory should say who is allowed to switch them off and how. And some are hidden: the transitive dependencies you did not choose, like the status of the cloud region your SaaS database itself runs in, or the certificate and DNS infrastructure underneath everything. Hidden dependencies are the ones that produce the “everything is down and nothing we pay for is listed” outage, and you find them by asking each vendor, once, what they depend on.
Keep the inventory where an incident can reach it: in the runbook, next to the escalation list, not in a wiki page nobody can open when the identity provider is the thing that is down. One page is enough. Its job during an outage is to turn “something is wrong” into a checklist — for each hard dependency, up or down, in thirty seconds — and its job afterwards is to be corrected, because every real incident teaches you about one dependency you had ranked wrong. An inventory that is never corrected is a historical document, not an operational one.
The first ten minutes of a provider outage
Minute zero to two: confirm and locate. Check your own monitoring to be sure the failure is real and yours to announce, then open the status board and the suspect provider’s page. You are answering one question: is there a declared incident that overlaps what you use — component, region, product? If yes, you have your cause and your first message. If no, widen the check: another provider in your inventory, your own last deploy, your DNS. Most false dramas die in these two minutes, and most real ones get their shape here.
Minutes two to five: communicate, plainly and early. Tell your users, in the channels they actually read, that a named upstream provider is having an incident, what it means for them in practical terms — what does not work, what still works, whether their data is safe — and when you will update them next. An honest holding message sent at minute four beats a precise explanation sent at minute forty, because silence is what users fill with their worst guesses and your competitors’ names. Internally, open one incident thread, appoint one person to speak, and timestamp everything; you will thank yourself at the postmortem, and so will the provider’s support team if you need them.
Minutes five to ten: stop making it worse, and prepare the soft landings. Freeze deploys — a change made into an active upstream incident cannot be evaluated and may not even be possible to roll back. Turn off aggressive retries and non-essential jobs that hammer the failing provider; recovery is measurably slower when ten thousand clients retry in lockstep, and some providers will rate-limit you into a second, self-inflicted outage. Switch off or degrade the soft dependencies your inventory marked as sheddable, post the holding page if you have one, and then settle into the rhythm the incident dictates: monitor the provider’s updates, update your users at the promised times whether or not there is news, and change nothing else unless the facts change. The outage will end on the provider’s schedule, not yours; your ten minutes decide whether your users remember it as their provider’s bad day or as yours.