The switch was fine. The cut-off wasn't.

What actually happens

The outages you plan for are not the ones that cost you the most. The expensive ones are the afternoons where every dashboard stays green.

Authorisation latency sits at 240ms most days. At 13:00 it is 290ms. By 15:00 it is 430ms. By 17:00 it is edging past 800ms. No alert fires at any point, because every threshold on the switch is set for an outage, and this is not an outage. It is a slope.

Each individual reading looks defensible. A latency of 430ms is slow but it is not broken, and the person looking at the graph at 15:00 has no reason to treat it as the middle of a trend rather than a blip. The signal is only obvious once you can see the whole afternoon at once, and by then the afternoon is gone.

The thing actually at risk is not uptime. It is the settlement cut-off. Transactions that queue behind a slow switch do not disappear, they carry into the next value date. That is a reconciliation problem, a treasury problem and, if the volume is large enough, a reportable one.

The cause, when it is eventually found, is usually unremarkable: a connection pool that never reclaimed after a morning failover, a certificate revocation check that started timing out, an index that went stale after an overnight load. None of these announce themselves. All of them show up as a slope.

What makes this pattern expensive is that the evidence needed to diagnose it is scattered across the switch, the host, the database and the network path, and no single team owns all four. The investigation begins with an argument about whose graph is wrong.

A threshold answers the question "is it broken". Nobody set a threshold for the question that actually matters here, which is "will this still clear by cut-off".

The same afternoon, two ways

The argument for detecting drift is not about a percentage. It is about which side of the cut-off you are on when someone finally looks.

The value in this scenario is not a faster repair. Recycling a switch node takes minutes in either column. The value is that the problem was named at 13:47 instead of 18:50.

That gap exists because threshold monitoring is built to answer a binary question, and slow degradation is not binary. A system can be entirely within every configured limit and still be on a path that misses a deadline.

Sentinel baselines behaviour rather than watching for a line to be crossed. A metric moving steadily away from its own normal pattern is treated as a signal in its own right, and the investigation that follows carries the projection with it: not just "this is rising" but "at this rate, here is what it threatens and when".

You can sanity check this against your own estate without us. Take your last six months of settlement exceptions and ask how many started with an alert. In most estates the answer is uncomfortable, because the drift cases never generated one.

Who decides to press go

Recycling a node on a live payments switch is not a low-risk action, so the decision about who presses go is made on the impact of the fix, not on how confident the diagnosis is.

The Action Ticket executes under policy: pre-check, execute, post-check, verify, with automatic rollback if the post-check fails. Nobody is paged.

The ticket is raised and held. The payments duty manager and change stakeholders receive the proposed MOP, the blast radius and the rollback path. Nothing runs until a named human approves it.

The thirteen minutes between the ticket being raised and the recycle running is a human deciding. That is the design working, not latency in the system.

Authorisation latency rising monotonically against its own baseline for 40 minutes. No configured threshold crossed. Queue depth beginning to build on one node.

Switch nodes, connection pools, database wait events and network path correlated. One node holding stale pooled connections since the morning failover. Two comparable prior incidents surfaced.

Action Ticket raised with cut-off projection attached. Recycle MOP staged with drain, rollback and post-check, held for the payments duty manager to approve.

Pool reclamation check added to the post-failover MOP. Latency slope promoted to a first-class signal alongside absolute latency. Cut-off projection added to the payments watch.

This is a platform capability, not a published customer deployment. The mechanism, which is correlated investigation followed by governed MOP execution with pre-check, post-check, rollback and approval gating, is running in production today in our carrier estates; see governed day-2 operations across 2,000+ nodes and closed-loop network automation. The scenario above is that same mechanism applied to a banking context. The timings shown are modelled, not measured at a named bank.