Case study · Texas (ERCOT) · February 2021
Winter Storm Uri was the most catastrophic grid failure in modern US history. It is also a perfect, if grim, test of what an honest monitor does when a model’s world comes apart — because the failure is written plainly in one number: the gap between what the Texas grid’s day-ahead forecast expected and what actually happened.
Every hour, ERCOT publishes a day-ahead forecast of electricity demand. Normally it is very good — over this window the forecast tracked actual demand to within a couple of percent. We monitor only the residual: error = actual − forecast. A well-behaved forecast leaves a small, stationary error. Uri did not leave a small error.
As the storm hit and firm load was shed across Texas in rolling blackouts, the demand the grid could actually serve collapsed far below what the forecast had called for. On February 16, 2021, actual demand came in at 44.3 GW against a forecast of 73.1 GW — a gap of 28.9 GW. The forecast was not wrong because the model was bad; it was wrong because the world it was predicting had stopped existing.
~3 months of ERCOT’s forecast error. Top: a static 3σ threshold (red ticks mark each crossing). Bottom: the anytime-valid evidence statistic, which stays flat through normal noise and climbs across the alarm line only when the error genuinely leaves its calibrated band. Same detector families, product-default parameters; the classical tier is unbudgeted.
The anytime-valid e-process crossed its alarm line on February 11, 2021 and stayed up through the event — one budgeted page, inside a false-alarm rate of about one per grid-year. A static threshold, meanwhile, had already fired 2 times over the same window — most of them on ordinary winter noise, days before Uri, the kind of alert an on-call mutes by February. The difference is not sensitivity. Both can be made to fire. The difference is that only one of them tells you how often it will cry wolf when nothing is wrong.
What we claim is narrow and exact: the forecast’s error entered a new regime, dated, inside a stated budget. We do not claim the grid failed, why it failed, or what anyone should have done — that is the human’s job. The forecast is ERCOT’s own; actual demand may be revised by EIA. A monitor’s honesty is in staying inside exactly that boundary, especially when the story is this dramatic.
This is the whole pitch, on a real model, when it mattered most: an alarm you can trust because it comes with a budget — not one more line on a dashboard that has already cried wolf twenty times this month.