Observatory
The Observatory points the same engine we sell at a real deployed model: each US grid’s day-ahead electricity-demand forecast. It decides a forecast’s error has changed regime by measurement, not vibes — and only when the accumulated evidence crosses a pre-committed false-alarm budget.
The US Energy Information Administration’s Hourly Electric Grid Monitor (EIA-930) publishes, for each balancing authority, both the operator’s day-ahead demand forecast and the actual demand that arrived. We monitor only the residual:
error = actual demand − day-ahead forecast
This is the trick that makes the board honest. Raw demand is dominated by a daily and seasonal cycle — monitoring it directly would just rediscover that it gets hot in the afternoon. Using the operator’s own forecast as the baseline cancels that structure for free, leaving a small, roughly stationary error — and a genuine change in that error is exactly what a model owner cares about. Because it’s public numeric data, there is no vantage point to defend: the number is identical no matter where we read it.
A split-window conformal calibrator learns each grid’s normal error band over a trailing 14-day window. A Shiryaev–Roberts e-detector then accumulates evidence whenever the error falls outside that band more often than it should, and raises a page when the log-evidence crosses ln(8760) ≈ 9.08. Because the statistic is anytime-valid, checking it every hour — forever — never inflates the false-alarm rate. That budget is the whole point: about one false page per grid-year. Across all 18 grids the fleet expects only about 18 false pages a year, by linearity.
A Shiryaev–Roberts detector will integrate even a fraction of a percent of chronic miscoverage into a runaway statistic over years. So each monitor is calibrated to its own grid’s normal forecast-error rate: it measures that grid’s baseline miss rate and bets only against a modest elevation above it, and detection runs on a bounded rolling window that resets each step. The live state you see on the board comes from the trailing window; the dated ledger comes from the full walk-forward. The result on real 2021–present data: no runaway anywhere, and grids with genuinely excellent forecasts stay honestly quiet for years.
An earlier version of this board probed vendor API latency from a single machine. That measurement reflected our own network path as much as any vendor, so a local hiccup could page several vendors at once — a false alarm we were measuring into existence. We removed it. Documenting and eliminating a false-alarm source in our own product is the whole ethos: a monitor is only worth trusting if it is honest about the one thing it must never do — cry wolf. Forecast error, read from public data, has no vantage point to get wrong.
Every grid exposes an embeddable SVG status badge at /observatory/badge/<grid> — for example /observatory/badge/ERCO. It reads no regime change while the forecast holds and forecast drift detected when the e-detector pages — the same honest boundary as everything above, and never “grid failure.” It updates on each poll and links back to that grid’s verdict page; copy-paste Markdown and HTML live on each grid’s page.
The dated history is in the ledger; the teaching walkthrough is on the main board, and the Winter Storm Uri case study shows it when reality broke the forecast.