ValidAnytime
  • Try it free
  • Observatory
  • Pricing
  • Docs
Sign inStart free
ValidAnytime

A guaranteed false-alarm budget across your whole fleet, valid no matter how often you look.

Made by Compiled Intelligence — a frontier AI lab working on quantitative finance from first principles; ValidAnytime is the monitoring we built for our own model fleets, productized.

Product

  • How it works
  • Pricing
  • Observatory
  • Try it free
  • Dashboard

Solutions

  • Who it's for
  • For LLM engineers
  • For MLOps engineers
  • For SRE & platform
  • LLM eval regression
  • Model drift detection
  • Alert fatigue

Developers

  • Docs
  • Quickstart
  • Python SDK
  • API reference

Learn

  • Benchmark
  • Detector guides
  • Glossary
  • Compare
  • Why dashboards lie
  • Blog
  • Anytime-valid 101
  • Winter Storm Uri case study
  • Incident ledger
  • Observatory methodology

Company

  • Contact
  • Privacy
  • Terms

Observatory alarms & product updates, by email

Rare emails only. We store your address and the tag “footer” — nothing else. Unsubscribe anytime.

© 2026 ValidAnytime. All rights reserved.

Every alarm ships with its guarantee_tag and theorem_ref.

Observatory

Live · fleet quiet

Production model-monitoring, in public.

Every hour, each US power grid publishes a day-ahead demand forecast — and then the demand actually arrives. We monitor the error with the same anytime-valid engine we sell, and raise a page only when it genuinely changes regime, inside a stated false-alarm budget. It’s a real deployed model we neither built nor own — standing in for yours.

Monitor your metrics See how it worksBacktest your data
18
grids monitored, live
18
quiet right now
5.5 years
longest quiet streak
107
dated pages since 2021

updated Jul 6, 22:08 UTC

Live board

This isn’t a replay — it’s live.

Every card below is this run’s real state, straight from the artifact the pipeline rewrites on each poll. Green is the win: it means no regime change was detected on that grid’s forecast error — not that the grid is fine, just that the evidence hasn’t moved.

Fleet quiet — no regime change detected
updated Jul 6, 22:08 UTC
18
grids monitored
18
quiet right now
5.5 years
longest quiet streak
≈1
false page / grid-year (budget)

The budget is the page tier’s guarantee, not a wish: across all 18 grids the engine expects only about 18 false pages per year total. A quiet grid means no regime change was detected in its forecast error — never proof the forecast is fine.

  • US48
    United States (Lower 48)
    Quiet
    evidence8.48 / 9.08 alarm
    Quiet for 22 days
  • CISO
    California (CAISO)
    Quiet
    evidence6.79 / 9.08 alarm
    Quiet for 5.5 years
  • ERCO
    Texas (ERCOT)
    Quiet
    evidence6.02 / 9.08 alarm
    Quiet for 26 days
  • PJM
    PJM (Mid-Atlantic)
    Quiet
    evidence7.59 / 9.08 alarm
    Quiet for 5.5 years
  • MISO
    MISO (Midwest)
    Quiet
    evidence7.85 / 9.08 alarm
    Quiet for 7 months
  • ISNE
    New England (ISO-NE)
    Quiet
    evidence7.02 / 9.08 alarm
    Quiet for 5.5 years
  • NYIS
    New York (NYISO)
    Quiet
    evidence6.58 / 9.08 alarm
    Quiet for 11 months
  • SWPP
    Southwest Power Pool
    Quiet
    evidence6.99 / 9.08 alarm
    Quiet for 5 months

More grids monitored

  • Arizona Public Service
    AZPS · quiet 1.4 years
    6.6/9.1
  • Bonneville Power (Northwest)
    BPAT · quiet 6 months
    6.5/9.1
  • Duke Energy (Carolinas)
    DUK · quiet 5 months
    6.9/9.1
  • Florida Power & Light
    FPL · quiet 2.3 years
    6.4/9.1
  • Los Angeles (LADWP)
    LDWP · quiet 6 months
    6.4/9.1
  • Nevada Power (Las Vegas)
    NEVP · quiet 2.5 years
    7.1/9.1
  • PacifiCorp East (Utah/Wyoming)
    PACE · quiet 6 months
    7.4/9.1
  • Public Service Colorado (Xcel)
    PSCO · quiet 6 months
    7.2/9.1
  • Southern Company (Southeast)
    SOCO · quiet 1 day
    8.9/9.1
  • Tennessee Valley Authority
    TVA · quiet 5 months
    7.9/9.1

Every grid publishes an embeddable status badge that updates on each poll — open any grid to copy the snippet for your README, status page, or dashboard.

Why your dashboard cries wolf

Same stream. Three alarm policies. One you can trust.

A fixed threshold you check every hour is a coin you flip every hour. Left up long enough, noise alone will trip it — so the more often you look, the more it cries wolf. That is not a tuning bug; it is what fixed thresholds are. An anytime-valid e-process instead spends a stated false-alarm budget, so checking as often as you like cannot inflate its error rate.

Below is ~5 months of the Southwest Power Pool’s real day-ahead demand-forecast error — the central-US grid across Kansas, Oklahoma, Nebraska and the Dakotas — run through three alarm policies at the product’s default parameters. Same numbers into all three.

Winter Storm FernForecast error (MW) · a static 3σ thresholdAnytime-valid evidence · pages only inside a false-alarm budgetalarmSepOctNovDecJanFeb
27
pages from a static 3σ threshold
236
pages from CUSUM at product defaults
3
anytime-valid pages, inside budget

Checking hourly for ~5 months, on the same real stream: the static threshold and the sensitive CUSUM page dozens to hundreds of times — unbudgeted, mostly noise. The anytime-valid e-process pages 3 times, each at a genuine winter cold snap that pushed the forecast into a new regime, every one inside a budget of about one false page per grid-year. Sensitive is easy; the product question is how many pages you can ask a human to trust. We detect that the operator’s own forecast changed regime — not that the grid failed.

Watch it through Winter Storm Uri How the detector works

Ledger highlights

When it does page, the date lands on a real event.

The engine has written 107 dated pages across the fleet since 2021. The ones below overlap documented grid-stress events — the page fired when the operator’s own forecast error entered a new regime, not when the grid failed. Each links to that grid’s full history.

See all 107 pages
  • Jul 3, 2026
    Southern Company (Southeast)
    Late-June 2026 heat wave
    peak 9.4 / 9.08 alarm
    −6,896 MW error at onset
  • Jan 24, 2026
    Duke Energy (Carolinas)
    January–February 2026 cold wave (Winter Storm Fern)
    peak 11.5 / 9.08 alarm
    +5,188 MW error at onset
  • Jan 21, 2025
    Texas (ERCOT)
    January 2025 cold wave
    peak 10.6 / 9.08 alarm
    −7,285 MW error at onset
  • Jan 13, 2024
    Southwest Power Pool
    January 2024 Arctic outbreak
    peak 15.2 / 9.08 alarm
    +2,693 MW error at onset
  • Dec 25, 2022
    Texas (ERCOT)
    December 2022 cold (Winter Storm Elliott)
    peak 9.4 / 9.08 alarm
    −5,966 MW error at onset
  • Feb 10, 2021
    Texas (ERCOT)
    Winter Storm Uri
    peak 11.5 / 9.08 alarm
    +6,088 MW error at onset

Get notified when a grid's forecast changes regime

Rare emails only. We store your address and the tag “observatory” — nothing else. Unsubscribe anytime.

The full dated history lives in the ledger; the exact recipe (and what we don’t claim) is on the methodology page; or read the Winter Storm Uri case study. Feed: RSS.

The grid’s forecast is a stand-in for your model.

Same engine, same two-tier alarms, on your metrics — LLM-judge scores, drift, latency, any KPI. Connect a stream and reproduce a “caught it on day X” verdict on your own history, free, in minutes.

Benchmark
Frozen synthetic fleets, reproducible from seed.
The wall
Two alarm policies on one labeled synthetic fleet.
The Observatory
This page — the live, real model.
Start free Run it on your data