Use case · Model drift detection
A KS or PSI test run every day will flag a healthy feature eventually; that's not drift, that's the peeking problem. ValidAnytime detects data and concept drift with anytime-valid e-detectors: check continuously, get one alarm per real shift, and prove it on your own history before it ever goes live.
Free to start · no credit card
Your input distribution creeps — not a cliff, a slow slide over weeks. Your PSI monitor at 0.2 has been firing on ordinary daily wobble for a month, so the channel is muted. The real shift, when it comes, looks just like the noise you've been ignoring — until precision drops and the business notices. You didn't lack a monitor. You lacked one you could trust.
PSI, KS, chi-squared, JS and Hellinger distances all assume a fixed sample and a single check. Run them repeatedly and false alarms are guaranteed on stable data — check a fixed 5%-error test 5 times and the real false-positive rate is already ~23%; at 20 looks it passes 64%. Per-feature thresholds mean there's no bound on how many of the fleet's alarms are false. The tests are fine; checking them continuously is what breaks the guarantee.
PSI, prediction distribution, a feature statistic, a calibration or conformal score, or labeled error — send the number, not your raw rows.
Auto-config replays your data and shows whether, and on which day, it would have caught your last drift — and stays quiet on the stable months. A config that false-alarms on normal history never ships.
ARL-calibrated Shiryaev–Roberts (SR) e-detectors accumulate evidence; online FDR (e-LOND) spends one false-discovery budget across every model and feature.
Load a drifting metric series — accuracy, PSI, a feature mean — and watch the detector find the exact point it would have alarmed, with nothing before it. Runs locally; nothing uploaded. The same engine runs live and continuously in our Observatory.
Model accuracy — bad retrain, slow drift
Synthetic, illustrative — runs in your browser, nothing uploaded.
From the seeded demo — synthetic stream, real engine. Load it in the live dashboard →
NannyML's label-free performance estimation (CBPE/DLE) and Evidently's test breadth are real strengths — keep them. Feed their metric outputs to ValidAnytime and get the one thing they don't provide: an alarm valid under continuous checking, with a certificate and a fleet-wide budget.
Point ValidAnytime at one stream and it replays your own history — free, no credit card.