ValidAnytime
  • Try it free
  • Observatory
  • Pricing
  • Docs
Sign inStart free
ValidAnytime

A guaranteed false-alarm budget across your whole fleet, valid no matter how often you look.

Made by Compiled Intelligence — a frontier AI lab working on quantitative finance from first principles; ValidAnytime is the monitoring we built for our own model fleets, productized.

Product

  • How it works
  • Pricing
  • Observatory
  • Try it free
  • Dashboard

Solutions

  • Who it's for
  • For LLM engineers
  • For MLOps engineers
  • For SRE & platform
  • LLM eval regression
  • Model drift detection
  • Alert fatigue

Developers

  • Docs
  • Quickstart
  • Python SDK
  • API reference

Learn

  • Benchmark
  • Detector guides
  • Glossary
  • Compare
  • Why dashboards lie
  • Blog
  • Anytime-valid 101
  • Winter Storm Uri case study
  • Incident ledger
  • Observatory methodology

Company

  • Contact
  • Privacy
  • Terms

Observatory alarms & product updates, by email

Rare emails only. We store your address and the tag “footer” — nothing else. Unsubscribe anytime.

© 2026 ValidAnytime. All rights reserved.

Every alarm ships with its guarantee_tag and theorem_ref.

Glossary

Test martingale

A test martingale is a running evidence score that, if nothing has changed, is not expected to grow — the honest core of an e-process.

Also known as: martingale / test supermartingale

A test martingale is a running evidence score that starts at one and, under the assumption that nothing has changed, has no upward drift — it can wander, but it is not expected to climb. The underlying idea is the martingale, the mathematical shape of a fair game: whatever has happened so far, your expected next value is exactly where you stand now. So if a test martingale does climb and keeps climbing, that is genuine evidence something shifted.

This is what makes an alarm trustworthy instead of trigger-happy. Because a test martingale cannot be expected to grow when all is well, a large value is hard to produce by luck — and exactly how hard is pinned down by Ville's inequality.

Every e-process is, at heart, a test martingale (or a supermartingale, the one-sided version). It is the object that turns 'this metric feels off' into a number you can bound and act on.

Go deeper

  • The math behind the demo (docs)
  • The e-detector — guide and in-browser playground

Related terms

  • E-processAn e-process is a running score of evidence against 'nothing has changed'; its value at any moment is an e-value, and it stays valid at every look.
  • Ville's inequalityVille's inequality is the theorem that makes 'valid no matter how often you look' true rather than wishful.
  • Anytime-valid inferenceAnytime-valid inference is a way of testing that stays statistically valid no matter how often you look at the results.
  • Sequential testingSequential testing is the practice of testing a hypothesis as data arrives, deciding to stop as soon as the evidence is conclusive.

Put the theory to work.

ValidAnytime turns these ideas into a live alarm you can trust — valid no matter how often you look. Prove it on your own data, free.

Start freeTry the detector