Glossary
A flaky test is a test that alternates between passing and failing without any code changes. The same test, same code, same environment - different result each run.
Google found that 84% of pass-to-fail transitions in their CI are caused by flaky tests, not real bugs. Flaky tests erode trust in CI pipelines. Engineers stop believing red builds, re-run and hope, or disable checks entirely. The result: real bugs slip through because nobody trusts the signal.
Most teams re-run failing tests and merge if they pass on retry. Some quarantine flaky tests (removing them from the blocking gate). Few teams systematically track flake rate or assign owners for fixes. The most common root causes are timing/race conditions, non-deterministic test data, and third-party dependency changes.
Zerocheck records failure cause, run history, screenshots, recordings, and trend data so engineers can triage likely product regressions versus noisy failures without guessing from red/green alone.