The cost of a browser test nobody trusts

A flaky test costs more than another CI run. Someone has to decide whether to investigate, retry or merge. When that decision becomes guesswork, the suite stops being a useful release signal.

I don't think you can put a general number on this, and I'm wary of anyone who does. An industry-wide percentage can't tell you which problems your team has. But you can get your own number in a couple of weeks.

Keep a short log

For each test failure over the next few releases, write down what happened. A table like this is enough:

Failure	What it was	Time spent	What fixed it
Checkout total wrong	Product bug	40 min	Code fix
Login timed out	Staging was down	15 min	Nothing, it recovered
Search returned no results	Shared test data	1 hour	Separate fixture per test
Save button not found	Test locator	30 min	Updated the test

Those rows are invented, but yours will look similar. Keep infrastructure errors, test-data problems and failed product assertions separate, because they have different fixes.

What the log tells you

If most of your time goes to shared test data, a new testing tool won't help much. Fix the fixtures. If it goes to locators after UI changes, better target handling might. If failures are real product bugs, the suite is doing its job, and the cost is worth paying.

The figure to watch is how often someone reruns without looking. Each of those reruns is a decision that the test isn't worth reading, and once a team has made that decision a few times, a real failure gets the same treatment. This guide has a process for working through them.

When you're comparing tools

Compare how quickly you can understand a failure, not just how quickly the first test goes green. That's what I'd want to know about Zerocheck too. It keeps the failing step, screenshots and console errors together, and a pass after a transient retry stays labelled flaky. The diagnosis is still yours.