Zero failures. Now what?

You ran a batch of trials and not one failed. Tempting to call the failure rate zero — but a clean run is also exactly what you'd see if the rate were small but real, and you just got lucky. The rule of threeA 95% upper confidence bound: the highest failure rate that's still plausible after seeing zero failures. The true rate is below it 95 times out of 100. says the most a clean run of n can promise is a rate around 3 ⁄ n — no lower. The fewer the trials, the weaker the promise. (See any failures? Reach for Clopper–Pearson — the exact interval for the general k-of-n case.)

The clean run
We ranwith zero failures.
0.99%ceiling on the rateThe upper bound on the true failure rate at your chosen confidence. It is not the rate you measured — that was zero — it's the most the rate could still be, given the run was clean.

At 95% confidence the failure rate could still be as high as 0.99% — about 1 in 101 test. Zero failures rules out a large rate, never a small one.

0.99%exact ceiling
1 in 101worst plausible odds
3 ⁄ nrule of 3The shortcut: divide this constant by n for a near-exact ceiling. It's −ln(1−confidence) — about 3 at 95%, which is where the rule gets its name. ≈ 1%
The ceiling falls as 3 ⁄ n
45.1%
5101001,00010,00030010,000

clean trials (log scale) →

Clean testsRate could still be
10
25.9%
1 in 4
30
9.5%
1 in 11
100
3%
1 in 34
300now
0.99%
1 in 101
1,000
0.3%
1 in 334
3,000
0.1%
1 in 1,000
10,000
0.03%
1 in 3,000

To halve the ceiling you must double the run. Proving a rate is truly low takes a lot of clean trials — which is exactly why "we tested it and it worked" is such weak evidence for anything that has to be rare.

Going the other way
To be 95% sure the failure rate is under%, we'd need
600 clean tests

That's 300 more than the 300 you've run so far.

the rule of three — the napkin-maths answer to "we tested it and nothing broke, so how safe is it really?"