5 defectives rule of thumb

Question

I have taught six sigma black belt classes using consultant sourced training materials which included the rule of thumb that when estimating the rate of occurrence of discrete events, like the proportion of defective units produced by a manufacturing process, a sample size large enough to include at least 5 defective units is desired.

For example if the defect rate is 1 in a million, you must sample at least 5,000,000 units to obtain a reasonably accurate estimate of the defect rate.

I need an authoritative reference supporting this rule of thumb.

Answer 1

This reference on Lean Six Sigma, p. 159, provides a formula to calculate the minimum sample size, and mentions the "$5$ defectives" rule of thumb, a formula that relates to the normal distribution, and it is perhaps more useful to the OP than my contested (see comments) reasoning. I cannot argue about the reference's level of authority though.

Also this US government websource in section "Transforming Poisson Data" mentions this rule of thumb, relating it to the normal approximation to the Poisson distribution.

But I would like to offer a specific argument which is consistent with this rule of thumb (not necessarily an optimal argument -see comments):

One should clarify what "reasonably accurate" estimate means.Taking the road of Confidence Interval, we would want to have a point estimate whose variance/standard deviation is small enough so that the associated confidence interval won't include the value zero (and hence, negative possible values also, which in our case, would be non-sensical, and would also render the point estimate "statistically insignificant").

Associating each unit produced $i$ with a Bernoulli random variable $X_i$ that takes the value $1$ if the unit is defective and $0$ if it is not, and assuming that all units have the same probability of being defective, and that each random variable is independent from all others, then we can estimate this probability of defect as

$$\hat p =\frac 1n\sum_i^nX_i $$

or, writing $n_1$ to denote the number of defective units,

$$\hat p = \frac {n_1}{n},\;\; \operatorname{\hat Var}(\hat p) = \frac {\hat p(1-\hat p)}{n} = \frac {n_1(n-n_1)}{n^3}$$

Using the normal approximation to the binomial, a $90$% Confidence Interval then will be

$$\frac {n_1}{n} \pm (z_{0.05}+0.5){\sqrt {\frac {n_1(n-n_1)}{n^3}}}= \frac {n_1}{n}\pm 2.15\frac {\sqrt {(n_1/n)(n-n_1)}}{n}$$

where to the critical value from the standard normal distribution, we have added the $0.5$ "continuity correction".

We want the $CI$ to not include negative values, so we require

$$\frac {n_1}{n} -2.15\frac {\sqrt {(n_1/n)(n-n_1)}}{n} >0 \Rightarrow n_1^2 > (2.15)^2\cdot (n_1/n)(n-n_1)$$

Manipulating, we want

$$ n_1n > (2.15)^2\cdot (n-n_1) \Rightarrow n_1 > \frac {(2.15)^2n}{n+(2.15)^2}$$

For large $n$, as will the case be, the right-hand side tends to $(2.15)^2 = 4.62$. Since $n_1$ is an integer, $n_1 > 4.62 \Rightarrow n_1 =5$.

Answer 2

I think this comes from the rule of thumb for using the normal approximation for a confidence interval (cf. @AlecosPapadopoulos' answer). In short, it is recommended that [the smaller of] $np$ [or $n(1-p)$] be greater than $5$ for the normal approximation to be used. If this condition holds, it is often considered that the normal approximation can be used, and tests that implicitly rely on it can be used instead of exact tests*.

---

This does not guarantee that you will see a defect, but seems to work pretty well. We can determine the probability you will see a defect by using the binomial distribution's cumulative distribution function (CDF). Here I use R to do so:

1-pbinom(0, size=5e+01, prob=1e-01)  # [1] 0.9948462
1-pbinom(0, size=5e+02, prob=1e-02)  # [1] 0.9934295
1-pbinom(0, size=5e+03, prob=1e-03)  # [1] 0.9932789
1-pbinom(0, size=5e+04, prob=1e-04)  # [1] 0.9932637
1-pbinom(0, size=5e+05, prob=1e-05)  # [1] 0.9932622
1-pbinom(0, size=5e+06, prob=1e-06)  # [1] 0.9932621
1-pbinom(0, size=5e+07, prob=1e-07)  # [1] 0.9932621
1-pbinom(0, size=5e+08, prob=1e-08)  # [1] 0.9932621
1-pbinom(0, size=5e+09, prob=1e-09)  # [1] 0.9932621
1-pbinom(0, size=5e+10, prob=1e-10)  # [1] 0.9932621

Since we want the probability of getting anything but $0$ defects, we calculate the probability of getting exactly $0$ and subtract that from $1$. The analytical solutions are listed to the right. They seem to converge to $\approx 99.33\%$ as the defect probability decreases (and $N$ necessarily goes up).

---

Some meta commentary: A standard paradigm contrasts accuracy vs. precision. Because common statistical procedures privilege unbiasedness (cf. maximum likelihood vs. shrinkage estimators), the process of determining the appropriate sample size (power analysis) tends to focus on getting an acceptable level of precision. In your case, this seems to be whether an interval estimate (e.g., a 95% CI) will have an accurate level of coverage.

_{* Technically, since the rule of thumb is $>5$, and not $\ge 5$, this would be based on an analogy to the lower limit of the rule of thumb, but then rules of thumb by their nature shouldn't be taken as so exacting.}