5

Statistical Process Control (SPC) can be used to determine if a process is "in statistical control". A common tool for SPC is the "mean control chart" -- essentially a time series of sample means obtained from the process one seeks to analyze.

A rule of thumb presented by John Oakland in his book Statistical Process Control is as follows. I quote:

A trend is a succession of points on the chart that are rising or falling, and may indicate gradual changes, such as tool wear. The rules concerning the detection of runs and trends are based on finding a series of seven points in a rising or falling trend (Figure 6.5), or in a run above or below the mean value (Figure 6.6). These are treated as out of control signals. The reason for choosing seven is associated with the risk of finding one point above the average, but [within 2 $\sigma$] being ca=0.475. The probability of finding seven point in such a series will be (0.475)7 = ca. 0.005. This indicates how a run or trend of seven has approximately the same probability of occurring as a point [3$\sigma$ away from the mean] (pg. 111, 7e)

The authors here motivate the discussion with use of a normal distribution, and so it stands to reason that all computations are performed with the normal. The authors include the following figure as illustration of a "trend"

enter image description here

The rationale behind this definition of "trend" is confusing. Oakland properly notes that

$$ \mathbf{\Phi}(2) - \mathbf{\Phi}(0) \approx 0.47 $$

However, the probability of observing such a sequence of observations is not 7*0.46>1, nor is it $0.46)^7$. The probability of observing a trend depends on the sequence of values observed.

Assuming the points are independent, then the probability of observing a trend $x_1 \gt x_2 \gt \cdots \gt x_7 $, where $x_i$ are iid, would be the probability I find $x_2<x_1$, multiplied by the probability that $x_3<x_2$, and so on. More compactly,

$$ \prod_{i=1}^6 \mathbf{\Phi}(x_i) $$

This product very clearly depends on how "steep" this trend is.

import numpy as np
from scipy.stats import norm

#Not so steep
trend = np.linspace(2,1,7)
np.prod(norm.cdf(trend))
>>>0.6094690320800531

#Very steep
trend = np.linspace(2,-2,6)
np.prod(norm.cdf(trend))
>>>0.0005112910660916415

Could someone more familiar with SCP justify to me why a trend of 7 points, regardless of where those points are observed, signals an out of control process. And more importantly, why do practitioners of SCP refer to these trends as "statistically significant"?

Demetri Pananos
  • 24,380
  • 1
  • 36
  • 94
  • 2
    +1 It's an awful quotation, isn't it? The fundamental concept is that of measuring decision errors in terms of the *expected rates* at which they will be made--that is, the mean length of an in-control series when the process is in control and the mean time before the control limits are hit when the process is out of control by a given tolerance (obvious analogs of statistical test size and power). The kinds of calculations suggested in this passage are useful heuristics but they are not the basis of any theoretically valid SCP procedure. – whuber Feb 21 '20 at 00:39
  • @whuber Truly awful. I'm tasked with helping an SCP group and I just can't understand where these come from. If you could clarify in an answer, even for a theoretically valid SCP procedure, I would gladly upvote and accept. – Demetri Pananos Feb 21 '20 at 00:41
  • 1
    If I get a chance I'll try, but the motivation behind the comment was I just wanted not to leave you in the lurch. I'm hopeful someone may come along with a more extended explanation or even a better perspective. – whuber Feb 21 '20 at 00:44
  • @whuber ok that’s fair. I appreciate you letting me know I’m not losing my mind. – Demetri Pananos Feb 21 '20 at 00:51

2 Answers2

1

I don't have Oakland's book, but I'd guess the text you quoted is meant to apply only to the discussion of runs (all above or below the mean) but not trends (all increasing or decreasing). I infer this based on the language of "risk of finding one point above the average..." in the quoted text.

Also, there's substantial disagreement amongst SPC people about what is useful in SPC, so I'd be hesitant to generalize anything beyond "use Rule 1." Wheeler, for example, considers trend-based rules as problematic and criticizes them in his Advanced SPC book, but also in this article:

https://www.spcpress.com/pdf/DJW322.Oct.17.Using%20Extra%20Detection%20Rules.pdf

"Recommendation: The best modern practice is to completely avoid using Nelson’s rule three and all other runs-up-and-down tests as well. Rule one will get there first in most cases."

If you really want to dig into why seven was chosen for the trend rule, you might have pull up Nelson's work from the 80s (see Wheeler's article), though I admittedly have never read those, so I can't be sure if he has theoretical explanations for the rules. Also, Nelson chose six instead of seven, so you'd have to figure out why Oakland decided to increase the number by 1.

Finally, my experience has been that experienced SPC practioners generally do not refer to anything as "statistically significant" in SPC applications, though that could be a function of those I've encountered. If a point is unusual, it's "out of control" not "statistically significant."

MichiganWater
  • 176
  • 2
  • 9
  • Nelson's article on this topic from 1984: https://asq.org/quality-resources/articles/column-technical-aids-the-shewhart-control-chart-tests-for-special-causes?id=5fd0ada4ca2a41b3a4d13e1439f47787 – Tavrock Feb 22 '20 at 04:06
  • The warning from my book by Montgomery is that if $k$ decision rules are used and that criterion $i$ has type I error probably $\alpha_i$, then the overall type I error based on all $k$ tests is $\alpha = 1-\Pi_{i=1}^{k}(1-\alpha_i)$ provided all rules are independent (which never happens, making this a best case scenario). – Tavrock Feb 22 '20 at 04:15
1

HERE is why a trend of 7 points, regardless of where those points are observed, signals an out of control process.

For an "in control" process, the order of occurrence of any given 7 data point values would be random, with all orders equally probable. Seven random points could occur in 7! = 5,040 possible orders. Only one of those orders would be successively increasing and only one would be decreasing. So for 7 successive points, your probability of a trend -- either increasing or decreasing -- would be 1/5040 x 2 = .0004 (approx).

Furthermore, if you restrict all 7 points to lie within +/- 2 sigma, then you need to multiply the .0004 x (0.95)^7 = .00028. (The 0.95 assumes Normal distribution.) This probability is certainly low enough to consider a 7-point trend as an "out of control" signal.

T. A. Wheeler
  • 105
  • 4