Steve,
Now you made me go and think
Seriously, as you point out, there is a big danger of people over-interpreting "false positives" (specifically, being outside the 3 sigma limits) from control charts.
I have come to realize this evening that the odds of a false positive with an individual x chart can vary from about 10% down to exactly zero, depending on the distribution. The one advantage of having a normal distribution (or forcing a normal distribution by averaging seveal data points ala the central limit theorem) is that you have a pretty good idea of the odds of a false positive are close to 0.3%. That's not a great number, but it is at least it is known.
If the data is not normal, that throws the probabilities out the window. I can come up with distributions that have some basis in reality that will produce an individual x chart that vary anywhere from
* 10% of the data fall outside the 3 sigma limits (the worst-case Tchebychev limit)
* all of the data are within the 3 sigma limits (or even all of it withinin 1 sigma limits!)
The bottom line is that for individual x charts the odds of a "false positive" range anywhere from 10% (bad) down to exactly 0% (great). If people are worried about a 0.3% chance of false positive, imagine what they would think of a 10% chance! On the other hand, imagine the calm if they knew there was essentially 0% chance of a false positive.
I see the normal distribution as a happy medium. The odds of a false positive are not zero, but they aer sure to be small.
Tim
P.S. As I think a little more, averaging data makes bad distributions better (from 10% down toward 0.3%) and makes good distribution worse (from exactly 0% up toward 0.3%). I'm thinking there is a "best" number of data points to average - somewhere around three. The good distributions are still guaranteed to be within the 3 sigma limits, but the bad distributions have dropped close to 0.3%. But then, I haven't even thought about the range part of the charts!
Now you made me go and think
Seriously, as you point out, there is a big danger of people over-interpreting "false positives" (specifically, being outside the 3 sigma limits) from control charts.
If the data is not normal, that throws the probabilities out the window. I can come up with distributions that have some basis in reality that will produce an individual x chart that vary anywhere from
* 10% of the data fall outside the 3 sigma limits (the worst-case Tchebychev limit)
* all of the data are within the 3 sigma limits (or even all of it withinin 1 sigma limits!)
The bottom line is that for individual x charts the odds of a "false positive" range anywhere from 10% (bad) down to exactly 0% (great). If people are worried about a 0.3% chance of false positive, imagine what they would think of a 10% chance! On the other hand, imagine the calm if they knew there was essentially 0% chance of a false positive.
I see the normal distribution as a happy medium. The odds of a false positive are not zero, but they aer sure to be small.
Tim
P.S. As I think a little more, averaging data makes bad distributions better (from 10% down toward 0.3%) and makes good distribution worse (from exactly 0% up toward 0.3%). I'm thinking there is a "best" number of data points to average - somewhere around three. The good distributions are still guaranteed to be within the 3 sigma limits, but the bad distributions have dropped close to 0.3%. But then, I haven't even thought about the range part of the charts!