# Profound Statistical Concepts 2019-10-31

#### Bev D

##### Heretical Statistician
Staff member
Super Moderator
Bev D submitted a new resource:

Profound Statistical Concepts - What to do when theory collides with reality: Normality, homogeneity, independence and p values

Too often quality practitioners and other engineers and scientists ignore the fundamental requirements of a study design that enable the use of p values and the null hypothesis approach to statistical studies. This renders the conclusions of the study unreliable at worst, at best it leads to confusion regarding what conclusions can be made.

Two case studies are presented that demonstrate the role of non-homogeneity in study designs. An analytic study approach is explained that makes...

#### Watchcat

Trusted Information Resource
There is ignoring and there is ignorance. I think a lot of people using statistics don't even know what they are.

I was also taught that different statistical tests are more robust than others, where "robust" means, as I recall, that the results are less likely to be affected by violations of the underlying assumptions?

#### Bev D

##### Heretical Statistician
Staff member
Super Moderator
‘Robust’ tests are only robust to deviations from Normality. No statistical or probabilistic test is robust against the other much more important requirements.
Homogeneity (independence and replication) is paramount. It’s the study design that matters not which option you click in your statistical software.

Last edited:

#### Watchcat

Trusted Information Resource

They do statistics with software now?

#### Steve Prevette

##### Deming Disciple
Staff member
Super Moderator
I am teaching a stats course for Southern Illinois University at Carbondale. Of course I am covering tests of hypothesis, but with caution. Even the textbook has some sidebar discussions of certain scientific journals moving away from P-values.

If you do look at the issue of Enumerative vs Analytic studies (there is a nice video from Michael Tveite that I am making the students watch) that goes through Dr. Demings concerns in that regards, and the preference towards Statistical Process Control.

Granted, one needs to understand the context of the data and the underlying process(es) to avoid costly misinterpretations.

#### Watchcat

Trusted Information Resource
I don't know what link(s) there might be between p-values, Deming, quality, and SPC. My training in p-values was as a scientist. To me it's all about generalizing results obtained from a sample to a population.

I consider Quality to be one of the best ideas that ever emerged from the human race. Sadly, its time came and went, and it never caught on. I'm only generally aware of SPC.

From that perspective:

certain scientific journals moving away from P-values
I've been hearing this since at least the 80s, always articulated with the exact same words..."moving away from." There is no good reason to do this. However, if the journal's readership has no understanding of p-values, nothing lost, I guess. I do understand p-values, so I would never read a journal that had "moved away from" them. Journals are bad enough as it is. Further reducing the amount of information reported in them doesn't appeal to me.

Last edited:

#### Jim Wynne

Staff member
The general problem with statistics in manufacturing is captured in the well-known (but of obscure origins) quote, "He uses statistics like a drunk man uses a light pole--for support rather than illumination."

Another pervasive problem is that QA is the only discipline I know of where people of very diverse levels of education and intellectual development are expected to understand and invoke complex mathematical theory. It becomes even more absurd when the people who establish requirements for statistical analysis so often don't understand even relatively simple concepts.

#### Watchcat

Trusted Information Resource
QA is the only discipline I know of where people of very diverse levels of education and intellectual development are expected to understand and invoke complex mathematical theory. It becomes even more absurd when the people who establish requirements for statistical analysis so often don't understand even relatively simple concepts.
I think the situation is similar in "science."

Because there are multiple levels of definitions of "science," people with very diverse levels of education and intellectual development can legitimately call themselves "scientists," and many do. Unfortunately, many of these "scientists" feel the need to conduct scientific research, even though they may lack the education and/or the intellect to do so. They are a "scientist" and therefore it seems that any research they do is, by definition, "scientific" research. In many cases, I'm not sure an understanding of statistics and experimental design is even expected.

#### Ronen E

##### Problem Solver
Staff member
Moderator
To me it's all about generalizing results obtained from a sample to a population.
This is a VERY general header, that I would shorthand as "statistical inference" or maybe just "inference" (depending on how you draw the generalizations).
Just as an example, you say "...to a population". A-la Deming, Enumerative studies deal with an existing, finite population (e.g. a certain electorate at a given time), while Analytic studies deal with future populations / processes that can in some cases be considered infinite. There are conceptual and mathematical implications to this difference; they should be handled differently. Otherwise one risks GIGO.
I consider Quality to be one of the best ideas that ever emerged from the human race. Sadly, its time came and went, and it never caught on. I'm only generally aware of SPC.
Never caught on? It's like saying that long-term thinking never caught on, or that prevention (rather than reaction) never caught on. I tend to agree that deep thinking has never been too popular, but it's always been around (scarsely ) and always will.

SPC is a tool in the QA toolbox.
"moving away from." There is no good reason to do this.
Yes there is - there are simply better (easier / more robust / more accurate) ways to achieve the same goals, that were only beginning to emerge in the 70's and 80's and have quite matured since.
Journals are bad enough as it is. Further reducing the amount of information reported in them doesn't appeal to me.
In that context "moving away from" doesn't mean just stopping to use something without suggesting an alternative. It means that instead of hypothesis tests and p-values there are now other concepts and methods. So hopefully the amount of relevant information is not reduced (whether a given journal does a good job at that or not, is a separate question).

#### Ronen E

##### Problem Solver
Staff member
Moderator
Another pervasive problem is that QA is the only discipline I know of where people of very diverse levels of education and intellectual development are expected to understand and invoke complex mathematical theory.
I only partly agree.

In most cases the "QA professional" is only required to apply (or put into use) techniques that rely on complex mathematical theory. True, some understanding is required for proper selection and implementation of these techniques, but not necessarily the complex mathematical foundations. Just like engineers many times successfully (and correctly) apply techniques that they are unable to fully understand the mathematical derivation of - these are sometimes simply too complex to practically master, and it's also unnecessary from an outcome perspective. The most important point is to not lose sight of the techniques limitations and underlying assumptions. Letting it slip is of course too easy - one has to actively and stubbornly fight for the maintenance of the latter, and this is were we usually fail.

Another issue is failing to recognise that "QA professional" doesn't equal "Jack of all trades". "People of very diverse levels of education and intellectual development" should not be drawing up experimental designs based on higher-than-undergrad-level statistical theory. Professional statisticians are there for that. Just like the average "QA professional" consults the plastics expert when they have issues with a plastic raw material, rather than diving into datasheets and chemical formulations. So maybe the problem is in the formal scoping (and internal classification) of the QA profession.

W Deming's SoPK (System of Profound Knowledge) Discussion Philosophy, Gurus, Innovation and Evolution 220
W Deming's SoPK (System of Profound Knowledge) Challenge Philosophy, Gurus, Innovation and Evolution 66
How to perform verification of the Statistical Analysis Software? Qualification and Validation (including 21 CFR Part 11) 3
V&V phase: Justification of acceptance criteria (statistical method ) - (Medical Device) Design and Development of Products and Processes 2
Statistical Alchemy Misc. Quality Assurance and Business Systems Related Topics 1
Looking for guidance to write an SOP on Statistical Methodologies? Statistical Analysis Tools, Techniques and SPC 7
Minimum sample size - Guidance and statistical rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Design Verification & Process Validation - Statistical sample sizes Design and Development of Products and Processes 2
Interactive visualization through graphical simulation of statistical concepts Statistical Analysis Tools, Techniques and SPC 3
Statistical Analysis - Check if these organisms at different concentrations affect the growth of wheat seedlings Using Minitab Software 4
Statistical Techniques Procedure - What should be included Document Control Systems, Procedures, Forms and Templates 4
O Statistical justification of sampling size in V&V tests ISO 13485:2016 - Medical Device Quality Management Systems 5
It’s time to talk about ditching statistical significance Statistical Analysis Tools, Techniques and SPC 6
Steve Prevette's Statistical Process Control (SPC) "Library" Statistical Analysis Tools, Techniques and SPC 0
A Balanced view of statistical tests Statistical Analysis Tools, Techniques and SPC 3
Statistical basis and justification while comparing / changing sampling plans Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
S SPC (Statistical Process Control) for Unilateral Tolerance - Questions Statistical Analysis Tools, Techniques and SPC 6
IATF 16949 9.1.1.3 Application of statistical concepts - Our technicians are quizzed for statistical knowledge IATF 16949 - Automotive Quality Systems Standard 3
Statistical basis for 30 pieces for FAI 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
B IATF 16949 clause 7.1.5.1.1 - Statistical studies shall be conducted IATF 16949 - Automotive Quality Systems Standard 3
A Statistical Process Control and Inspection in Footwear Production Statistical Analysis Tools, Techniques and SPC 0
M IATF 16949 Cl. 7.1.5.1.1 - Statistical studies shall be conducted IATF 16949 - Automotive Quality Systems Standard 3
Statistical Process Control Library Statistical Analysis Tools, Techniques and SPC 17
Happy Birthday Statistical Steven - 2015 Covegratulations 10
L When are Statistical techniques not applicable? Service Industry Specific Topics 16
FDA 21 CFR 820.250 - Does "valid statistical" always mean math? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
Common Statistical Errors Using Minitab Software 1
Y Statistical Analysis of Road Traffic Data Statistical Analysis Tools, Techniques and SPC 11
Class II Medical Device Manufacturer - SOP for 820.250 Statistical 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
E Correct Statistical Test comparing 2 Groups Statistical Analysis Tools, Techniques and SPC 14
A Statistical Software Calibration using Ford's "Sample Calibration File" Statistical Analysis Tools, Techniques and SPC 8
J Defining Martial Arts and Gymnastics Statistical Techniques Statistical Analysis Tools, Techniques and SPC 4
J Capability Analysis - Unusual Statistical Distribution of my Proccess Capability, Accuracy and Stability - Processes, Machines, etc. 5
M PhD Thesis Data Statistical Analysis Methods Statistical Analysis Tools, Techniques and SPC 2
J Statistical Significance and SPC Control Chart Reports Statistical Analysis Tools, Techniques and SPC 9
N Statistical Quality Improvement Action for Small Batch Production Statistical Analysis Tools, Techniques and SPC 17
Validation of macro - scripts - programs used in statistical software (Minitab-SAS... Qualification and Validation (including 21 CFR Part 11) 5
Statistical Process Control Crash Course - Question Quality Manager and Management Related Issues 10
H Statistical Models for Predictive Management of Software Processes Software Quality Assurance 2
A Statistical Correlation between ordered SKUs Statistical Analysis Tools, Techniques and SPC 8
S Minitab and Crystal Ball Statistical Analysis Software Using Minitab Software 13
M Determining if two different X's have any Statistical Significance on the Y's Statistical Analysis Tools, Techniques and SPC 4
Statistical Stability for the PQ of Analytical Equipment Qualification and Validation (including 21 CFR Part 11) 1
F Statistical Comparison of Product: High Average vs. Low Range Capability, Accuracy and Stability - Processes, Machines, etc. 13
E Using ANOVA during the PQ Validation Run to evaluate Statistical Differences Statistical Analysis Tools, Techniques and SPC 4
W Gage R&R for gage pins used to inspect a hole ID called a Statistical Tolerance Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
B AS 9100C sec 8.2.4 - " Recognized Statistical Principles" meaning AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 7
R What is PSW - Statistical Process Package + Level 5 APQP and PPAP 7
O Is SPC (Statistical Process Control) always required? Statistical Analysis Tools, Techniques and SPC 4