Statistical Hypothesis Testing to prove signficant quality improvement or decline

K

Kelly Speiser

Statistical Hypothesis Testing

Is statistical hypothesis testing used by our Quality professionals to help "prove" signficant quality improvements/declines? It seems to be a valueable tool but I do not see or hear much about it, especially since focus has been on 'QS/ISO system' quality.

Is there interest from anyone to learn more about hypothesis testing if were easy to learn? The reason I'm asking is this: an easy-to-use workbook has been writen and the author needs feedback and peer reviews.

I find hypothesis testing fun and exciting. It is not so hard to do as DOE and one step above control charting.

Kelly Speiser

D

Don Winton

I use hypothesis testing a lot. I find it an invaluable tool in determining significant difference between processes and performance. It is relatively simple (t-test, ANOVA, ANOM, F, Chi-square, etc)and the results are easily presented. I would love to have a peek at the book referenced. Write me at [email protected] or [email protected]

Regards,
Don

N

Re: Statistical Hypothesis Testing

Hi Don and all other forum members,

I have a question related to hypothesis testing. I tried to find out in the book but couldn't get it. Hope any Statistic Champion would help me in finding the answer.

A hypothesis which contains 20 items and all these items are tested by using paired-samples t-test. If one or two items show significant difference whereas rest of the items are not statistically significant. In this case, should we reject the null hypothesis as a whole or we need some more items to be statistically significant. If yes, then how many items out of 20 should be statistically significant to reject the null hypothesis as a whole.

Thanks for any input or response.

Regards,

Tim Folkerts

Super Moderator
Re: Statistical Hypothesis Testing

A little more information would help, but let me try to address your questions.

> A hypothesis which contains 20 items and all these items are tested by
> using paired-samples t-test.

So it sounds like you have 10 pairs of data (or perhaps 20 pairs, but that isn't really important.) You want to know if the first hafl of each pair is different than the second half of each pair.

> If one or two items show significant difference whereas rest
> of the items are not statistically significant. In this case,
> should we reject the null hypothesis as a whole

With a paired t-test, you aren't really interested in any specific pairs. The test is already designed to examine the data as a whole. If you already have a good idea about statistical significance, then I think some other tests might be more informative, but it is hard to know without more info.

Tim

N

Re: Statistical Hypothesis Testing

Tim,

Thanks for taking time to respond my question that I had and my apologies for a bit late reply. Actually, I came to the conclusion that there isn't any clear answer for the hypothesis based on items. IMU, we should look for whether these items are interlinked with each other or not and how much impact would be on the whole hypothesis of an item that is significant. In my case the items are interlinked to each other. Therefore, rejecting the null hypothesis on the basis of one or two items that are significant would not be wrong.

Thanks again for your input.

Regards,

Steve Prevette

Deming Disciple
Staff member
Super Moderator
Dr. Deming was very much against hypothesis testing. I am formally trained (as an Operations Researcher) in hypothesis testing, and I teach statistics for City University where I am required by curriculum to teach hypothesis testing.

The big flaws to look out for are:

1. People tend to play games with significance levels. We'll specify 5%, but when the data comes in at 6%, well that was close enough.

2. People use 10% significance levels, and do hundreds of tests on the data to see what they can find. Well, 10% of what they find will be "significant".

3. Hypothesis testing (and ANOVA) both lose the time series aspect of the data. If the data have occurred in a time sequence, you are much better off with SPC.

N

Dr. Deming, really a BIG name in Quality and Statistics. One can not neglect his argument. In my research, the data was gathered one time and then analysis was made to interpret the outcomes. Therefore, hypothesis testing was plausible to analyze the data to accept or reject the formulated hypotheses.

Agreed, if this were the case on production line where the data is collected at a regular interval of time, then SPC would be a better choice of analyzing data.

There is a thread about 'Deming vs. Hypothesis Testing' which can provide you more thoughts about this topic. IMU, this issue needs an extensive research based on data to make any concrete findings.

Thread starter Similar threads Forum Replies Date
U Hypothesis Testing relationships - Is Statistical power controlled by the sample size Statistical Analysis Tools, Techniques and SPC 5
Deming vs. Statistical Hypothesis Testing Statistical Analysis Tools, Techniques and SPC 9
Design Verification & Process Validation - Statistical sample sizes Design and Development of Products and Processes 2
Interactive visualization through graphical simulation of statistical concepts Statistical Analysis Tools, Techniques and SPC 3
Profound Statistical Concepts Misc. Quality Assurance and Business Systems Related Topics 63
Statistical Analysis - Check if these organisms at different concentrations affect the growth of wheat seedlings Using Minitab Software 4
Statistical Techniques Procedure - What should be included Document Control Systems, Procedures, Forms and Templates 4
Statistical justification of sampling size in V&V tests ISO 13485:2016 - Medical Device Quality Management Systems 5
It’s time to talk about ditching statistical significance Statistical Analysis Tools, Techniques and SPC 6
Steve Prevette's Statistical Process Control (SPC) "Library" Statistical Analysis Tools, Techniques and SPC 0
A Balanced view of statistical tests Statistical Analysis Tools, Techniques and SPC 3
Statistical basis and justification while comparing / changing sampling plans Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
SPC (Statistical Process Control) for Unilateral Tolerance - Questions Statistical Analysis Tools, Techniques and SPC 6
IATF 16949 9.1.1.3 Application of statistical concepts - Our technicians are quizzed for statistical knowledge IATF 16949 - Automotive Quality Systems Standard 3
Please help identify appropriate statistical treatment Statistical Analysis Tools, Techniques and SPC 13
Statistical basis for 30 pieces for FAI 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
IATF 16949 clause 7.1.5.1.1 - Statistical studies shall be conducted IATF 16949 - Automotive Quality Systems Standard 3
Statistical Process Control and Inspection in Footwear Production Statistical Analysis Tools, Techniques and SPC 0
IATF 16949 Cl. 7.1.5.1.1 - Statistical studies shall be conducted IATF 16949 - Automotive Quality Systems Standard 3
Statistical Process Control Library Statistical Analysis Tools, Techniques and SPC 17
Happy Birthday Statistical Steven - 2015 Covegratulations 10
L When are Statistical techniques not applicable? Service Industry Specific Topics 16
FDA 21 CFR 820.250 - Does "valid statistical" always mean math? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
Common Statistical Errors Using Minitab Software 1
Y Statistical Analysis of Road Traffic Data Statistical Analysis Tools, Techniques and SPC 11
Class II Medical Device Manufacturer - SOP for 820.250 Statistical 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
E Correct Statistical Test comparing 2 Groups Statistical Analysis Tools, Techniques and SPC 14
A Statistical Software Calibration using Ford's "Sample Calibration File" Statistical Analysis Tools, Techniques and SPC 8
Defining Martial Arts and Gymnastics Statistical Techniques Statistical Analysis Tools, Techniques and SPC 4
J Capability Analysis - Unusual Statistical Distribution of my Proccess Capability, Accuracy and Stability - Processes, Machines, etc. 5
M PhD Thesis Data Statistical Analysis Methods Statistical Analysis Tools, Techniques and SPC 2
J Statistical Significance and SPC Control Chart Reports Statistical Analysis Tools, Techniques and SPC 9
N Statistical Quality Improvement Action for Small Batch Production Statistical Analysis Tools, Techniques and SPC 17
Validation of macro - scripts - programs used in statistical software (Minitab-SAS... Qualification and Validation (including 21 CFR Part 11) 5
Statistical Process Control Crash Course - Question Quality Manager and Management Related Issues 10
H Statistical Models for Predictive Management of Software Processes Software Quality Assurance 2
A Statistical Correlation between ordered SKUs Statistical Analysis Tools, Techniques and SPC 8
S Minitab and Crystal Ball Statistical Analysis Software Using Minitab Software 13
M Determining if two different X's have any Statistical Significance on the Y's Statistical Analysis Tools, Techniques and SPC 4
Statistical Stability for the PQ of Analytical Equipment Qualification and Validation (including 21 CFR Part 11) 1
F Statistical Comparison of Product: High Average vs. Low Range Capability, Accuracy and Stability - Processes, Machines, etc. 13
E Using ANOVA during the PQ Validation Run to evaluate Statistical Differences Statistical Analysis Tools, Techniques and SPC 4
W Gage R&R for gage pins used to inspect a hole ID called a Statistical Tolerance Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
AS 9100C sec 8.2.4 - " Recognized Statistical Principles" meaning AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 7
R What is PSW - Statistical Process Package + Level 5 APQP and PPAP 7
O Is SPC (Statistical Process Control) always required? Statistical Analysis Tools, Techniques and SPC 4
Any experience/feedback on statistical software tools...minitab -design expert -JMP Using Minitab Software 4
M SPC (Statistical Process Control) Book Recommendations wanted Book, Video, Blog and Web Site Reviews and Recommendations 1
How to use Statistical Process Control (SPC)? Statistical Analysis Tools, Techniques and SPC 8
TS 16949 8.1.1 Identification of Statistical Tools - Requirement Scope? IATF 16949 - Automotive Quality Systems Standard 13