The correct way of doing a 2 Sample T test

kuyakut

Involved In Discussions
Hi Guys,

I just need a clarification regarding 2 Sample T Test.
What is correct way of doing it
Base on my customer , if the P value is zero my 2 Sample T test fail.
But here is the catch , if my alternative hypothesis is greater than the P value is 1.00 but if my alternative hypothesis is less than the P Value is zero.
So which alternative hypothesis should I use.
I have attached a 2 sample t test result

Attachments

• 50.7 KB Views: 157

Miner

Forum Moderator
Staff member
There are three potential alternate hypotheses from which you may select for a 2-sample t-test:
• There is a difference between A and B (2-tailed test)
• A > B (1-tailed test)
• A < B (1-tailed test)
The correct alternate hypothesis depends on what you are attempting to prove. In your case, you are trying to prove the machines are identical. A 2-sample t-test CAN NOT prove they are identical. It can only prove they are different. You should use a 2-sample equivalence test, which IS designed to prove the samples are identical.

kuyakut

Involved In Discussions
There are three potential alternate hypotheses from which you may select for a 2-sample t-test:
• There is a difference between A and B (2-tailed test)
• A > B (1-tailed test)
• A < B (1-tailed test)
The correct alternate hypothesis depends on what you are attempting to prove. In your case, you are trying to prove the machines are identical. A 2-sample t-test CAN NOT prove they are identical. It can only prove they are different. You should use a 2-sample equivalence test, which IS designed to prove the samples are identical.

Hi Miner,

I did do a test for equal variance and the P value is more than 0.05.
So does this prove my machine is identical or not.
I'm not really sure how to prove that they are identical.

Miner

Forum Moderator
Staff member
I did do a test for equal variance and the P value is more than 0.05.
So does this prove my machine is identical or not.
I'm not really sure how to prove that they are identical.
The test for equal variances found insufficient evidence to indicate that the variances between the machines were STATISTICALLY different. This is not the same as proving that they are identical. In addition, this test only considers the variances. It does not consider the means.

Watchcat

Trusted Information Resource
Hi Miner,
I'm not really sure how to prove that they are identical.
You can't. No matter what anyone tells you, you can't.

You can show no difference, usually very easily, with low power to detect a difference, but if you are going to present this to FDA to support a claim of substantial equivalence, I would hope FDA would not look kindly on this sleight of hand. Plus your sample size justification is probably going to sound pretty lame.

On the other hand, you can show no difference with a design that offers reasonable statistical power (usually it's all about the sample size) which is to say, a design that doesn't make it so easy. A good research statistician can help you with this. You can try to DIY, but I don't recommend it. (If there is anyone in this game that is worth their weight in gold, it's a good statistician. And they don't usually charge that much, so a bargain to boot. )

If you decide to secure the services of a statistician, don't hesitate to get back to me. I was looking for one a few years ago, and they were not easy to find. Would be happy to point you in a few directions...

John Predmore

Trusted Information Resource
My stats professor summed it up succinctly: "p-value is the likelihood, if you reject the null hypothesis, that you would be wrong". In other words, the experiment produced a result due to luck of the draw which led you to conclude the null hypothesis was violated, when in actuality, the null hypothesis was true. Thus, for many experiments, you want p-value close to zero. What you conclude depends on what you are trying to show, the test you lay out, and the results of your sample of course.

Rejecting the null hypothesis is the outcome most people care about and most statistical experiments are fashioned to produce, because rejecting the null hypothesis is a stronger statistical statement than being unable to refute the null hypothesis due to insufficient evidence.

Watchcat

Trusted Information Resource
for many experiments, you want p-value close to zero
That's true when you are doing scientific research, and especially if you are doing it in academia, where you hope to publish instead of perish. When you are doing it for more practical reasons (e.g., demonstration of substantial equivalence), I personally wouldn't set the bar quite so high (or, in this case, low?). I'd go risk-based. That said, if it's important to persuasively reject the null, I'd look for a lower p-value than if I was simply trying to decide whether to reject or accept.

R First Pass Yield - Am I doing correct the First Pass Yield Metric Quality Tools, Improvement and Analysis 2
J RPN numbers in FMEA - It is not correct to go only by the RPN numbers when doing FMEA FMEA and Control Plans 12
How to ensure dimension of tapping hole before plating is correct? Manufacturing and Related Processes 2
Correct Calibration Method for Dial Depth Gage General Measurement Device and Calibration Topics 6
Is using ANSI/ASQ Z1.4-2008 the correct sampling plan to determine Pass/Fail of Apparel measurements? AQL - Acceptable Quality Level 4
AS9100 - 7.3 - Are these black and white questions with specific correct answers AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
What is the correct way to refer to NIST? Calibration Frequency (Interval) 4
Correct MSA study for an automated camera system which makes attribute inspection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
NANDO codes of products - These are not the GMDN codes, correct? EU Medical Device Regulations 4
PED 2014/68/EU - Is this Traceability audit finding correct? Other ISO and International Standards and European Regulations 4
Assigning the Correct Procedures Applicable to Job Titles General Auditing Discussions 15
How important is using the "correct" GMDN? Other Medical Device Regulations World-Wide 14
Specifications Aerospace - Who is responsible for providing the correct specification(s) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 14
OEM Changing the name of the manufacturer - length of process? Is 1-3 months correct? Other Medical Device Regulations World-Wide 0
How long should I have to correct an audit finding? VDA Standards - Germany's Automotive Standards 7
E SPC for Dummies - The correct meaning of all the graphs Statistical Analysis Tools, Techniques and SPC 3
C UDI Label Verification - How do we ensure the barcode is correct and prove it? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
N Correct reading for analog pressure gauge ISO 17025 related Discussions 2
J How to measure process quality? Is DPPM the correct metric? Capability, Accuracy and Stability - Processes, Machines, etc. 5
P The correct way to do a Linearity and Stability Study on a Critical Dimension Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 8
L Choosing the correct Distribution for Acceptance Sampling Inspection, Prints (Drawings), Testing, Sampling and Related Topics 19
Safety Shoes & Temporary Workers - Is it correct to reuse again or not? Occupational Health & Safety Management Standards 11
S Definition MTBF and MCBF Correct Definitions Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 5
T What is the correct UMDNS Code for Tracheostomy Tube? Other Medical Device Regulations World-Wide 2
E Correct Statistical Test comparing 2 Groups Statistical Analysis Tools, Techniques and SPC 14
What is the easiest way to verify we have the correct customer supplied materials? Misc. Quality Assurance and Business Systems Related Topics 5
D The correct way to analyze Skewed Data using Minitab Using Minitab Software 3
C Rolling the correct M8 1.25 Pin Thread Manufacturing and Related Processes 2
E Database search is correct to capture medical device ISO 13485:2016 - Medical Device Quality Management Systems 2
S Choosing the correct Elisa Test Food Safety - ISO 22000, HACCP (21 CFR 120) 1
TS 16949 - Which definition of Automotive is correct? IATF 16949 - Automotive Quality Systems Standard 3
L Correct Sample Size for Attribute Gage R&R (Good / Bad parts) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
R Correct Cable Ports After Work and Weekend Discussion Topics 2
Is it gramatically correct to say Singapore people or Singaporeans ? Coffee Break and Water Cooler Discussions 12
H Determining the Correct Sample Size when AQL points to two Sample Sizes AQL - Acceptable Quality Level 7
J How IAQG plans to ensure that correct PEAR level determination is done by Auditors? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
K AS9102 Form 3 for Assemblies - Verification that the Assembly is Correct AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
J MSA Emergency - PV%=100（PV/TOL）in American? And that is correct? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 13
F Can you tell me if this table is correct? Gage R&R Problems Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
Is the following flow for Training Needs Analysis correct? Training - Internal, External, Online and Distance Learning 6
M Calculating Cp and Cpk - Is my calculation correct? Capability, Accuracy and Stability - Processes, Machines, etc. 7
J Are my Cpk and Cp Calculations Correct? Capability, Accuracy and Stability - Processes, Machines, etc. 4
What is the correct Working Standard for Digital Micrometer? General Measurement Device and Calibration Topics 21
Correct use of NPT (National Pipe Thread) Thread Gages General Measurement Device and Calibration Topics 15
C Correct Tool for Gage R&R - Assay Dilutions Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
B How to Correct Bad Employee Performance-Attendance and/or Tardiness Quality Manager and Management Related Issues 84
R How to determine the correct Sample Frequency for SPC Features Statistical Analysis Tools, Techniques and SPC 66
Q Latest ISO 9001:2008 Rev. E - Correct? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
The CORRECT steps to implement an SPC chart Imported Legacy Blogs 16