# Correct Statistical Test comparing 2 Groups

E

#### El Capten

I am looking for the correct statistical test to determine whether the average performance of test method B compares favorably with the average of standard method A. If method B average is within 20% of method A average, I want to call method B valid. If method B average differs by more than 20%, I want to call method B invalid.

I am familiar with using Z and t tests, which are designed to determine whether B = A, but can they ( or some other test) be used when B < 80% of A? I have tried various parameters using both Z and t tests, but cant find the correct way to set up the equations. Thanks

#### dgriffith

##### Quite Involved in Discussions
Wouldn't something like a t or z test at the 80% level work?
SHould either pass or fail.
2-sample t test at 0.20?

E

#### El Capten

I have been playing with the various permutations on my Excel worksheet
A= 4.725, sd = .2, n=10, B=3.78 (20% of A). When B is 3.78 or greater I want the test to say "Accept", and when B = 3.77 or less it should say "Reject"
Perhaps you could write out the correct formula for me to try. Thanks

#### Bev D

##### Heretical Statistician
Staff member
Super Moderator
you don't get there by adjusting confidence or power. the 20% allowable difference is the DELTA in the sample size equation. this constructs the confidence interval for B and if the conf interval overlaps teh valu of A you have confidence that B is within 20% of A. if the confidence interval doesn't overlap A you have confidence that B is more than 20% different than A. you can use a similar approach with a hypothesis test for the difference between A and B being 20% of A but the math is more complicated...

a second note is that the 20% must be in true units of A not in a percentage...

#### Bev D

##### Heretical Statistician
Staff member
Super Moderator
hmmm. you say 'test method' do you really mean that A and B are tests? or are they processes and you are 'testing' if method B is not worse than 20% less than A? it does matter.

if A and B are test methods, I suggest using the Bland-Altmon approach.
if A and B are processes or product versions, I suggest using the delta approach as above but with 3 replicates if you can afford it. (more details from you on A and B would be helpful)

E

#### El Capten

Yes, method A and B are tests. Method A is the approved lab test, and method B is a porposed alternate test. We want to say that as long as method B gives results that are within 80% of the method A standard, we will call method B good.

Initially I was looking for a quick way to compare the difference in performance between them, since I would not be the one doing the analysis. I wanted to hand that part off to someone else using the Statistical Data Analysis add-on pack in Excel.

Based on your input as well as further reading on my part, I decided to use the formula:

X = (A x .8) + ( 1.96 x (s/sqrt n) ) and create a spreadsheet for the users.

I am not familiar with the Bland - Altmon approach - can you explain? Thanks

#### Bev D

##### Heretical Statistician
Staff member
Super Moderator
Yes, method A and B are tests. Method A is the approved lab test, and method B is a porposed alternate test. We want to say that as long as method B gives results that are within 80% of the method A standard, we will call method B good.
I am not familiar with the Bland - Altmon approach - can you explain? Thanks
from this description I would recommend the bland-altman approach to method comparison. in this case this case teh sample size is roughly 10-30 matched pairs where teh same part, sample or event is tested by both the A and B method. the method calculates the average bias or difference between the mthods - for which you want B to be no more than A-.2A. The method does exactly this.

E Help on identifying the correct statistical tool - Defects vs. Total products Statistical Analysis Tools, Techniques and SPC 11
How to correct these NCs? Oil and Gas Industry Standards and Regulations 4
Would you helpe me that how to correct these NCRs? Oil and Gas Industry Standards and Regulations 4
How to ensure dimension of tapping hole before plating is correct? Manufacturing and Related Processes 2
Correct Calibration Method for Dial Depth Gage General Measurement Device and Calibration Topics 6
Is using ANSI/ASQ Z1.4-2008 the correct sampling plan to determine Pass/Fail of Apparel measurements? AQL - Acceptable Quality Level 4
AS9100 - 7.3 - Are these black and white questions with specific correct answers AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
What is the correct way to refer to NIST? Calibration Frequency (Interval) 4
Correct MSA study for an automated camera system which makes attribute inspection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
NANDO codes of products - These are not the GMDN codes, correct? EU Medical Device Regulations 4
The correct way of doing a 2 Sample T test Reliability Analysis - Predictions, Testing and Standards 7
PED 2014/68/EU - Is this Traceability audit finding correct? Other ISO and International Standards and European Regulations 4
Assigning the Correct Procedures Applicable to Job Titles General Auditing Discussions 15
How important is using the "correct" GMDN? Other Medical Device Regulations World-Wide 14
Specifications Aerospace - Who is responsible for providing the correct specification(s) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 14
OEM Changing the name of the manufacturer - length of process? Is 1-3 months correct? Other Medical Device Regulations World-Wide 0
How long should I have to correct an audit finding? VDA Standards - Germany's Automotive Standards 7
E SPC for Dummies - The correct meaning of all the graphs Statistical Analysis Tools, Techniques and SPC 3
C UDI Label Verification - How do we ensure the barcode is correct and prove it? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
N Correct reading for analog pressure gauge ISO 17025 related Discussions 2
J How to measure process quality? Is DPPM the correct metric? Capability, Accuracy and Stability - Processes, Machines, etc. 5
P The correct way to do a Linearity and Stability Study on a Critical Dimension Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 8
L Choosing the correct Distribution for Acceptance Sampling Inspection, Prints (Drawings), Testing, Sampling and Related Topics 19
Safety Shoes & Temporary Workers - Is it correct to reuse again or not? Occupational Health & Safety Management Standards 11
S Definition MTBF and MCBF Correct Definitions Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 5
T What is the correct UMDNS Code for Tracheostomy Tube? Other Medical Device Regulations World-Wide 2
What is the easiest way to verify we have the correct customer supplied materials? Misc. Quality Assurance and Business Systems Related Topics 5
D The correct way to analyze Skewed Data using Minitab Using Minitab Software 3
C Rolling the correct M8 1.25 Pin Thread Manufacturing and Related Processes 2
E Database search is correct to capture medical device ISO 13485:2016 - Medical Device Quality Management Systems 2
S Choosing the correct Elisa Test Food Safety - ISO 22000, HACCP (21 CFR 120) 1
TS 16949 - Which definition of Automotive is correct? IATF 16949 - Automotive Quality Systems Standard 3
L Correct Sample Size for Attribute Gage R&R (Good / Bad parts) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
R Correct Cable Ports After Work and Weekend Discussion Topics 2
Is it gramatically correct to say Singapore people or Singaporeans ? Coffee Break and Water Cooler Discussions 12
H Determining the Correct Sample Size when AQL points to two Sample Sizes AQL - Acceptable Quality Level 7
J How IAQG plans to ensure that correct PEAR level determination is done by Auditors? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
K AS9102 Form 3 for Assemblies - Verification that the Assembly is Correct AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
J MSA Emergency - PV%=100（PV/TOL）in American? And that is correct? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 13
F Can you tell me if this table is correct? Gage R&R Problems Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
Is the following flow for Training Needs Analysis correct? Training - Internal, External, Online and Distance Learning 6
M Calculating Cp and Cpk - Is my calculation correct? Capability, Accuracy and Stability - Processes, Machines, etc. 7
J Are my Cpk and Cp Calculations Correct? Capability, Accuracy and Stability - Processes, Machines, etc. 4
What is the correct Working Standard for Digital Micrometer? General Measurement Device and Calibration Topics 21
Correct use of NPT (National Pipe Thread) Thread Gages General Measurement Device and Calibration Topics 15
R First Pass Yield - Am I doing correct the First Pass Yield Metric Quality Tools, Improvement and Analysis 2
C Correct Tool for Gage R&R - Assay Dilutions Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
B How to Correct Bad Employee Performance-Attendance and/or Tardiness Quality Manager and Management Related Issues 84
R How to determine the correct Sample Frequency for SPC Features Statistical Analysis Tools, Techniques and SPC 66