Correct Statistical Test comparing 2 Groups

E

El Capten

#1
I am looking for the correct statistical test to determine whether the average performance of test method B compares favorably with the average of standard method A. If method B average is within 20% of method A average, I want to call method B valid. If method B average differs by more than 20%, I want to call method B invalid.

I am familiar with using Z and t tests, which are designed to determine whether B = A, but can they ( or some other test) be used when B < 80% of A? I have tried various parameters using both Z and t tests, but cant find the correct way to set up the equations. Thanks
 
Elsmar Forum Sponsor

dgriffith

Quite Involved in Discussions
#2
Wouldn't something like a t or z test at the 80% level work?
SHould either pass or fail.
2-sample t test at 0.20?
 
E

El Capten

#3
I have been playing with the various permutations on my Excel worksheet
A= 4.725, sd = .2, n=10, B=3.78 (20% of A). When B is 3.78 or greater I want the test to say "Accept", and when B = 3.77 or less it should say "Reject"
Perhaps you could write out the correct formula for me to try. Thanks
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#4
you don't get there by adjusting confidence or power. the 20% allowable difference is the DELTA in the sample size equation. this constructs the confidence interval for B and if the conf interval overlaps teh valu of A you have confidence that B is within 20% of A. if the confidence interval doesn't overlap A you have confidence that B is more than 20% different than A. you can use a similar approach with a hypothesis test for the difference between A and B being 20% of A but the math is more complicated...

a second note is that the 20% must be in true units of A not in a percentage...
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#7
hmmm. you say 'test method' do you really mean that A and B are tests? or are they processes and you are 'testing' if method B is not worse than 20% less than A? it does matter.

if A and B are test methods, I suggest using the Bland-Altmon approach.
if A and B are processes or product versions, I suggest using the delta approach as above but with 3 replicates if you can afford it. (more details from you on A and B would be helpful)
 
E

El Capten

#8
Yes, method A and B are tests. Method A is the approved lab test, and method B is a porposed alternate test. We want to say that as long as method B gives results that are within 80% of the method A standard, we will call method B good.

Initially I was looking for a quick way to compare the difference in performance between them, since I would not be the one doing the analysis. I wanted to hand that part off to someone else using the Statistical Data Analysis add-on pack in Excel.

Based on your input as well as further reading on my part, I decided to use the formula:

X = (A x .8) + ( 1.96 x (s/sqrt n) ) and create a spreadsheet for the users.

I am not familiar with the Bland - Altmon approach - can you explain? Thanks
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#10
Yes, method A and B are tests. Method A is the approved lab test, and method B is a porposed alternate test. We want to say that as long as method B gives results that are within 80% of the method A standard, we will call method B good.
I am not familiar with the Bland - Altmon approach - can you explain? Thanks
from this description I would recommend the bland-altman approach to method comparison. in this case this case teh sample size is roughly 10-30 matched pairs where teh same part, sample or event is tested by both the A and B method. the method calculates the average bias or difference between the mthods - for which you want B to be no more than A-.2A. The method does exactly this.
 
Thread starter Similar threads Forum Replies Date
E Help on identifying the correct statistical tool - Defects vs. Total products Statistical Analysis Tools, Techniques and SPC 11
lanley liao How to correct these NCs? Oil and Gas Industry Standards and Regulations 4
lanley liao Would you helpe me that how to correct these NCRs? Oil and Gas Industry Standards and Regulations 4
G How to ensure dimension of tapping hole before plating is correct? Manufacturing and Related Processes 2
C Correct Calibration Method for Dial Depth Gage General Measurement Device and Calibration Topics 6
S Is using ANSI/ASQ Z1.4-2008 the correct sampling plan to determine Pass/Fail of Apparel measurements? AQL - Acceptable Quality Level 4
K AS9100 - 7.3 - Are these black and white questions with specific correct answers AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
E Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
R What is the correct way to refer to NIST? Calibration Frequency (Interval) 4
T Correct MSA study for an automated camera system which makes attribute inspection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
M NANDO codes of products - These are not the GMDN codes, correct? EU Medical Device Regulations 4
K The correct way of doing a 2 Sample T test Reliability Analysis - Predictions, Testing and Standards 7
B PED 2014/68/EU - Is this Traceability audit finding correct? Other ISO and International Standards and European Regulations 4
K Assigning the Correct Procedures Applicable to Job Titles General Auditing Discussions 15
JoshuaFroud How important is using the "correct" GMDN? Other Medical Device Regulations World-Wide 14
M Specifications Aerospace - Who is responsible for providing the correct specification(s) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 14
supadrai OEM Changing the name of the manufacturer - length of process? Is 1-3 months correct? Other Medical Device Regulations World-Wide 0
J How long should I have to correct an audit finding? VDA Standards - Germany's Automotive Standards 7
E SPC for Dummies - The correct meaning of all the graphs Statistical Analysis Tools, Techniques and SPC 3
C UDI Label Verification - How do we ensure the barcode is correct and prove it? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
N Correct reading for analog pressure gauge ISO 17025 related Discussions 2
J How to measure process quality? Is DPPM the correct metric? Capability, Accuracy and Stability - Processes, Machines, etc. 5
P The correct way to do a Linearity and Stability Study on a Critical Dimension Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 8
L Choosing the correct Distribution for Acceptance Sampling Inspection, Prints (Drawings), Testing, Sampling and Related Topics 19
A Safety Shoes & Temporary Workers - Is it correct to reuse again or not? Occupational Health & Safety Management Standards 11
S Definition MTBF and MCBF Correct Definitions Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 5
T What is the correct UMDNS Code for Tracheostomy Tube? Other Medical Device Regulations World-Wide 2
N What is the easiest way to verify we have the correct customer supplied materials? Misc. Quality Assurance and Business Systems Related Topics 5
D The correct way to analyze Skewed Data using Minitab Using Minitab Software 3
C Rolling the correct M8 1.25 Pin Thread Manufacturing and Related Processes 2
E Database search is correct to capture medical device ISO 13485:2016 - Medical Device Quality Management Systems 2
S Choosing the correct Elisa Test Food Safety - ISO 22000, HACCP (21 CFR 120) 1
J TS 16949 - Which definition of Automotive is correct? IATF 16949 - Automotive Quality Systems Standard 3
L Correct Sample Size for Attribute Gage R&R (Good / Bad parts) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
R Correct Cable Ports After Work and Weekend Discussion Topics 2
S Is it gramatically correct to say Singapore people or Singaporeans ? Coffee Break and Water Cooler Discussions 12
H Determining the Correct Sample Size when AQL points to two Sample Sizes AQL - Acceptable Quality Level 7
J How IAQG plans to ensure that correct PEAR level determination is done by Auditors? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
K AS9102 Form 3 for Assemblies - Verification that the Assembly is Correct AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
J MSA Emergency - PV%=100(PV/TOL)in American? And that is correct? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 13
F Can you tell me if this table is correct? Gage R&R Problems Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
T Is the following flow for Training Needs Analysis correct? Training - Internal, External, Online and Distance Learning 6
M Calculating Cp and Cpk - Is my calculation correct? Capability, Accuracy and Stability - Processes, Machines, etc. 7
J Are my Cpk and Cp Calculations Correct? Capability, Accuracy and Stability - Processes, Machines, etc. 4
D What is the correct Working Standard for Digital Micrometer? General Measurement Device and Calibration Topics 21
T Correct use of NPT (National Pipe Thread) Thread Gages General Measurement Device and Calibration Topics 15
R First Pass Yield - Am I doing correct the First Pass Yield Metric Quality Tools, Improvement and Analysis 2
C Correct Tool for Gage R&R - Assay Dilutions Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
B How to Correct Bad Employee Performance-Attendance and/or Tardiness Quality Manager and Management Related Issues 84
R How to determine the correct Sample Frequency for SPC Features Statistical Analysis Tools, Techniques and SPC 66

Similar threads

Top Bottom