Correct Statistical Test comparing 2 Groups

E

El Capten

I am looking for the correct statistical test to determine whether the average performance of test method B compares favorably with the average of standard method A. If method B average is within 20% of method A average, I want to call method B valid. If method B average differs by more than 20%, I want to call method B invalid.

I am familiar with using Z and t tests, which are designed to determine whether B = A, but can they ( or some other test) be used when B < 80% of A? I have tried various parameters using both Z and t tests, but cant find the correct way to set up the equations. Thanks
 

dgriffith

Quite Involved in Discussions
Wouldn't something like a t or z test at the 80% level work?
SHould either pass or fail.
2-sample t test at 0.20?
 
E

El Capten

I have been playing with the various permutations on my Excel worksheet
A= 4.725, sd = .2, n=10, B=3.78 (20% of A). When B is 3.78 or greater I want the test to say "Accept", and when B = 3.77 or less it should say "Reject"
Perhaps you could write out the correct formula for me to try. Thanks
 

Bev D

Heretical Statistician
Leader
Super Moderator
you don't get there by adjusting confidence or power. the 20% allowable difference is the DELTA in the sample size equation. this constructs the confidence interval for B and if the conf interval overlaps teh valu of A you have confidence that B is within 20% of A. if the confidence interval doesn't overlap A you have confidence that B is more than 20% different than A. you can use a similar approach with a hypothesis test for the difference between A and B being 20% of A but the math is more complicated...

a second note is that the 20% must be in true units of A not in a percentage...
 

Bev D

Heretical Statistician
Leader
Super Moderator
hmmm. you say 'test method' do you really mean that A and B are tests? or are they processes and you are 'testing' if method B is not worse than 20% less than A? it does matter.

if A and B are test methods, I suggest using the Bland-Altmon approach.
if A and B are processes or product versions, I suggest using the delta approach as above but with 3 replicates if you can afford it. (more details from you on A and B would be helpful)
 
E

El Capten

Yes, method A and B are tests. Method A is the approved lab test, and method B is a porposed alternate test. We want to say that as long as method B gives results that are within 80% of the method A standard, we will call method B good.

Initially I was looking for a quick way to compare the difference in performance between them, since I would not be the one doing the analysis. I wanted to hand that part off to someone else using the Statistical Data Analysis add-on pack in Excel.

Based on your input as well as further reading on my part, I decided to use the formula:

X = (A x .8) + ( 1.96 x (s/sqrt n) ) and create a spreadsheet for the users.

I am not familiar with the Bland - Altmon approach - can you explain? Thanks
 

Bev D

Heretical Statistician
Leader
Super Moderator
Yes, method A and B are tests. Method A is the approved lab test, and method B is a porposed alternate test. We want to say that as long as method B gives results that are within 80% of the method A standard, we will call method B good.
I am not familiar with the Bland - Altmon approach - can you explain? Thanks

from this description I would recommend the bland-altman approach to method comparison. in this case this case teh sample size is roughly 10-30 matched pairs where teh same part, sample or event is tested by both the A and B method. the method calculates the average bias or difference between the mthods - for which you want B to be no more than A-.2A. The method does exactly this.
 
Top Bottom