Search the Elsmar Cove!
**Search ALL of Elsmar.com** with DuckDuckGo including content not in the forum - Search results with No ads.

The correct way of doing a 2 Sample T test

kuyakut

Involved In Discussions
#1
Hi Guys,

I just need a clarification regarding 2 Sample T Test.
What is correct way of doing it
Base on my customer , if the P value is zero my 2 Sample T test fail.
But here is the catch , if my alternative hypothesis is greater than the P value is 1.00 but if my alternative hypothesis is less than the P Value is zero.
So which alternative hypothesis should I use.
I have attached a 2 sample t test result
 

Attachments

Miner

Forum Moderator
Staff member
Admin
#2
There are three potential alternate hypotheses from which you may select for a 2-sample t-test:
  • There is a difference between A and B (2-tailed test)
  • A > B (1-tailed test)
  • A < B (1-tailed test)
The correct alternate hypothesis depends on what you are attempting to prove. In your case, you are trying to prove the machines are identical. A 2-sample t-test CAN NOT prove they are identical. It can only prove they are different. You should use a 2-sample equivalence test, which IS designed to prove the samples are identical.
 

kuyakut

Involved In Discussions
#3
There are three potential alternate hypotheses from which you may select for a 2-sample t-test:
  • There is a difference between A and B (2-tailed test)
  • A > B (1-tailed test)
  • A < B (1-tailed test)
The correct alternate hypothesis depends on what you are attempting to prove. In your case, you are trying to prove the machines are identical. A 2-sample t-test CAN NOT prove they are identical. It can only prove they are different. You should use a 2-sample equivalence test, which IS designed to prove the samples are identical.

Hi Miner,

Thanks for the reply,

I did do a test for equal variance and the P value is more than 0.05.
So does this prove my machine is identical or not.
I'm not really sure how to prove that they are identical.
 

Miner

Forum Moderator
Staff member
Admin
#4
I did do a test for equal variance and the P value is more than 0.05.
So does this prove my machine is identical or not.
I'm not really sure how to prove that they are identical.
The test for equal variances found insufficient evidence to indicate that the variances between the machines were STATISTICALLY different. This is not the same as proving that they are identical. In addition, this test only considers the variances. It does not consider the means.
 

Watchcat

Quite Involved in Discussions
#5
Hi Miner,
I'm not really sure how to prove that they are identical.
You can't. No matter what anyone tells you, you can't.

You can show no difference, usually very easily, with low power to detect a difference, but if you are going to present this to FDA to support a claim of substantial equivalence, I would hope FDA would not look kindly on this sleight of hand. Plus your sample size justification is probably going to sound pretty lame.

On the other hand, you can show no difference with a design that offers reasonable statistical power (usually it's all about the sample size) which is to say, a design that doesn't make it so easy. A good research statistician can help you with this. You can try to DIY, but I don't recommend it. (If there is anyone in this game that is worth their weight in gold, it's a good statistician. And they don't usually charge that much, so a bargain to boot. :) )

If you decide to secure the services of a statistician, don't hesitate to get back to me. I was looking for one a few years ago, and they were not easy to find. Would be happy to point you in a few directions...
 

John Predmore

Involved In Discussions
#6
My stats professor summed it up succinctly: "p-value is the likelihood, if you reject the null hypothesis, that you would be wrong". In other words, the experiment produced a result due to luck of the draw which led you to conclude the null hypothesis was violated, when in actuality, the null hypothesis was true. Thus, for many experiments, you want p-value close to zero. What you conclude depends on what you are trying to show, the test you lay out, and the results of your sample of course.

Rejecting the null hypothesis is the outcome most people care about and most statistical experiments are fashioned to produce, because rejecting the null hypothesis is a stronger statistical statement than being unable to refute the null hypothesis due to insufficient evidence.
 

Watchcat

Quite Involved in Discussions
#7
for many experiments, you want p-value close to zero
That's true when you are doing scientific research, and especially if you are doing it in academia, where you hope to publish instead of perish. When you are doing it for more practical reasons (e.g., demonstration of substantial equivalence), I personally wouldn't set the bar quite so high (or, in this case, low?). I'd go risk-based. That said, if it's important to persuasively reject the null, I'd look for a lower p-value than if I was simply trying to decide whether to reject or accept.
 
Top Bottom