Hello All

I wonder if someone can help me understand the relationship between the P value, statistical power and significance levels in Hypothesis testing or are they totally independent of one another :confused:.

When I change the alpha level on a set of data the P value remains the same indicating no relationship. Is this so?

Is Statistical power controlled by the sample size, if so what sample size gives you a power of 0.8:confused:?

Many thanks
Steve

Hello All

I wonder if someone can help me understand the relationship between the P value, statistical power and significance levels in Hypothesis testing or are they totally independent of one another :confused:.

When I change the alpha level on a set of data the P value remains the same indicating no relationship. Is this so?

Is Statistical power controlled by the sample size, if so what sample size gives you a power of 0.8:confused:?

Many thanks
Steve

Steve, The Cove's Excellent Statistical Experts ;) will be back on Monday. No doubt they will respond to your post.

Stijloor.

Hello All

I wonder if someone can help me understand the relationship between the P value, statistical power and significance levels in Hypothesis testing or are they totally independent of one another :confused:.

When I change the alpha level on a set of data the P value remains the same indicating no relationship. Is this so?

Is Statistical power controlled by the sample size, if so what sample size gives you a power of 0.8:confused:?

Many thanks
Steve

OK let's start with the concept of errors.
There are 2 types called Type I and Type II errors. Simply put, Type I is rejecting your null hypothesis when in fact it is true. Type II is failing to reject your null hypothesis when in fact it is false. The risk of Type I error is called the alpha. You can control alpha by specifying it on you hypothesis test. Some use 0.05, others 0.01. Still others who are more conservative take 0.001. Alpha therefore can be subjective. And depending on which value you set it, your conclusion of the hypothesis test may change.
Here comes in the p-value. The p-value tells you the level of alpha in which your hypothesis test would reject the null hypothesis. If the p-value of the test is lets say 0.04, this can be presented and let the reviewer decide if this is significant or not based on his/her comfortable level of alpha.That is, if my alpha is 0.05, i will reject the null hypothesis. But if your alpha is 0.01, you will accept the null hypothesis. P-value therefore is more objective. Changing you alpha will not change p-value.

Beta on the other hand is the risk of having Type II error. Power is (1-Beta), or the ability to reject the null hypothesis when it is false. You are correct that it is controlled by increasing the sample size. The sample size that will give you a power of 0.8 will depend on the nature of your test (p-test, z-test, t-test?). But minitab can help you on this.

Excellent response from Reynald. Here (http://www.indiana.edu/%7Estatmath/stat/all/power/power.pdf) is some additional information.

Reynald and Miner,

Thank you very much. :applause::applause:

Both of you are ahead of the curve! :agree1:

Exceeding expectations.

Stijloor.

Another piece of power if the difference (delta) you consider to be statistically significant. As delta gets smaller, you need a larger sample size to have the same power to detect such a difference.