How to determine if an interaction is significant in GRR Anova Study

[llapanowski - 2007]

I'm a software guy trying to implement an MSA study using the ANOVA method as discussed in the AIAG MSA book 3rd Edition. I've been using Minitab to verify my results. In Minitab it computes a p-value for an F value with n1,n2 degrees of freedom. Does anybody know how this is computed. I know it matches the FDIST() function in EXCEL, but I actually have to implement it. Also, I've noticed on some of the EXCEL samples instead of computing the p-value, they use the FINV() function with a probability of 0.05. I'm pretty sure this value comes from an F-Table (which I can't find one that has all the numerators, so you have to interpolate, I guess). My question on this why choose 0.05 probability. I don't see this defined in the MSA book, so is this a de facto standard? Does anyone know how to compute the F-table for a given probablity? Thanks for any input.

 

[Eric York - 2010]

You pose, as some math geeks would say, some "nontrivial" questions.

I'll back up a bit, please forgive me if you are already intimately familiar with these concepts.

Start with the question of alpha. Yes, a significance level of .05 is sort of a default, but this is mainly because a lot of times (most even) we don't know how to set alpha. Alpha is "the probability of making a type-I error." Helpful eh? That is to say, it is the maximum probability (risk) you are willing to accept that you will be wrong if you conclude based on your sample that the population really does have the characteristic you are testing against. (Any purists out there will rail at this statement but it is more or less accurate and slightly more intuitive than the technical definition.) In a very small nutshell, the more risk-averse you are, the lower you want to set alpha.

The cool thing about choosing a level of significance is that it vastly simplifies your calculations. When you have a value x of F, you just go to the appropriate F-table for the alpha you have chosen, go to the intersection of your two degrees of freedom, and verify whether your x is bigger than the number shown. If it is, you are "statistically significant at the .05 level."

Much "less trivial" is the question of the F distribution. You will need to either code some numerical integrations that will run "on the fly" or write a routine to run a jillion integrations, then construct a few hundred F-tables that fit your expected needs and that your code will reference during the calculations performed by your application. If you choose the second method you may want to design your tables with all alpha for a specific pair of df in one table instead of all df for a set alpha in one table. This way you can target more easily your expected sample sizes and shorten lookup times.

Here is a decent link that will give you the equations you need for the calculations: https://en.wikipedia.org/wiki/F_distribution There are a couple different ways to go about the calculations. You will need to at the least reference the Beta function or the regularized incomplete Beta function. Possibly the most direct would be to go directly to the cumulative density function (cdf). (If you use the pdf you will be nesting integrals.) Your p-value for a given value x of F and degrees of freedom d1, d2 is 1-G(x, d1, d2) where G is the cdf.

Hope this helps. :

Eric

 

[Bill Ryan - 2007]

Wow :mg:

Excellent first posts by both of you

I'm going to have to reread (and probably think a bit about) Eric's answer to a very good question before I can do anything other than welcome you both to the Cove

 

[Eric York - 2010]

llapanowski,
I displayed quite prominently that fact that I'm not a programming guy! Of course you don't need a bunch of tables; you can simply put do your jillion integrations up front and store everything in a single 4-D array: One coordinate for the level of significance, one for each df, and one for the value of the F-statistic. Duh.

Bill,
Thanks for the kind words. It does me good to occasionally return, if only momentarily, to my geek roots.