ANOVA GR&R: Minitab vs AIAG MSA results

[G.Pito]

Dear all,

I have copied values for GR&R from AIAG MSA 4th edition into an excel spreadsheet and then entered them in Minitab 18 in order to compare results with those written in the manual. It is the typical configuration with:
N.3 Appraisers
N.3 Trials
N.10 Operators

I don't understand why AIAG and Minitab results are slightly different, especially for the way they consider Interaction and Degrees of Freedom.

In the first image, you can see the results as they come from AIAG Manual. In the manual, the F ratio is calculated using Operator*Part interaction at the denominator, and then evaluated: since its value is lee the F critical one, the interaction is calculated pooling with equipment (error) value, maintaining the same degrees of freedom (and thus the MS Mean Squares):

(please find the attached image ANOVA MSA AIAG.png)

while Minitab calculates the same way the 1st part but then (considered that P > 0.25) another table is displayed where F is calculated versus Equipment (Error) and using different degrees of freedom. Here the Minitab results:

(please find the attached image ANOVA MINITAB.png)


1) Does anybody could explain why AIAG MSA and Minitab use different degrees of freedom when they calculate F ratio versus the Equipment?
2) Where does the P value 0.25 comes from? (it is the value Minitab use for choosing if to display the "Without interaction" panel if Alpha 0.05 is chosen)

Thank you very much!

 

[Miner]

I don't understand why AIAG and Minitab results are slightly different, especially for the way they consider Interaction and Degrees of Freedom.

In the first image, you can see the results as they come from AIAG Manual. In the manual, the F ratio is calculated using Operator*Part interaction at the denominator, and then evaluated: since its value is lee the F critical one, the interaction is calculated pooling with equipment (error) value, maintaining the same degrees of freedom (and thus the MS Mean Squares):

while Minitab calculates the same way the 1st part but then (considered that P > 0.25) another table is displayed where F is calculated versus Equipment (Error) and using different degrees of freedom. Here the Minitab results:

1) Does anybody could explain why AIAG MSA and Minitab use different degrees of freedom when they calculate F ratio versus the Equipment?
2) Where does the P value 0.25 comes from? (it is the value Minitab use for choosing if to display the "Without interaction" panel if Alpha 0.05 is chosen.

1) A Gage R&R is considered a random effects model wherein both the parts and the operators are selected at random. In random effects models, the interaction term is used to calculate the F-ratio. This was identically and correctly done for both AIAG and Minitab.

Your attachments do not show the revised AIAG table, so I'm not certain what you meant. However, Minitab is doing it correctly. When the interaction is pooled, the interaction and repeatability degrees of freedom and sum of squares are added. The new sum of squares is divided by the new degrees of freedom to calculate a new mean square. The mean squares for parts and operators are divided by this new mean square to calculate new F-ratios.

2) Good question. I think it came from an older version of the AIAG manual, but cannot prove that. Minitab 18 has changed this to 0.05.

 

[G.Pito]

First of all thank you for your reply. I'll try to be more clear: I am going to share with you (I attached the files since the Image link does not work):

1) The result table from AIAG MSA - Appendix A pag.198 where F-Ratio is obtained considering Appraiser/Parts interaction(file 1_ANOVA AIAG MSA with Interaction.png)


2) The result table from AIAG MSA pag.127 (referring to the same measurements data), where results are obtained pooling data with Equipment and not considering Appraiser * Part interaction (The document explains that it is that way because F value from point 1 was far lower the Critical F Critical for Alpha 0.05, and I assume it is the same as considering P value higher the Alpha). (File 2_ANOVA AIAG MSA without Interaction.png)


3) The Minitab Results both With Interaction and without Interaction (file 3_ANOVA MiniTab with and without Interaction.png) referrinng to the SAME measurements data.


I notice that in Minitab, the degrees of freedom change from "With Interaction" to "Without Interaction table" and this leads to different SS values...while in MSA AIAG Tables the degrees of freedom do not change: can you explain why?

(I hope you are able to check my images out since links seem not to be working)

Thanks a lot!

 

[Miner]

To clarify: When an interaction is "pooled" the df and SS associated with the interaction are added to the existing df and the SS of the residual error term (in this case the repeatability).

Following the graphic, the 18 and 60 df are added to equal 78. The 0.3590 and 2.7589 SS are added to equal 3.1179. The 3.1179 is divided by 78 to obtain the new MS of 0.03997, which becomes the new denominator for the F-ratios.

 

[G.Pito]

Hello Miner,
it makes sense but...I see that in AIAG MSA table, a different F value for Parts.
It is due to the fact MSe is computed in a different way (or at least this is what I can see):
it is always SSe / Dfe but in MSA AIAG it seems that they did not take into account the SUM of df and SS...

This is the core of my doubts...

 

[Miner]

The AIAG manual is in error on page 127. In this section, they say nothing (that I could find) about pooling. They just calculate the F ratio using the MSE term. This would commonly be used in a Fixed Effects model (i.e., parts and operators are not randomly selected), but would be incorrect for a random effects model (i.e., parts and operators are selected at random).

The table on page 198 is correct, but is the correct table prior to pooling. The manual does not mention pooling until the next paragraph and they do not show the pooled ANOVA table.

Again, the Minitab versions are correct.

 

[G.Pito]

Great, your help is very valuable and the fog is finally fading. so:

- Page 198: The table showing results PRIOR to polling is correct.
- Page 127: The Table after Polling seems to be not correct (it refers to "Fixed effects").

A Last clarification:
in page 198, there is something referred to page 127:
"Since the calculated F Value for the interaction (0.434)is less than the critical value of F Alpha, 18, 60, the interaction term in pooled with the equipment (error) term. That is, the estimate of variance is based on the model without interaction."

1) In your opinion, why in AIAG MSA the P value is not present and they use F-ratio only
2) I've read somewhere that F-Ratio and P values might have discordant values (one indicates NO INTERACTION while the other one POSSIBLE INTERACTION). In this case how can we tell if the Interaction needs to be considered?

Thanks!

 

[Miner]

A Last clarification:
in page 198, there is something referred to page 127:
"Since the calculated F Value for the interaction (0.434)is less than the critical value of F Alpha, 18, 60, the interaction term in pooled with the equipment (error) term. That is, the estimate of variance is based on the model without interaction."

1) In your opinion, why in AIAG MSA the P value is not present and they use F-ratio only
2) I've read somewhere that F-Ratio and P values might have discordant values (one indicates NO INTERACTION while the other one POSSIBLE INTERACTION). In this case how can we tell if the Interaction needs to be considered?

1) This is just an older method used before software could easily calculate p-values. You would consult a table for the F distribution and find the F ratio that was equivalent to a p-value of 0.05. This was called the critical F value. If the experiment F ratio was greater than F critical, the factor was deemed significant. If less than or equal to it was deemed not significant.
2) This is not correct. The two approaches are the same.