Interpreting the ANOVA
One area where the MSA manual is silent is the interpretation of the ANOVA table, particularly the p-values. See the following ANOVA table:
Source of Variation df SS MS F p
Reproducibility (3 operators) 2 ## ## # 0.47
Parts (10 parts) 9 ## ## # .001
Pure Error (Repeatability) 78 ##
Total Variation (90 measurements) 89
The AIAG MSA manual and statistical software always show a value for Reproducibility. However, if the p-value for Reproducibility is greater than your alpha risk (typically 0.05 or 0.10) then Reproducibility is not a significant source of measurement variation. The variation shown for Reproducibility in this situation should be pooled together with Repeatability. If the p-value is less than the alpha value, at least one operator is significantly different than the others.
If the gage is used for process control or statistical studies, the p-value for the parts should always be less than the alpha value. If it is not, the gage is not acceptable. Note: a p-value less than alpha does not guarantee that the gage is acceptable.
Interpreting the graphs
Step 1: Measurement Stability
A stable measurement system shows no out of control points or "non-random" patterns or trends in the range chart
Step 2: Resolution/Discrimination
Adequate resolution or discrimination means that the measurement units (inches, tenths of inches, thousandths of inches…) are sufficiently small enough to be able to "see" variation.
Bias in a measurement study is a "shift" in the pattern on the X-bar chart between operators (i.e., the same part pattern is evident, but one operator reads consistently higher or lower than the others).
Step 4: Measurement Capability
Measurement capability is the comparison of Measurement Variation to Product Variation to determine whether the current measurement process can see part to part differences. More than 50% of the part measurements should be out-of-control to be considered marginally acceptable.
Step 5: Operator Bias
Operator Bias is the comparison of Between-Operator variation to Within-Operator variation. The difference from operator to operator should be less than the variation within operator
Step 6: Operator Inconsistency
Operator Inconsistency is the comparison of Within-Operator variation for a specific operator to the overall Within-Operator variation.
What are the acceptance criteria?
The acceptance criteria are the same for both range and ANOVA methods.
% Tolerance or P/T Ratio (gages used for inspection):
Consider Reproducibility first. Is it acceptable? If Reproducibility is not acceptable, operators probably use different measurement techniques or some may have difficulty using or reading the gage. Identify the operator with the smallest variation from the Range chart. Study this operator’s technique, document and standardize it, train all operators in the new method, then repeat the R&R study. This should also address any operator*part interactions.
If Repeatability is large, consider the condition of the gage, rigidity of the gage or the part, locating of the part or gage, or excessive within-part variation.
The next article will be:
Intro to Measurement System Analysis (MSA) of Continuous Data – Part 6: Repeatability & Reproducibility for Non-Replicable Measurement Systems
One area where the MSA manual is silent is the interpretation of the ANOVA table, particularly the p-values. See the following ANOVA table:
Source of Variation df SS MS F p
Reproducibility (3 operators) 2 ## ## # 0.47
Parts (10 parts) 9 ## ## # .001
Pure Error (Repeatability) 78 ##
Total Variation (90 measurements) 89
The AIAG MSA manual and statistical software always show a value for Reproducibility. However, if the p-value for Reproducibility is greater than your alpha risk (typically 0.05 or 0.10) then Reproducibility is not a significant source of measurement variation. The variation shown for Reproducibility in this situation should be pooled together with Repeatability. If the p-value is less than the alpha value, at least one operator is significantly different than the others.
If the gage is used for process control or statistical studies, the p-value for the parts should always be less than the alpha value. If it is not, the gage is not acceptable. Note: a p-value less than alpha does not guarantee that the gage is acceptable.
Interpreting the graphs
Step 1: Measurement Stability
A stable measurement system shows no out of control points or "non-random" patterns or trends in the range chart
Step 2: Resolution/Discrimination
Adequate resolution or discrimination means that the measurement units (inches, tenths of inches, thousandths of inches…) are sufficiently small enough to be able to "see" variation.
- Stratification on a range chart is a good indication that there is a problem with inadequate resolution.
- The number of stratified levels on the Range Chart is an indicator of the degree of the problem.
- Fewer "levels" means less adequate resolution:
- A rule of thumb: There should be approximately 5 levels of resolution between the control limits on the Range Chart and less than 25% of the ranges equal to zero to be considered adequate.
Bias in a measurement study is a "shift" in the pattern on the X-bar chart between operators (i.e., the same part pattern is evident, but one operator reads consistently higher or lower than the others).
Step 4: Measurement Capability
Measurement capability is the comparison of Measurement Variation to Product Variation to determine whether the current measurement process can see part to part differences. More than 50% of the part measurements should be out-of-control to be considered marginally acceptable.
Step 5: Operator Bias
Operator Bias is the comparison of Between-Operator variation to Within-Operator variation. The difference from operator to operator should be less than the variation within operator
Step 6: Operator Inconsistency
Operator Inconsistency is the comparison of Within-Operator variation for a specific operator to the overall Within-Operator variation.
What are the acceptance criteria?
The acceptance criteria are the same for both range and ANOVA methods.
% Tolerance or P/T Ratio (gages used for inspection):
- < 10% is ideal;
- 10 - 20% is acceptable;
- 20 - 30% is marginal but may acceptable if the characteristic measured is not critical/significant and better gaging is not economical or feasible.
- < 10% is ideal;
- 10 - 20% is acceptable;
- 20 - 30% is marginal but may acceptable if the characteristic measured is not critical/significant and better gaging is not economical or feasible.
- < 1% is ideal;
- 1 - 9% is acceptable;
- > 9% is unacceptable.
- > 10 Ideal
- 5 -10 acceptable
- < 5 unacceptable
Consider Reproducibility first. Is it acceptable? If Reproducibility is not acceptable, operators probably use different measurement techniques or some may have difficulty using or reading the gage. Identify the operator with the smallest variation from the Range chart. Study this operator’s technique, document and standardize it, train all operators in the new method, then repeat the R&R study. This should also address any operator*part interactions.
If Repeatability is large, consider the condition of the gage, rigidity of the gage or the part, locating of the part or gage, or excessive within-part variation.
The next article will be:
Intro to Measurement System Analysis (MSA) of Continuous Data – Part 6: Repeatability & Reproducibility for Non-Replicable Measurement Systems
Last edited by a moderator:
