Resource icon

MSA 2019-11-11

Bill Levinson

Industrial Statistician and Trainer
Bill - did you read my paper? From what you just said you clearly don’t trust several critical items of the method AIAG has adopted.

The next part involves three supplier appraisers and one internal appraiser, who gets very different (and also more consistent) results. This looks like an interlaboratory issue similar to those addressed by ASTM. Henry Ford gave an example of this about 100 years ago. He said arguments between inspectors on opposite sides of a shop sometimes degenerated into accusations and even lawsuits because they got different measurements from the same part. I assume from context that the customer's inspector said it was out of specification, and the supplier's inspector said it was in specification, and then they accused each other of trying to cheat their employers. The answer was much simpler; one side of the shop was warmer and both inspectors were telling the truth.

Your example, though, suggests they are in fact using different methods which slide 54 confirms.
 

Bev D

Heretical Statistician
Leader
Super Moderator
Your article raises some good points; I am going through it in a little more detail.

On page 26, "A subgroup size of 10 can be biased by one outlier" ... the outlier should be evident in the MSA assessment such as the range chart and also in anything supporting, like a normal probability plot. At this point, we know something is wrong with the study (or that there is a risk of non-random assignable cause measurement errors).

Also, 10 is a clearly inadequate sample for estimating the part variation; I see part variation as more of an academic exercise where we compare what we get from the MSA to what we get from a process capability study that uses 30 or more parts. I would not rely at all on the MSA's part variation to reflect the actual process performance. Also, I am not sure why part variation is even relevant (unless one wants to compare part variation to gage variation) because the key metric is the ratio of the gage standard deviation to the specification width or tolerance, and not to the process standard deviation.

I would also, as you point out on page 28, be hesitant to add %EV and %AV as calculated for exactly the reason you describe. The deliverable is the total gage standard deviation for which a precision/tolerance ratio can be calculated. I know that various software reports these things but my webinar on MSA does not bother with them at all because the deliverable is the P/T ratio.

.

You are so far agreeing with the points made in my paper? Again you say there are things you don’t agree with in the AIAG method, but yet you trust it?

There are many people who don’t understand the flaws of the AIAG method - they just blindly apply the rules, treating Guage R&R like a check box and then get confused when they don't ‘Pass’. There is a better way and a better use of MSA for understanding your measurement system and improving it so it that it helps you and it doesn’t hinder you. It is not and should not be treated like a just another black box requirement hurdle that needs to be jumped.

I wil leave you with one thought: learning and research - like experiments - is best facilitated by approaches that challenge our thinking (conventional wisdom) than by seeking to confirm our current thinking.
 

Bill Levinson

Industrial Statistician and Trainer
You are so far agreeing with the points made in my paper? Again you say there are things you don’t agree with in the AIAG method, but yet you trust it?

There are many people who don’t understand the flaws of the AIAG method - they just blindly apply the rules, treating Guage R&R like a check box and then get confused when they don't ‘Pass’. There is a better way and a better use of MSA for understanding your measurement system and improving it so it that it helps you and it doesn’t hinder you. It is not and should not be treated like a just another black box requirement hurdle that needs to be jumped.

People should always be cautioned against using a black box, as in "garbage in, garbage out." I have seen this with non-normal data being put into a computer and a process performance index coming out the other end (based on the assumption of normality) which can underestimate the non-normal fraction by orders of magnitude.

Same for MSA--one has to follow the process for randomization of the parts, and also checking the results with the control chart of ranges to identify outliers that can, as you pointed out, contaminate the results.

If on the other hand we really use 10 parts and have two or three inspectors measure them, with randomization to exclude extraneous variation sources, and also consistent environmental conditions (where required, e.g. if temperature affects part dimensions), then the AIAG procedure, which also appears in textbooks and (as I recall) an ASTM standard, will deliver a reasonably good estimate of the equipment variation and appraiser variation. An alternative to the average and range method is to use two-way Analysis of Variance in which we know the F test will show the parts to be different (we expect them to be different due to part variation) and we hope the null hypothesis that the appraisers are identical will not be rejected. The variance components also can be isolated.

Even given the issue of the appraiser as fixed rather than random variation, detection of appraiser variation means this is an issue that can hopefully be addressed as shown in your example with the supplier appraisers versus the internal appraiser who used a different, and superior, method.
 

Ronen E

Problem Solver
Moderator
According to hypothesis testing theory, when the null hypothesis is not rejected it doesn't mean it's true. It only means we can't be sure it's false. If one is looking to establish confidence through this method (which, nevertheless, may be argued against as a whole), the hypotheses should be phrased so that the desired outcome is demonstrated through rejection of the null hypothesis - this scenario is (theoretically) providing assurance that the null hypothesis is false, and therefore the alternative hypothesis (its complement) is true.
 

Bill Levinson

Industrial Statistician and Trainer
According to hypothesis testing theory, when the null hypothesis is not rejected it doesn't mean it's true. It only means we can't be sure it's false. If one is looking to establish confidence through this method (which, nevertheless, may be argued against as a whole), the hypotheses should be phrased so that the desired outcome is demonstrated through rejection of the null hypothesis - this scenario is (theoretically) providing assurance that the null hypothesis is false, and therefore the alternative hypothesis (its complement) is true.

That's correct; we can never prove the null hypothesis (that the appraisers are identical). We can only fail to reject it beyond a specified reasonable doubt. Gage R&R can reject the null hypothesis, especially when the ANOVA method is used, that there is no difference between the appraisers.
 

Moncia

Quite Involved in Discussions
HI Guys

Long time no talk. Nice to see you all in here
Hope you are all well?

We were asked to supply R&R Anova MSA study. Never done it. I've looked online and there are many tools available to purchase.
I've got OK from the boss so can you recommend some easy to use excel spreadsheet for reasonable price?
What are you using ?

Looking forward to your replies
 

Bill Levinson

Industrial Statistician and Trainer
That's like saying, " I trust the AIAG manuals because they contain the AIAG methods." They got to be "generally accepted methods" because of AIAG requirements.

AIAG uses the same methods that are seen in textbooks, ASQ certification review materials, and are also used by StatGraphics and Minitab. Also, AIAG has a vested interest in making sure they work as MSA is used to help quantify gage precision in automotive manufacture.
 

Jim Wynne

Leader
Admin
AIAG uses the same methods that are seen in textbooks, ASQ certification review materials, and are also used by StatGraphics and Minitab. Also, AIAG has a vested interest in making sure they work as MSA is used to help quantify gage precision in automotive manufacture.
None of that contradicts the fact that the origin of it all was the automotive industry. If you believe that the AIAG sponsors always do what makes sense, you might not have been paying attention for a long time.
 

Johnnymo62

Haste Makes Waste
HI Guys

Long time no talk. Nice to see you all in here
Hope you are all well?

We were asked to supply R&R Anova MSA study. Never done it. I've looked online and there are many tools available to purchase.
I've got OK from the boss so can you recommend some easy to use excel spreadsheet for reasonable price?
What are you using ?

Looking forward to your replies
Hi Monica,

I've only used Minitab. My customer was Ford and they prefer it. It took me a bit of getting used to. I hear its expensive.

You can try the AIAG APQP workbook. It has an ANOVA GRR in it.
 
Last edited:

Bill Levinson

Industrial Statistician and Trainer
HI Guys

Long time no talk. Nice to see you all in here
Hope you are all well?

We were asked to supply R&R Anova MSA study. Never done it. I've looked online and there are many tools available to purchase.
I've got OK from the boss so can you recommend some easy to use excel spreadsheet for reasonable price?
What are you using ?

Looking forward to your replies

It can be done in Excel as well as StatGraphics and Minitab. If the data are not confidential, I could give it a try.
 
Top Bottom