Resource icon

MSA 2019-11-11

Bill Levinson

Industrial Statistician and Trainer
#11
Bill - did you read my paper? From what you just said you clearly don’t trust several critical items of the method AIAG has adopted.
The next part involves three supplier appraisers and one internal appraiser, who gets very different (and also more consistent) results. This looks like an interlaboratory issue similar to those addressed by ASTM. Henry Ford gave an example of this about 100 years ago. He said arguments between inspectors on opposite sides of a shop sometimes degenerated into accusations and even lawsuits because they got different measurements from the same part. I assume from context that the customer's inspector said it was out of specification, and the supplier's inspector said it was in specification, and then they accused each other of trying to cheat their employers. The answer was much simpler; one side of the shop was warmer and both inspectors were telling the truth.

Your example, though, suggests they are in fact using different methods which slide 54 confirms.
 
Elsmar Forum Sponsor

Bev D

Heretical Statistician
Staff member
Super Moderator
#12
Your article raises some good points; I am going through it in a little more detail.

On page 26, "A subgroup size of 10 can be biased by one outlier" ... the outlier should be evident in the MSA assessment such as the range chart and also in anything supporting, like a normal probability plot. At this point, we know something is wrong with the study (or that there is a risk of non-random assignable cause measurement errors).

Also, 10 is a clearly inadequate sample for estimating the part variation; I see part variation as more of an academic exercise where we compare what we get from the MSA to what we get from a process capability study that uses 30 or more parts. I would not rely at all on the MSA's part variation to reflect the actual process performance. Also, I am not sure why part variation is even relevant (unless one wants to compare part variation to gage variation) because the key metric is the ratio of the gage standard deviation to the specification width or tolerance, and not to the process standard deviation.

I would also, as you point out on page 28, be hesitant to add %EV and %AV as calculated for exactly the reason you describe. The deliverable is the total gage standard deviation for which a precision/tolerance ratio can be calculated. I know that various software reports these things but my webinar on MSA does not bother with them at all because the deliverable is the P/T ratio.

.
You are so far agreeing with the points made in my paper? Again you say there are things you don’t agree with in the AIAG method, but yet you trust it?

There are many people who don’t understand the flaws of the AIAG method - they just blindly apply the rules, treating Guage R&R like a check box and then get confused when they don't ‘Pass’. There is a better way and a better use of MSA for understanding your measurement system and improving it so it that it helps you and it doesn’t hinder you. It is not and should not be treated like a just another black box requirement hurdle that needs to be jumped.

I wil leave you with one thought: learning and research - like experiments - is best facilitated by approaches that challenge our thinking (conventional wisdom) than by seeking to confirm our current thinking.
 

Bill Levinson

Industrial Statistician and Trainer
#13
You are so far agreeing with the points made in my paper? Again you say there are things you don’t agree with in the AIAG method, but yet you trust it?

There are many people who don’t understand the flaws of the AIAG method - they just blindly apply the rules, treating Guage R&R like a check box and then get confused when they don't ‘Pass’. There is a better way and a better use of MSA for understanding your measurement system and improving it so it that it helps you and it doesn’t hinder you. It is not and should not be treated like a just another black box requirement hurdle that needs to be jumped.
People should always be cautioned against using a black box, as in "garbage in, garbage out." I have seen this with non-normal data being put into a computer and a process performance index coming out the other end (based on the assumption of normality) which can underestimate the non-normal fraction by orders of magnitude.

Same for MSA--one has to follow the process for randomization of the parts, and also checking the results with the control chart of ranges to identify outliers that can, as you pointed out, contaminate the results.

If on the other hand we really use 10 parts and have two or three inspectors measure them, with randomization to exclude extraneous variation sources, and also consistent environmental conditions (where required, e.g. if temperature affects part dimensions), then the AIAG procedure, which also appears in textbooks and (as I recall) an ASTM standard, will deliver a reasonably good estimate of the equipment variation and appraiser variation. An alternative to the average and range method is to use two-way Analysis of Variance in which we know the F test will show the parts to be different (we expect them to be different due to part variation) and we hope the null hypothesis that the appraisers are identical will not be rejected. The variance components also can be isolated.

Even given the issue of the appraiser as fixed rather than random variation, detection of appraiser variation means this is an issue that can hopefully be addressed as shown in your example with the supplier appraisers versus the internal appraiser who used a different, and superior, method.
 

Ronen E

Problem Solver
Staff member
Moderator
#14
According to hypothesis testing theory, when the null hypothesis is not rejected it doesn't mean it's true. It only means we can't be sure it's false. If one is looking to establish confidence through this method (which, nevertheless, may be argued against as a whole), the hypotheses should be phrased so that the desired outcome is demonstrated through rejection of the null hypothesis - this scenario is (theoretically) providing assurance that the null hypothesis is false, and therefore the alternative hypothesis (its complement) is true.
 

Bill Levinson

Industrial Statistician and Trainer
#15
According to hypothesis testing theory, when the null hypothesis is not rejected it doesn't mean it's true. It only means we can't be sure it's false. If one is looking to establish confidence through this method (which, nevertheless, may be argued against as a whole), the hypotheses should be phrased so that the desired outcome is demonstrated through rejection of the null hypothesis - this scenario is (theoretically) providing assurance that the null hypothesis is false, and therefore the alternative hypothesis (its complement) is true.
That's correct; we can never prove the null hypothesis (that the appraisers are identical). We can only fail to reject it beyond a specified reasonable doubt. Gage R&R can reject the null hypothesis, especially when the ANOVA method is used, that there is no difference between the appraisers.
 

Moncia

Quite Involved in Discussions
#16
HI Guys

Long time no talk. Nice to see you all in here
Hope you are all well?

We were asked to supply R&R Anova MSA study. Never done it. I've looked online and there are many tools available to purchase.
I've got OK from the boss so can you recommend some easy to use excel spreadsheet for reasonable price?
What are you using ?

Looking forward to your replies
 

Bill Levinson

Industrial Statistician and Trainer
#17
That's like saying, " I trust the AIAG manuals because they contain the AIAG methods." They got to be "generally accepted methods" because of AIAG requirements.
AIAG uses the same methods that are seen in textbooks, ASQ certification review materials, and are also used by StatGraphics and Minitab. Also, AIAG has a vested interest in making sure they work as MSA is used to help quantify gage precision in automotive manufacture.
 

Jim Wynne

Staff member
Admin
#18
AIAG uses the same methods that are seen in textbooks, ASQ certification review materials, and are also used by StatGraphics and Minitab. Also, AIAG has a vested interest in making sure they work as MSA is used to help quantify gage precision in automotive manufacture.
None of that contradicts the fact that the origin of it all was the automotive industry. If you believe that the AIAG sponsors always do what makes sense, you might not have been paying attention for a long time.
 

Johnnymo62

Haste Makes Waste
#19
HI Guys

Long time no talk. Nice to see you all in here
Hope you are all well?

We were asked to supply R&R Anova MSA study. Never done it. I've looked online and there are many tools available to purchase.
I've got OK from the boss so can you recommend some easy to use excel spreadsheet for reasonable price?
What are you using ?

Looking forward to your replies
Hi Monica,

I've only used Minitab. My customer was Ford and they prefer it. It took me a bit of getting used to. I hear its expensive.

You can try the AIAG APQP workbook. It has an ANOVA GRR in it.
 
Last edited:

Bill Levinson

Industrial Statistician and Trainer
#20
HI Guys

Long time no talk. Nice to see you all in here
Hope you are all well?

We were asked to supply R&R Anova MSA study. Never done it. I've looked online and there are many tools available to purchase.
I've got OK from the boss so can you recommend some easy to use excel spreadsheet for reasonable price?
What are you using ?

Looking forward to your replies
It can be done in Excel as well as StatGraphics and Minitab. If the data are not confidential, I could give it a try.
 
Thread starter Similar threads Forum Replies Date
G MSA check list to audit IATF 16949 - Automotive Quality Systems Standard 4
R MSA for ATE (Automatic Test Equipment Embedded Software) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
Bev D MSA Tools Misc. Quality Assurance and Business Systems Related Topics 3
R MSA studies for Heat Treatment Equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
C Industrial scales and MSA (IATF 16949 requirement 7.1.5.1.1) IATF 16949 - Automotive Quality Systems Standard 30
M Difference between MSA and MSE? General Measurement Device and Calibration Topics 1
M Test method validation - Is MSA (MSA1, MSA2, MSA3 and linearity) a good solution? Medical Device and FDA Regulations and Standards News 1
A MSA results differences - Supplier results vs. My results Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
R Preliminary MSA and Sockets' correlation Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 0
I MSA requirement for 5 Micrometers + CP changes need customer approval? IATF 16949 - Automotive Quality Systems Standard 2
T Correct MSA study for an automated camera system which makes attribute inspection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
Emran.mi Measurement system analysis - Can you help me about implementation MSA for CMM device Manufacturing and Related Processes 2
N MSA Study for a Leak Testing Device Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 6
N Justifications for not performing MSA (Measurement System Analysis) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 6
D Do I need part variation while doing Destructive Variable Gage R&R MSA study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 19
Prashant G MSA Study - AS 9100 and and our customer want us to do MSA study for their parts dimension Reliability Analysis - Predictions, Testing and Standards 4
D Compression Spring Force MSA Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
E MSA Study on MTS dynamic rate measurements Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
A The FILE recommended by AIAG MSA 4ed for LINEARITY Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 0
S IATF 16949 Audit MSA IATF 16949 - Automotive Quality Systems Standard 10
C Do Calibration Reference Standards require an MSA study? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
A MSA When an Instrument Measures More than One Parameter Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
G MSA Type 1 - Cg and Cgk ranges Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
O MSA Study for a steel ruler Manufacturing and Related Processes 6
P IATF 16949 MSA Studies for CMM machines IATF 16949 - Automotive Quality Systems Standard 3
B 7.1.5.1.1 MSA for inline dimension vision inspection equipment IATF 16949 - Automotive Quality Systems Standard 5
G MSA on a Counting System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
C AIAG's IATF 16949 section 7.1.5.1.1 - Torque MSA Requirement Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 10
S IATF 16949 7.1.5.1.1 Measurement System Analysis (MSA - Die casting) IATF 16949 - Automotive Quality Systems Standard 5
J MSA on Part Specific Attribute Fixtures Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
G ANOVA GR&R: Minitab vs AIAG MSA results Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
S Test Method Validation or MSA for Batch Manufacturing Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
S MSA on a process using different fixtures Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
B Camera Vision System Measurement System Analysis (MSA) Reliability Analysis - Predictions, Testing and Standards 3
D Cp/Cpk on Gages for MSA (vs. Cg/Cgk) Capability, Accuracy and Stability - Processes, Machines, etc. 21
L Number of Distinct Categories" or NDC Calculation in MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 8
Q Doing an MSA on a Climatic Chamber Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
L IATF 16949 7.1.5.1.1 Measurement System Analysis (MSA) - Visual Inspection IATF 16949 - Automotive Quality Systems Standard 14
C MSA for Non Manufacturing - Design Validation, Characterization and Qualification Lab Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
C PPAP Rejection for "MSA-GR&R are not using the latest AIAG format" APQP and PPAP 9
C MSA (Measurement System Analysis) - Instrument Selection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
C MSA (Measurement System Analysis) - Dimension Selection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
P Frequency and Sample Size Requirements for an MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
A Automated Measurement Machine MSA - Small Parts Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
Crimpshrine13 MSA for a part Automated and part Manual Inspection Machine IATF 16949 - Automotive Quality Systems Standard 7
S MSA (Measurement System Analysis) - Process Tolerances Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
B How to do an MSA on an Automated Torque / Angle System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
B How to perform a MSA - GRR (Gage R&R) A&R/ANOVA Vs GUM/VDA Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
B Gage R&R MSA - Variation, Tolerance, Variance by different methods IATF 16949 - Automotive Quality Systems Standard 23
I MSA related questions Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
Similar threads


















































Top Bottom