Automated Testing Gage R&R (Repeatability and Reproducibility) Issues

A

avalon

#1
GR&R Issues in Automated Testing

Rob Nix said:
Welcome QeSmithy. :bigwave:

IMHO :frust:

Juran's QC Handbook (4th & 5th ed.) calls Cpk a "performance index". The AIAG SPC Manual (1995) calls Ppk a "performance index", and Cpk a "capability index for a stable process". So it gets a little muddied after a while (and let me say, not all that important).

Cp and Cpk are at least as old as Ishikawa's 1982 "Guide to Quality Control" and are simply the inverse of capability ratios. They applied to short and long term measures of capability. Ppk was first used in the context of new product submissions (ISIR/PPAP) and still exists on some company's warrant forms for their initial (and obviously short term) estimates of process potential. Ford Motor company was one of the first. Ford did not invent Cpk & Ppk, but they DID champion their use - with the same understanding that Cpk = long term and Ppk = short term. Ppk was used also in other earlier PPAPs for Short Term estimates of capability. However, somehow, in the last decade the roles have reversed (especially by the 6 sigma people). This is partly due to the AIAG's SPC Manual defining Ppk as a nebulous "performance index". Does that definition imply performance over time? Who knows. Somehow the issue got confused.

Interestingly, the 1999 printing of the AIAG PPAP Manual speaks of using Cpk when historical (long term) data is available - enough to divide into subgroups, and using Ppk when it is not. In any event, the general calculations are the same (sure, the estimate or calculation for standard deviation varies, to which I say, blah blah blah :bonk: ), and since it is all really a matter of semantics, if everyone agrees (especially you and your customer) then that is all that matters.

I stand by the older Cpk = long term, Ppk = short term understanding. Cpk is a more valid index for on-going process capability measurement.

PS: You'll find tons of debate on this if you do a Google search on "cpk vs ppk".

Ok, this is all well and good, however what in the world does this have to do with qualifying an automated TEST machine that has no operator influence? I fail to see how the standard AIAG methods apply to this situation. The controversy surrounding what is needed to accept a machine for shipment seems to be widespread. Typical GR&R software requires 2 or more appraisers. What if there are no appraisers? I found the following links useful:

http://www.symphonytech.com/freeware.htm

http://www.qualitymag.com/CDA/ArticleInformation/features/BNP__Features__Item/0,6425,100281,00.html (Invalid URL)
http://www.mathoptions.com/grandron.htm (Invalid URL)

I don't want to create an encyclopedia of data and meaningless charts. All I want is to demonstrate my machine will meet EV requirements. Therefore, why wouldn't the Instantaneous method be acceptable and more practical for short term analysis on automated test equipment than complicated Cpk/Ppk calculations? IMO, until the machine is in production where data can be collected over long periods, the more simplified method as described by John Raffaldi should suffice.
Also, what about uncertainty of the measurement equipment when doing GR&R? Is it a simple matter of subtracting the uncertainty of each device based on the supplier's data from the total tolerance of the test being performed? Uncertainty is another area that to me is neither well understood or explained.

I'm open for constructive criticism and instruction. Thanks.

:confused:

P.S. The customer is just as confused.
 
Last edited by a moderator:
Elsmar Forum Sponsor
R

Rob Nix

#2
Re: Capability- Short Term / Long Term?

Avalon,

There may be a bit of confusion to clear up. Process Capability (short, long, cpk, ppk) and Gage Repeatability & Reproducibility are two entirely different subjects.

My post (that you quoted) discussed Process Capability. Your question is regarding GR&R (Cpks and Ppks do not apply). We design and manufacture many special automated test machines. (NOTE: We do not do Cpk studies as there are no parts produced!) You are right: there is no operator influence. Therefore, the second "R" in GR&R, "reproducibility" does not apply. You simply have to do a repeatability study. There are many ways to get valid results. Take one or many parts, check them repeatedly, estimate the standard deviations, multiply by 5.15, and divide by the tolerance zone. Check if percentage of tolerance used is acceptable. THAT'S IT.

Well, that's not it entirely. This subject is fraught with secondary considerations and questions, such as: does the part or machine change as tests are being run? is there a tolerance to compare against, and if so, is it valid? are results significantly different throughout the measurement range? And so on. I believe there are also other threads on this forum that discuss these issues in more depth.
 
A

avalon

#3
Re: Capability- Short Term / Long Term?

Rob Nix said:
Avalon,

There may be a bit of confusion to clear up. Process Capability (short, long, cpk, ppk) and Gage Repeatability & Reproducibility are two entirely different subjects.

My post (that you quoted) discussed Process Capability. Your question is regarding GR&R (Cpks and Ppks do not apply). We design and manufacture many special automated test machines. (NOTE: We do not do Cpk studies as there are no parts produced!) You are right: there is no operator influence. Therefore, the second "R" in GR&R, "reproducibility" does not apply. You simply have to do a repeatability study. There are many ways to get valid results. Take one or many parts, check them repeatedly, estimate the standard deviations, multiply by 5.15, and divide by the tolerance zone. Check if percentage of tolerance used is acceptable. THAT'S IT.

Well, that's not it entirely. This subject is fraught with secondary considerations and questions, such as: does the part or machine change as tests are being run? is there a tolerance to compare against, and if so, is it valid? are results significantly different throughout the measurement range? And so on. I believe there are also other threads on this forum that discuss these issues in more depth.

Thanks. I was not questioning your knowledge on the subject, just that currently we have a situation where we performed a simple repeatability study on a part through a test machine. 3 parts, 10 trials. The %P/T was 10.5% which is very good. However, once at the customer's plant, they performed their own "GR&R study" and came up with these fantastic numbers that in no way resembles ours. This is not the first time this has happened, nor is it limited to one customer. We have asked for the data, but I'm suspecting their $$$ software is giving them incorrect results. They want reference material to validate our claim that all we are concerned with bottom line is repeatability. The only issue may be uncertainty (what a nightmare) of the measuring devices on the machine, but it's minor.

When we receive the data, I will crunch the numbers (wow, isn't that hard!) and I'll bet it will be OK.

Now my next question is, if there are 2 or more identical machines, AV is now involved. Would this be considered correlation? Again, I don't see AIAG models useful for this because the AV is not influenced by human intervention, but rather a comparison between machines, although it's still technically reproducibility, correct? In that case, are there quick and easy calculations? I would think so. Thanks again.
 
A

Atul Khandekar

#4
I just split this thread off the "Capability- Short Term / Long Term" thread, beacuse I think Avalon's questions deserve a separate thread of discussion on its own.
 
Thread starter Similar threads Forum Replies Date
Hami812 Gage R&R on Automated Tool that supports Pass/Fail and some Parametric Testing Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
M GR&R (Gage R&R) for Automated Electrical Testing Equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
Y Is NDC needed for gage used for ATE? Automated Test Equipment IC Testing Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 6
gramps What do you think about automated QA testing For software app industry? Misc. Quality Assurance and Business Systems Related Topics 5
S Automated regression testing software Software Quality Assurance 4
B Automated Testing - R&R study - calculation Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
J Seeking Info: Automated Torque Testing General Measurement Device and Calibration Topics 2
L Gage RandR on automated equipment. IATF 16949 - Automotive Quality Systems Standard 7
Ron Rompen MSA on automated measurement system - Multiple Step Vision System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
J Sub-supplier change from manual to automated process - same specs - Report to FDA? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
P Images of Product for Automated Process - How long to keep? US Food and Drug Administration (FDA) 2
T Correct MSA study for an automated camera system which makes attribute inspection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
M Applicability of Means of Protection, working voltage in an Automated External Defibrillator IEC 60601 - Medical Electrical Equipment Safety Standards Series 0
T Tests IEC 60601-1 - Grounding Resistance, Leakage Current and Dielectric Rigidity in an automated way IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
Ron Rompen GR&R on Automated Inspection Cell Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 15
S What are you looking for in an automated quality management system? Quality Assurance and Compliance Software Tools and Solutions 8
F Automated Platform (IVD) Classification/Registration (New to RA) EU Medical Device Regulations 3
M Is Automated Internal Auditing Possible? Internal Auditing 13
A Automated Measurement Machine MSA - Small Parts Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
T Automated and Manual Measurement System Comparison (Correlation) Quality Tools, Improvement and Analysis 3
Crimpshrine13 MSA for a part Automated and part Manual Inspection Machine IATF 16949 - Automotive Quality Systems Standard 7
B How to do an MSA on an Automated Torque / Angle System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
A Automated calibration of Digital Multimeter without communication port Calibration and Metrology Software and Hardware 2
S Bench Marking Automated Data Collection Benchmarking 4
C How to perform Gauge R & R on Automated Test & Measurement Equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 8
P Looking for recommendations for an automated vision measuring system Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
M Automated Revision Updates - Notifications of new revisions for ISO's to subscribers Document Control Systems, Procedures, Forms and Templates 2
T Tools Six Sigma for my Automated Machine Six Sigma 8
C Gage R&R on In-Process Automated Inspection Equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 13
Q 21 CFR 820.30 - Automated with Computer Software - Applicable? US Food and Drug Administration (FDA) 5
S How to calibrate AOI (Automated Optical Inspection) for IOPQ Qualification and Validation (including 21 CFR Part 11) 3
P GR&R on automated test system which produces hundreds of results Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
C Multiple Variation Gage R&R for Automated Testers Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
J Classes for Hydraulic Calibration - Dead Weight Tester or Automated Controller General Measurement Device and Calibration Topics 1
Q Automated Test Software and System Validation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
L Automated Calculators like EXCEL sheets for different processes Quality Assurance and Compliance Software Tools and Solutions 2
L Dry Lube Application (Ever lube 620) - Manual Application vs. Automated Spray Process Manufacturing and Related Processes 1
E MSA (Measurement System Analysis) Study on an Automated Process Quality Tools, Improvement and Analysis 4
G Automated Production Line Quandary... FMEA and Control Plans 1
R Use of Automated Liquid Handling Robot for a PCR based Assay CE Marking (Conformité Européene) / CB Scheme 2
D Automated Vision Systems - Cost, Value and Payback of Implementation Statistical Analysis Tools, Techniques and SPC 2
I Gage R&R for Automated Measurement Equipment Systems Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
A Classification of Automated Gait Training Device for Rehabilitation EU Medical Device Regulations 5
R GAMP (Good Automated Manufacturing Practice) simplified please. Quality Manager and Management Related Issues 3
G Is Agilent?s automated network analyzer verification ISO/IEC-17025 compliant? ISO 17025 related Discussions 4
T Gage Repeatability for an Automated Leak Tester Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 10
sagai Automated System's Signature on Electronic Record - PART11 Qualification and Validation (including 21 CFR Part 11) 4
C Multiple Fixture Automated Vision System GR&R (Gage R&R) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
B Automated paint process control on conveyerized lines - Million dollar question Capability, Accuracy and Stability - Processes, Machines, etc. 14
Q Validating the AOI process (Automated Optical Inspection) Reliability Analysis - Predictions, Testing and Standards 9

Similar threads

Top Bottom