Determining Repeatability without Gage R&R

M

Muegge

#1
Hi. This is my first thread on Elsmar and I hope I have posted it in the correct area. My question is as follows:

I work in an FDA regulated industry and am responsible for validating our measurement equipment used to verify our assembly processes. I should note most of our measurement equipment is custom developed. In our validation efforts, we normally do Gage R&R and use AIAG recommendations for our specifications (less than 10% for our critical quality tools and less than 30% for the others). This works for us for most measurement systems.

There are instances where because of the design of the measurement equipment, the operator will absolutely have no affect on the outcome of the results. As such the reproducibility is 0%. We would like to be smarter with our test designs for these systems and not use multiple operators, multiple trials when we know going into it they will have no influence on the outcome of the results. The question then is what is the best way to do a repeatability study for this situation...and more importantly what specifications should be set. Because we are in a highly regulated industry, we will need to reference the source of the specifications (like we have for GR&R and AIAG) and justify them.

I have probably missed a few key details, so I will be happy to fill them in as others point them out to me. Thanks in advance for the help, and I know I am not alone in praising this forum.
 
Elsmar Forum Sponsor
A

arios

#2
Re: Question about determining repeatability without Gage R&R

A question just for curiosity: Why are you using AIAG's manuals on a FDA regulated company?

No that you can't because AIAG has good information, but I have the impression that it can bring further complexity to your system.
 

bobdoering

Stop X-bar/R Madness!!
Trusted Information Resource
#4
I work in an FDA regulated industry and am responsible for validating our measurement equipment used to verify our assembly processes. I should note most of our measurement equipment is custom developed. In our validation efforts, we normally do Gage R&R and use AIAG recommendations for our specifications (less than 10% for our critical quality tools and less than 30% for the others). This works for us for most measurement systems.

There are instances where because of the design of the measurement equipment, the operator will absolutely have no affect on the outcome of the results. As such the reproducibility is 0%. We would like to be smarter with our test designs for these systems and not use multiple operators, multiple trials when we know going into it they will have no influence on the outcome of the results. The question then is what is the best way to do a repeatability study for this situation...and more importantly what specifications should be set. Because we are in a highly regulated industry, we will need to reference the source of the specifications (like we have for GR&R and AIAG) and justify them.
You can run a Gage R, or if it has several nests that present the part to the gage, you can use individual nests as "operators'. Another option is to run it at several points in time - to capture any difference in conditions...if any.
 
M

Muegge

#5
Thanks Bobdoering. I understand how to using the nests as operators, are you suggesting to also use the different time points as 'operators' if using that method? If so is this method of substituting operators for another factor a popular method?
 

bobdoering

Stop X-bar/R Madness!!
Trusted Information Resource
#6
Thanks Bobdoering. I understand how to using the nests as operators, are you suggesting to also use the different time points as 'operators' if using that method? If so is this method of substituting operators for another factor a popular method?
I am suggesting using different points in time as an alternative to using different nests, in the event there is only one nest. It is referred to as "between-conditions". It really only makes sense if there is any type of issue that can occur at different points in time for the measuring device - change in room temp, humidity, light, etc.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#7
I have attached a 'generic' copy of the statistical guidelines my organization uses for measurement system V&V. (We are a veterinary medical device company) The document includes references to published articles, many of which are peer reviewed and are accepted statistical practices by the FDA and USDA.

It is important to note two things about the AIAG method
  • The use of 10%, 20% and 30% measurement error to tolerance categories are NOT scientifically or statistically derived. They are in fact popularly accepted "rules of thumb" that are also statistically incorrect in the manner in which the %contribution of measurement error is calculated.
  • The sample sizes utilized are statistically under-powered; you are much better served by using two repeated measures across 30 parts.

Remember - the mere fact of publication doesn't imply truth.
 

Attachments

A

arios

#8
Arios:
Do you have another recommendation?
Not that I had a particular recommendation, but it was a little odd to me seing the combination that you are using or proposing. The FDAs QSR does not prescribe particular techniques.

This is the requirement of the QSR

21 CFR 820.250
(a) Where appropriate, each manufacturer shall establish and maintain procedures for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics.

It brought my attention to see that you are using an automotive standard for a medical device application. Being an standard it should many "shalls", so my only concern is how you will handle those automotive requirements on a medical device sector. I understand tools and techniques can have a vast an valid application accross industry sectors, however I can also see a possible disanvangte when it comes to implementation from a compliance perspective, not from a practical or technical point of view and MSA should be an strong manual, no doubt about it.

So, How are planning to combine both? If it can be known. I imagine other Cove members can also let us hear from their experience.

And I reiterate, I am not opposed to see the combination or "importation" of techniques that could be of universal use, I am only curious about the use of manuals from the Automotive sector on the Medical device arena.
 

bobdoering

Stop X-bar/R Madness!!
Trusted Information Resource
#9
It brought my attention to see that you are using an automotive standard for a medical device application. Being an standard it should many "shalls", so my only concern is how you will handle those automotive requirements on a medical device sector. I understand tools and techniques can have a vast an valid application across industry sectors, however I can also see a possible disadvantage when it comes to implementation from a compliance perspective, not from a practical or technical point of view and MSA should be an strong manual, no doubt about it.
It is not a standard, but a reference, as would be any book on MSA. Any shalls associated with the AIAG MSA are in other documents - mostly customer specific or TS interpretations - but it is not a standard, nor is it written as one. The fact that it is held in high esteem in automotive does not in and of itself create a problem using it in other sectors. Now, whether you agree with the techniques or math utilized is another issue - and that is an issue with all MSA, SPC and statistical application books and articles - so that should not be an issue unique to the AIAG MSA Book. By the way, I highly recommend the list of between and within errors discussed in the gage R&R section - you should have a copy of them sitting in front of you while you observe a gage R&R study....as one should do when performing them.
 
A

arios

#10
It is not a standard, but a reference, as would be any book on MSA. Any shalls associated with the AIAG MSA are in other documents - mostly customer specific or TS interpretations - but it is not a standard, nor is it written as one. The fact that it is held in high esteem in automotive does not in and of itself create a problem using it in other sectors. Now, whether you agree with the techniques or math utilized is another issue - and that is an issue with all MSA, SPC and statistical application books and articles - so that should not be an issue unique to the AIAG MSA Book. By the way, I highly recommend the list of between and within errors discussed in the gage R&R section - you should have a copy of them sitting in front of you while you observe a gage R&R study....as one should do when performing them.
Thanks Bobdoering! :thanx:
 
Thread starter Similar threads Forum Replies Date
D Determining Gauge R&R (Repeatability and Reproducibility) Interval Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
S Determining the requirements for the products and services ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
Q Determining Adverse Effects of Corrective/Preventive Actions ISO 13485:2016 - Medical Device Quality Management Systems 2
S Determining sample sizes for PQ Qualification and Validation (including 21 CFR Part 11) 4
M Determining if an Insulin Pen Testing Machine is a Medical Device? EU Medical Device Regulations 4
M Determining a tolerance value for Measuring devices in-house inspection General Measurement Device and Calibration Topics 12
D Determining the the maximum number of reprocessing cycles of attachments CE Marking (Conformité Européene) / CB Scheme 2
R Determining Uncertainty from Gage R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
A IATF 16949 4.3.1 - Determining the scope of the quality management system - supplemental IATF 16949 - Automotive Quality Systems Standard 9
D ISO 9001:2015 4.3 Determining the Scope of the QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
J Determining SPC tolerance Statistical Analysis Tools, Techniques and SPC 21
B Determining SAT Offsets vs TUS Offsets per SAE AMS 2750E AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 0
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
R Question on determining defective units - I am not recording fixture to part rejected Statistical Analysis Tools, Techniques and SPC 5
S Clause 8.2.2 Determining the requirements for products and services ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Determining Calibration Frequency schedule for items used in production Manufacturing and Related Processes 2
C Determining if Maintenance Contractor is an External Service subject to ISO 9001 Clause 8.4 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 43
S AS9100D PEAR - Examples for organization's method for determining process results? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
I Determining Calibration Tolerance of a Measurement Device General Measurement Device and Calibration Topics 2
J ISO 17025 Documented Procedure for 6.2.5 - Determining competency ISO 17025 related Discussions 4
V Determining FDA 820 (registration) vs ISO 13485 - Supplier gives us the kit ISO 13485:2016 - Medical Device Quality Management Systems 1
J ISO 9001 8.4.1 - Determining controls applied to externally provided processes ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
E Determining what is good and what is bad can be subjective - when is it a quality issue? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
F Determining what type of scrap to include in my internal PPM calculation Quality Manager and Management Related Issues 5
M Determining number of employees within the "Scope" of the QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
A Determining the Scope of the QMS during Stage 1? Registrars and Notified Bodies 11
W Minor Audit Nonconformance Against Determining the scope of QMS IATF 16949 - Automotive Quality Systems Standard 12
D Determining Critical Components for conformity with IEC 60601-1 IEC 60601 - Medical Electrical Equipment Safety Standards Series 21
Q ISO 9001, section 4.3 Determining the scope of our QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
A What does 8.2.2.1, Determining the requirements related to products and services,mean IATF 16949 - Automotive Quality Systems Standard 1
A Determining Retention Period for Medical Device QMS documents Document Control Systems, Procedures, Forms and Templates 5
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
F ISO 9001:2015 4.3 - Determining the scope of the quality management system ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
B Standards Needed In House - Determining what standards are applicable Various Other Specifications, Standards, and related Requirements 3
W Determining Medical Device Classification in Mexico Other Medical Device Regulations World-Wide 5
K Determining Effect of Failure without a DFMEA (Design FMEA) FMEA and Control Plans 1
W Determining the Status and Importance of the Processes and Areas to be Audited Internal Auditing 7
T Determining Customer Requirements for the US Postal Service (USPS) IATF 16949 - Automotive Quality Systems Standard 4
Crimpshrine13 Rules of achieving and maintaining IATF recognition - Determining audit days IATF 16949 - Automotive Quality Systems Standard 2
R Developing procedure for Determining Company's Context And Interested Parties ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 53
W Determining Asset Life or Depreciation Life for M&TE General Measurement Device and Calibration Topics 1
N Reason for determining no adverse effect on reworked product ISO 13485:2016 - Medical Device Quality Management Systems 8
N Procedure for determining pinhole position in condom Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
alonFAI Determining PCBA Xray Test Sampling Size Statistical Analysis Tools, Techniques and SPC 1
S Determining sample size for inspection to achieve x% confidence re defects Misc. Quality Assurance and Business Systems Related Topics 10
S Determining Sample Size - AQL & LQ AQL - Acceptable Quality Level 10
R Determining the feasibility/scope for FDA IQ/OQ against a very simple application US Food and Drug Administration (FDA) 2
B Determining the "storage volume" (ISO 8536-8:2004 clause 6.14) Other Medical Device Related Standards 4

Similar threads

Top Bottom