Decision Rules When Taking Measurment of Uncertainty into Account with Specifications

J

jfgunn

#1
I have reviewed a couple of threads on this topic and good responses from MichelleKay and Wesley Richardson. I will explain this issue, my plan, and then ask some questions about how others have handled the same issue....

Currently, we are an ISO 17025 accredited commercial lab for many different things. For most of the items we currently calibrate, uncertainty is not a huge factor. We currently state that measuremetn of uncertainty is not taken into account in Pass/Fail Results. We take the reading we get and compare it to the tolerance. If it is good, it passes, if not it fails.

We recently purchased a P&W LabMaster Universal and made some major lab improvements to properly control temperature and humidity. This gives us the ability to calibrate gage blocks and ring gages (and a few other things).

Since we are now nearing a state of the art limitation where uncertainty can be equal to the tolerance of the gage we are testing, I have begun to look at better defining our policy in regards to Pass/Fail decisions. For guidance, I went to ASME B89.7.3 .1-2001. Based upon this document there are basically three options I have (as well as some variations on these options):

1.) The specification of the UUT is the acceptance zone, ignore uncertainty as long as we meet a 2:1 or 3:1 or 4:1 TUR.

2.) Relaxed Acceptance/Stringent Rejection: The acceptance zone equals tolerance PLUS uncertainty. We only reject something if we know it is bad (and by "know" I mean we are 95% sure as that is the confidence interval of the uncertainty).

3.) Stringent Acceptance/Relaxed Rejection: The acceptance zone equals tolerance MINUS uncertainty. We only accept something if we know it is good. Any customer I have that needs this service should be directed to use the most accurate lab they can find for each individual item.

Number 1 and 3 are OK except for items where the uncertainty is relatively large (ie Gage blocks and Class X, XX, XXX Ring and Plug Gages). I feel like I would be telling my customers that they should replace things when I really do not know if that is the case.

Number 2 is Ok for the gage blocks, rings, and plugs except I may be passing things that might be bad. :bonk:

Of course, the 4th option would be to report the measurement with its uncertainty and make no statement as to a compliance to a specification. This would be great, but does not provide a customer with value.

My plan is to do the following in all quotes to my customers: Use number 1 above for almost everything except for gage blocks, Class X, XX, XX Plug and Ring Gages where I will use number 2. Any certificate of calibration that indicates a Pass where the Relaxed Acceptance/Stringent Rejection Policy was used will have a remark to that effect.

Does this seems reasonable?

Is is sufficient to bury this decision rule in a quote? Should I obtain a more formal written approval to use the rule for each customer?

Has anyone ran into problems when trying to implement decision rules with customers? I picture a new customer asking me why I am making this so difficult when his current provider has never asked these questions. I am always happy to explain what I am doing (or not doing) and what my competitiion is doing (or not doing).

I have an audit on May 2nd-4th to add all of these items. Already passed Gage Block Proficiency Test and I am well on my way to completing all of the uncertainty budgets. This is one of the last issues I need to address. Any help would be apprecaited.

Joe
 
Elsmar Forum Sponsor

BradM

Staff member
Admin
#2
Re: Decision Rules When Taking Measurment of Uncertainty into Account with Specificat

First, let me state that I am not an expert with IS0 17025, and that I do not operate a commercial laboratory. My two cents worth would be from somebody who utilizes labs.

Currently, we are an ISO 17025 accredited commercial lab for many different things. For most of the items we currently calibrate, uncertainty is not a huge factor. We currently state that measurement of uncertainty is not taken into account in Pass/Fail Results. We take the reading we get and compare it to the tolerance. If it is good, it passes, if not it fails.

Joe
Not sure I understand the value of the pass/fail. Why not approach all measurements the same? Here's the uncertainty, here's what we did, and here's what we found.


Number 2 is Ok for the gage blocks, rings, and plugs except I may be passing things that might be bad.

Joe
When working with ratios and confidence intervals, it is entirely possible to make a type II error (accepting when you should reject)

Of course, the 4th option would be to report the measurement with its uncertainty and make no statement as to a compliance to a specification. This would be great, but does not provide a customer with value.

Joe
My opinion: I'm not sure I would entirely agree with this. Your customer may find more value with consistent/ defensible documentation. Personally, many times that is all that I want, as I have established other acceptance tolerances independent of the MFG. tolerance. Ideally I would like to have a correct pass/fail assessment by the calibration source, but sometimes we have to work together for my best interests.

Does this seems reasonable?

Is is sufficient to bury this decision rule in a quote? Should I obtain a more formal written approval to use the rule for each customer?

Has anyone ran into problems when trying to implement decision rules with customers? I picture a new customer asking me why I am making this so difficult when his current provider has never asked these questions. I am always happy to explain what I am doing (or not doing) and what my competitiion is doing (or not doing).

Joe
In the end, this will be a business decision you will need to make. Personally, I would rather do it right and consistent, and explain to my customer what I am doing. If the customer does not understand your approach, they should ask, or you can take the initiative to explain it to them (that's value your competitor is not giving them). Or, the customer don't care what's on the sheet, and it won't matter anyway.
 
Last edited:

BradM

Staff member
Admin
#3
Re: Decision Rules When Taking Measurment of Uncertainty into Account with Specificat

Joe, thank you for the awesome post! I know you are asking some questions, but I ended up learning and getting some good ideas from your post.

That level of thought and detail is greatly appreciated.

It is always helpful hearing from individuals actually in the trenches like yourself running calibration labs. Please visit whenever you can.
 

Hershal

Metrologist-Auditor
Staff member
Super Moderator
#4
Re: Decision Rules When Taking Measurment of Uncertainty into Account with Specificat

Generally, there are three requirements for reporting uncertainty.....four actually for a cal lab.....

1. Customer wants it.
2. Regulatory requirement for it.
3. Uncertainty will push the results into failure region.

And for calibration:

4. Certain types of calibrations CANNOT rely on the ratio (e.g., 4:1), specifically scales/balances and a few others.

BUT.....let me stress this.....get in touch with your accrediting body and verify their requirements also......

Hershal
 
Thread starter Similar threads Forum Replies Date
J Rules for Achieving IATF Recognition 3rd Edition FAQ?s - Certification Decision 5.12 IATF 16949 - Automotive Quality Systems Standard 2
S What decision rules should be applied for x bar-R chart? Statistical Analysis Tools, Techniques and SPC 10
dgrainger New decision by MHRA on disclaimers - addition of "manner of use as perceived by the consumer" UK Medical Device Regulations 1
M Clinical Decision Support Software Question 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
J Need Change Control Yes/No Decision Tree Template ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
Steve Prevette Informational I am presenting a webinar Thursday - "Data Driven Decision Making" - 19 November 2020 Statistical Analysis Tools, Techniques and SPC 5
J Masters Degree - Advice about a decision I"m going to be making Career and Occupation Discussions 9
A Medical Device Vigilance decision tree for Japan for class 2 devices. Japan Medical Device Regulations 1
M Informational US FDA Draft Guidance – Clinical Decision Support Software Medical Device and FDA Regulations and Standards News 0
M Informational EU – Commission Implementing Decision (EU) as regards the designation of expert panels in the field of medical devices Medical Device and FDA Regulations and Standards News 0
M Informational Commission Implementing Decision (EU) 2019/939 of 6 June 2019 designating issuing entities designated to operate a system for the assignment of Unique Medical Device and FDA Regulations and Standards News 0
M Informational US FDA – Patient-Reported Outcomes (PROs) in Medical Device Decision Making Medical Device and FDA Regulations and Standards News 0
M Informational US FDA – Patient Preference Information (PPI) in Medical Device Decision-Making Medical Device and FDA Regulations and Standards News 0
J Medical Device Vigilance decision tree FORM - Please share yours EU Medical Device Regulations 15
O Reinstating a disqualified supplier—questions to ask before making a decision Supplier Quality Assurance and other Supplier Issues 2
P Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
M IATF 16949 Cl. 8.7.1.4 - Risk analysis for decision making about rework IATF 16949 - Automotive Quality Systems Standard 2
L A Complete Control Chart Decision Tree Statistical Analysis Tools, Techniques and SPC 5
C IATF 16949 Cl. 8.4.1.2 d) Multidisciplinary Decision Making IATF 16949 - Automotive Quality Systems Standard 2
T Policy or Good Guidance for a Traceability Decision Tree (Serial and Batch Numbers) Misc. Quality Assurance and Business Systems Related Topics 4
somashekar Uptodate.com: A clinical decision support resource - Point-of-care decisions Medical Information Technology, Medical Software and Health Informatics 1
N Decision to file a new 510(k) - ETO (Ethylene Oxide) Sterilization Other US Medical Device Regulations 4
Chennaiite Can an Auditor question Top Management decision to approve deviation? IATF 16949 - Automotive Quality Systems Standard 30
J Decision Tree using for CCP (Critical Control Points) Food Safety - ISO 22000, HACCP (21 CFR 120) 2
AnaMariaVR2 Structured Approach to Benefit-Risk Assessment in Drug Regulatory Decision-Making Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 0
S Supplier Disapproval Decision Process Federal Aviation Administration (FAA) Standards and Requirements 2
N Decision Making... (joke) Funny Stuff - Jokes and Humour 1
R Risk Analysis and Hazard Identification concerning Clinical Decision Support Systems ISO 14971 - Medical Device Risk Management 1
P CMC Decision 3 - Manufacturers Address Content Requirements EU Medical Device Regulations 4
Sidney Vianna AS91X0 Transition Deadline - New IAQG Decision - 2012 AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 54
L Decision Matrix to help Operator driven Continuous Improvement Lean in Manufacturing and Service Industries 3
F ISO 11137 - Flow Chart | Decision Tree sterilization Other Medical Device Related Standards 4
Q Exactly which are the records needed by ISO9001? or is it my decision? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
K Seeking help in making a decision - Project behind schedule Quality Manager and Management Related Issues 11
R EU Medical Device Change Decision Tree (when to notify the NB) EU Medical Device Regulations 4
Z Is adoption of ISO 9001 a strategic decision? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 24
S Federal Register - Software Reclassification (MDDS) - Final Decision by FDA yet? Other US Medical Device Regulations 3
A Special 510k Update - Cease Sales and Distribution until a Decision is Rendered? Other US Medical Device Regulations 11
M Transforming a Decision Tree into Excel Excel .xls Spreadsheet Templates and Tools 7
L Can one question a 510(k) decision? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
Manix The Toughest Decision I have ever had to make! Director or Quality Engineer Career and Occupation Discussions 16
T Change Control Decision Tree - When medical devices that are approved with a PMA Document Control Systems, Procedures, Forms and Templates 9
D PA, CA and Risk-Based Decision Making - Need Input Preventive Action and Continuous Improvement 7
P This Is It - Tomorrow is the final decision - Finished the certification audit ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
D Flow Chart Decision Symbol Arrows - Which flowchart is correct? Process Maps, Process Mapping and Turtle Diagrams 32
C Is Risk Based Decision Making part of Preventive Action Preventive Action and Continuous Improvement 5
P MSA Study Types - How to make an informed decision about a gauge Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 18
C Need guide to make judgement for decision. Career and Occupation Discussions 5
T Omitting Decision to Quote Procedure - Small engineering consultancy ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
C Supplier Qualification/Selection and Decision Methods Supplier Quality Assurance and other Supplier Issues 1

Similar threads

Top Bottom