Decision Rules When Taking Measurment of Uncertainty into Account with Specifications

J

jfgunn

I have reviewed a couple of threads on this topic and good responses from MichelleKay and Wesley Richardson. I will explain this issue, my plan, and then ask some questions about how others have handled the same issue....

Currently, we are an ISO 17025 accredited commercial lab for many different things. For most of the items we currently calibrate, uncertainty is not a huge factor. We currently state that measuremetn of uncertainty is not taken into account in Pass/Fail Results. We take the reading we get and compare it to the tolerance. If it is good, it passes, if not it fails.

We recently purchased a P&W LabMaster Universal and made some major lab improvements to properly control temperature and humidity. This gives us the ability to calibrate gage blocks and ring gages (and a few other things).

Since we are now nearing a state of the art limitation where uncertainty can be equal to the tolerance of the gage we are testing, I have begun to look at better defining our policy in regards to Pass/Fail decisions. For guidance, I went to ASME B89.7.3 .1-2001. Based upon this document there are basically three options I have (as well as some variations on these options):

1.) The specification of the UUT is the acceptance zone, ignore uncertainty as long as we meet a 2:1 or 3:1 or 4:1 TUR.

2.) Relaxed Acceptance/Stringent Rejection: The acceptance zone equals tolerance PLUS uncertainty. We only reject something if we know it is bad (and by "know" I mean we are 95% sure as that is the confidence interval of the uncertainty).

3.) Stringent Acceptance/Relaxed Rejection: The acceptance zone equals tolerance MINUS uncertainty. We only accept something if we know it is good. Any customer I have that needs this service should be directed to use the most accurate lab they can find for each individual item.

Number 1 and 3 are OK except for items where the uncertainty is relatively large (ie Gage blocks and Class X, XX, XXX Ring and Plug Gages). I feel like I would be telling my customers that they should replace things when I really do not know if that is the case.

Number 2 is Ok for the gage blocks, rings, and plugs except I may be passing things that might be bad. :bonk:

Of course, the 4th option would be to report the measurement with its uncertainty and make no statement as to a compliance to a specification. This would be great, but does not provide a customer with value.

My plan is to do the following in all quotes to my customers: Use number 1 above for almost everything except for gage blocks, Class X, XX, XX Plug and Ring Gages where I will use number 2. Any certificate of calibration that indicates a Pass where the Relaxed Acceptance/Stringent Rejection Policy was used will have a remark to that effect.

Does this seems reasonable?

Is is sufficient to bury this decision rule in a quote? Should I obtain a more formal written approval to use the rule for each customer?

Has anyone ran into problems when trying to implement decision rules with customers? I picture a new customer asking me why I am making this so difficult when his current provider has never asked these questions. I am always happy to explain what I am doing (or not doing) and what my competitiion is doing (or not doing).

I have an audit on May 2nd-4th to add all of these items. Already passed Gage Block Proficiency Test and I am well on my way to completing all of the uncertainty budgets. This is one of the last issues I need to address. Any help would be apprecaited.

Joe
 

BradM

Leader
Admin
Re: Decision Rules When Taking Measurment of Uncertainty into Account with Specificat

First, let me state that I am not an expert with IS0 17025, and that I do not operate a commercial laboratory. My two cents worth would be from somebody who utilizes labs.

Currently, we are an ISO 17025 accredited commercial lab for many different things. For most of the items we currently calibrate, uncertainty is not a huge factor. We currently state that measurement of uncertainty is not taken into account in Pass/Fail Results. We take the reading we get and compare it to the tolerance. If it is good, it passes, if not it fails.

Joe

Not sure I understand the value of the pass/fail. Why not approach all measurements the same? Here's the uncertainty, here's what we did, and here's what we found.


Number 2 is Ok for the gage blocks, rings, and plugs except I may be passing things that might be bad.

Joe

When working with ratios and confidence intervals, it is entirely possible to make a type II error (accepting when you should reject)

Of course, the 4th option would be to report the measurement with its uncertainty and make no statement as to a compliance to a specification. This would be great, but does not provide a customer with value.

Joe

My opinion: I'm not sure I would entirely agree with this. Your customer may find more value with consistent/ defensible documentation. Personally, many times that is all that I want, as I have established other acceptance tolerances independent of the MFG. tolerance. Ideally I would like to have a correct pass/fail assessment by the calibration source, but sometimes we have to work together for my best interests.

Does this seems reasonable?

Is is sufficient to bury this decision rule in a quote? Should I obtain a more formal written approval to use the rule for each customer?

Has anyone ran into problems when trying to implement decision rules with customers? I picture a new customer asking me why I am making this so difficult when his current provider has never asked these questions. I am always happy to explain what I am doing (or not doing) and what my competitiion is doing (or not doing).

Joe

In the end, this will be a business decision you will need to make. Personally, I would rather do it right and consistent, and explain to my customer what I am doing. If the customer does not understand your approach, they should ask, or you can take the initiative to explain it to them (that's value your competitor is not giving them). Or, the customer don't care what's on the sheet, and it won't matter anyway.
 
Last edited:

BradM

Leader
Admin
Re: Decision Rules When Taking Measurment of Uncertainty into Account with Specificat

Joe, thank you for the awesome post! I know you are asking some questions, but I ended up learning and getting some good ideas from your post.

That level of thought and detail is greatly appreciated.

It is always helpful hearing from individuals actually in the trenches like yourself running calibration labs. Please visit whenever you can.
 

Hershal

Metrologist-Auditor
Trusted Information Resource
Re: Decision Rules When Taking Measurment of Uncertainty into Account with Specificat

Generally, there are three requirements for reporting uncertainty.....four actually for a cal lab.....

1. Customer wants it.
2. Regulatory requirement for it.
3. Uncertainty will push the results into failure region.

And for calibration:

4. Certain types of calibrations CANNOT rely on the ratio (e.g., 4:1), specifically scales/balances and a few others.

BUT.....let me stress this.....get in touch with your accrediting body and verify their requirements also......

Hershal
 
Top Bottom