Organization: Disorganised Inc.
Newsgroups: misc.industry.quality
Subject: Measurement uncertainty
Date: Sat, 15 Apr 2000 17:06:30 GMT

We regularly measure the effects of certain treatments on textile materials and fabrics.

These effects are evaluated by a panel and the result is a discrete value on a scale, say 0 to 5 (or in another range).

Our auditor requires uncertainty of measurement for all of our tests. I have doubts whether the concept makes sense in this context, but apparantly that is not a discussion that our auditor is prepared to enter in. He wants UM, point.

Any hints or ideas ?

TIA

Johan

----------------

Newsgroups: misc.industry.quality
Subject: Re: Measurement uncertainty
Organization: WebUseNet Corp
Date: Sun, 16 Apr 2000 08:36:41 -0400

If you are not performing a quantitative measurement then the
measurement uncertainty will be null and void. It sounds as if what
you are doing is more of a survey to say. Without a measureable unit
i.e. ohms, liters, etc you can not do this. I would go around the
auditor and call someone else at the same agency to get assistance.
Remember, the auditors are folks just like us and may have been
instructed that "EVERYTHING" must have an uncertainty when in fact
some tests are simply for mild confidence.

---------------------

Newsgroups: misc.industry.quality
Subject: Re: Measurement uncertainty
Date: Sun, 16 Apr 2000 15:41:56 GMT

Greetings Johan,

I think the concept of uncertainty does make sense in this context. Why don't you just go ahead and measure the repeatability etc. of the measurement. You will probably find that the rating scale is more variable than a physical measurement would be.

Do this experiment. Select 6-7 colored samples that cover your range of products. Cut each sample in half so that you have two identical samples of seven products. (The exact number of products and samples is not important.)

Select your panel members and others as testers to perform the experiment. Assign code numbers to the samples so that the testers cannot tell which samples are pairs. Let the testers see only one sample at a time. Don't let them see each other do the test, and don't let them collaborate to achieve agreement. Randomize the order of testing.

Perform the complete test under standardized lighting conditions on three different days at least one week apart. Let one day be early morning, next at mid-day, and last late in the day. (Eyeballs vary.)

Analyze the individual test scores (not averages) with analysis of variance. Determine the size of these sources of variability:

variability between identical samples (within day, product, tester)

variability between days (within product, tester)

variability between testers (within day, product, sample)

Report the means and standard deviations that describe each component of variability. Do not round the results to discrete values.

Perform hypothesis tests for each component of variance. This will show which components are probably real. If you need information on hypotheses testing see the message on this forum by Kelly Speiser, Subject: "Statistical Hypothesis Testing". She has a book on hypothesis testing.

When you finish, you will have better knowledge of your test method. This will lead to improvements.

And you will exceed the expectations of your auditor.

If you want to go the next step and make the accept/reject color decisions statistically, see my website at: www.samplingplans.com/colormeasurement.htm

The method shown there appplies to rating scale color judgements as well as physical measurements.

Sincerely, Stan Hilliard
CQE,CQR,CQA,PE

------------------------

Newsgroups: misc.industry.quality
Subject: Re: Measurement uncertainty
Date: 16 Apr 2000 23:31:37 GMT

Yes, I agree with everyone that this auditor is wacky in asking measurement uncertainty.

The ISO wording (section 4.11.1 para 2) Inspection, measuring, and test equipment shall be used in a manner which ensures that the measurement uncertainty is known and is consistent with the required measurement capability.

I would also be curious if this were a required inspection you are doing versus something that you just test for process control. If it is a process control item and there is other equipment or such other inspections that take place which would verify the textile as meeting the specified requirements... then calibration is not required.

I believe what the auditor is doing mixing an inspection item concept and a workmanship criteria item together. And since the auditor was unwilling to discuss, their ignorance on the subject was obvious.

An auditor under the scope of 10011 is supposed to have documented objective evidence of what they are discussing and how the standard applies as well as what makes up the nonconformance. The nonconformance also has to be of a benefit to your company (it's in the standard folks).

Obviously they bungled their finding and I would challenge it with their appeals process and make them demonstrate what it is they are trying to enforce.

Slan Leat!

Phil McManus

Johan,
You state that you 'measure' effects. I think this gets you off to a bad start since, to measure implies an objective judgement. Yet yours appears to be totally subjective. So I suggest you drop the word measure and replace it with something like 'assign a subjective value' or whatever.
If the auditor still requires you to apply measurement uncertainty then tell him/her that if you had the equipment to evaluate all the variables associated with turning what are generally considered to be subjective assessments, into scientifically valid objective visual and mental tests, then you would use that equipment directly and not via that roundabout route but, you don't have the equipment and, as far as you know, it does't exist.
Stan,
Be vary careful when suggesting methods to meet unreasonable expectations because, whether or not they are practicable and effective, the auditor will be using them to claim that he has seen a method of achieving this and will be asking for more of it in the future, convinced that some people actually do it.
rgds, John C

Johan,
I'd say you're already one step ahead of your auditor. Your organisation must have already identified some measurement uncertainty in order to make the decision to use a panel to do the evaluations. Surely the purpose of measuring the uncertainty is so you can either allow for it or improve the measuring system. You have taken the second approach and gone for a group decision to lessen the subjectivity.
I'd definitely agree with John C from Cork, don't placate auditors unless it's absolutely necessary.
If there's no alternative with your man, then do a simple attribute MSA (see the shiny blue QS9000 book)first with single inspectors then with the group, proving that the group has less variation in the measurement.

Cheers

Brian

i've been tryin to locate the word "measurement uncertainty" in the iso9k2k and i cannot find one.

if it really is not there, what was the reason for its deletion? is it still required in this new revision?

anybody? thanks.

ISO 9001:2000 does not specifically address measurement uncertainty, but as with QS-9000 there is an expectation that you understand measurement systems and account for measurement uncertainty 'where appropriate'.

In the old standard, the phrase "measurement uncertainty is known" could have and probably has driven some companies to calculate mesurement uncertainty for all devices (usually in the form of Gage R&R). The real world is dominated by over selection of instrument capabilities. For example, choosing an instrument that can resolved one more decimal place than the tolerance applied to the specification - otherwise known as the 10:1 rule. In that case, the measurement uncertainty isn't known exactly, however, the capability is known to exceed the need. In ISO 9001:2000, the governing words are "ensure valid results". The result is much less confusion and unnecessary work.

Do we really need to do this?

I'm a supervisor of a calibration lab within a company. We calibrate ~ 97% of all our equipment in house. Our standards are sent out either to the mfg or to an approved vendor. Some of the other standards are calibrated in house.

I was thinking that it might be a good idea to do measurement uncertainties on the calibrations we do for our standards, but not for other instruments like calipers, micrometers, etc. that are used by mfg. Thoughts?

Most of the measurement uncertainty stuff I've read, including on this board, gives me a headache! :bonk: I have a decent Excel spreadsheet I use for reporting this, but I'm still not sure if I'm doing this right. For example:

I have an Omega Thermocouple Calibrator that does RTD simulation. It has an accuracy of ±.18°C, and a resolution of .1°C.

We calibrate this Omega Thermocouple Calibrator using the following equipment:
RTD Simulator with an accuracy of ±.005% Resolution:.001 ohm
Keithley 2000 DMM accuracy: 30ppm + 30ppm of range resolution .0001 ohm.

After performing the calculations, I get an Expanded Uncertainty of .115874. My questions is: should I be including the resolution of the Omega Thermocouple Calibrator in this? If I leave it out, i get .009662. Remember, the Omega Thermocouple Calibrator is the device I'm calibrating. It seems like it would be OK to include the resolution of the Omega Thermocouple Calibrator as the Expanded Uncertainty is less than the ±.18°C spec for this unit of measure.

Any thoughs are greatly appreciated!

Since it is a temp calibrator, if you are generating output, the calibrator's resolution is not a factor in it's accuracy. For example if you put out 100 degrees C into a UUT. You set the calibrator for 100.0. The calibrator's resolution is not a factor. It is spec'd at +/-0.18 deg C. Therefore when you apply that 100 deg C output to the UUT, you know that the 100 C applied is within +/-0.18 degrees of nominal. The only limiter is that if you want to fine adjust the calibrator output until the UUT reaches a nominal, you are limited to 0.1 degree increments.

Resolution is not a contributor to variability. In the analogue days, when you had to read a meter, parallax error was always a factor. But the inception of digits gave us a meter reader who took out parallax error. If, for example you have a 0.1 degree resolution on a meter. If you measure a 100 ohm resistor on that meter. If you make 1000 measurements over a course of a month, and on every reading, the meter reads 100.0, that digit of resolution did not impact variability. If theoretically you were able to see those smaller digits of resolution not present, there would be variation at some lower level.

If it is a meter for it's input, specs mean something a little different than for a monitor meter for a calibrator's output. The least significant digit in an output monitor meter (the model that comes to mind, although I haven't verified whether this is the one you are discussing, is the CL505A). When you use a CL505A in an output mode, the digits are present to provide information as to what you have the the output for. The digit (in that case) will stay where you set it. The output will match that fixed setpoint to within the tolerance.

As long as you are at least 4:1, the UUT should never have a tolerance of less than +/-0.72 (or whatever 4:1 to the RSS of multiple standards is).

Charles,

I was thinking that it might be a good idea to do measurement uncertainties on the calibrations we do for our standards, but not for other instruments like calipers, micrometers, etc. that are used by mfg. Thoughts?

It is very common (including the company that I work for) to NOT calculate uncertainty estimates for instruments such as micrometers and calipers. Most apply a TUR statement, which is (at this time) acceptable by the accreditation bodies that I am familiar with. I would assume that you calibrate calipers/micrometers with a Grade 2 gage block set, which easily exceeds the accepted 4:1 ratio.

My questions is: should I be including the resolution of the Omega Thermocouple Calibrator in this?

The resolution of the UUT must be included in the uncertainty estimate. With a resolution of 0.1°C, if you were to state that your measurement uncertainty was 0.010°C, I would certainly question you. Basically, I could vary my source by as much as the resolution without seeing a switch in reading.

Example: Input a value of 74.951°C - unit reads 75°C. Input a value of 75.049 - unit reads 75.0°C. The difference in input is 0.098°C with no change in reading.

My estimated uncertainty could not possibly be any tighter that 0.098ºC, and I didn't take into account standard uncertainties, Type A uncertainties, etc.

Here is the misconception: Expanded Measurement uncertainty and tolerance are related. They are not. The Expanded Measurement Uncertainty is the uncertainty of the measurement that you are taking - regardless of tolerance. In other words, the UUT is reading 75.0°C ± 0.116°C. If it reads 74.5°C, the measurement is 74.5°C ± 0.116°C (your uncertainty), but it is out of tolerance.

Where tolerance and uncertainty meet is in TUR calculation. Tolerance is 0.18°C, the combined uncertainty of my standards is 0.009°C. That is approximately 20:1 in my book, and I am very satisfied that I am making good measurements that will reflect UUT condition with very little influence from my standards.

Ryan

Thank you both for your inputs.

I do have one question remaining: do you guys document your TUR on the calibration certificate, or do you simply file somewhere the calculation? I just had an audit last week, and the auditor asked me if I maintain a 10:1 ratio when performing calibrations. I stated to him that 10:1 is ideal, but 4:1 is the minimum. There were no further questions on this, but I'm wondering if this should be documented somewhere. Thoughts?

Originally posted by Charles Wathen
Thank you both for your inputs.

I do have one question remaining: do you guys document your TUR on the calibration certificate, or do you simply file somewhere the calculation? I just had an audit last week, and the auditor asked me if I maintain a 10:1 ratio when performing calibrations. I stated to him that 10:1 is ideal, but 4:1 is the minimum. There were no further questions on this, but I'm wondering if this should be documented somewhere. Thoughts?

Like you, I am working with a corporate internal calibration lab. (No customers that are not part of the company.) I handle the whole issue of measurement uncertainty and TUR at the calibration procedure (CP) level. When gathering data to write a CP, I evaluate the equipment and methods to determine the measurement uncertainty, which is then used to find the TUR as Ryan described.

If it comes out to 4:1 or better that is documented with a statement in the CP. The uncertainty calculations are saved with my general notes for the CP.
If it is less than 4:1, then I re-evaluate the methods and equipment.
If we cannot make it better we talk to the user to see if it is acceptable for their use of the instrument. If they can use it with a lower TUR or with guardbanding then I document that in the CP.
Otherwise, we look for an outside calibration vendor for that model.


We do not print certificates for the customers, although they are automagically generated and stored in the database. (The customers do not want or need a certificate. All they want is a new cal sticker, preferably yesterday!) There is a statement in the template that the TUR meets or exceeds 4:1 unless noted otherwise.

All of this made our ISO 9001:2000 auditor happy, but I am not sure if it would be fully acceptable for ISO 17025.
.
.

Originally posted by Charles Wathen
Thank you both for your inputs.

I do have one question remaining: do you guys document your TUR on the calibration certificate, or do you simply file somewhere the calculation? I just had an audit last week, and the auditor asked me if I maintain a 10:1 ratio when performing calibrations. I stated to him that 10:1 is ideal, but 4:1 is the minimum. There were no further questions on this, but I'm wondering if this should be documented somewhere. Thoughts?

Graeme's idea of documenting the TUR in the CP is brilliant, I think I'll use that. :) Most companies that I have seen report something to the effect of "The TUR of this calibration meets or exceeds 4:1 unless otherwise noted" directly on the calibration certificate.

Where I vary from Graeme is when the TUR is less than 4:1. Quite often, a 4:1 TUR is not possible when calibrating high-end equipment, even following the manufacturer's procedure (Try calibrating a bank of DC Reference cells - it's a minimum of 4 identical items compared and charted. If you send it to NIST, they compare your reference cell to theirs - it is the top echelon equipment.). If the TUR is something in the region of 2:1, you can effectively calibrate using half of the tolerance (guardbanding), and in effect retain a 4:1 TUR. We do this a lot, although it requires more calculations.

At times, this is not even possible (such as the DC Reference cells). In this case the TUR is reported on the calibration certificate, preferrably at each data point. At this point it is generally better to report the estimated measurement uncertainty directly on the certificate, so the user can take the uncertainty into account in their measurements (if they understand uncertainty - which they probably do not).

Ryan

Thank you gentlemen for all your input. I think I have a better understanding of what I need to do in the future for our lab. Now if I only had the time to do it. :vfunny:

A 4:1 TUR is something all cal labs should strive for in all of the tests. There will be times when obtaining a 4:1 tur CANNOT BE ACHIEVED especially when doing standards and when it cannot be achieved the calibration procedure should state it.

Geting off the subject::

HELLO GRAEME, where you at now, remember me..e-mail me

A minimum 4:1 TUR is something that should be written into the labs quality manual.

This is a really good string. Let me throw my two pennies (OK, I'm cheap) into the mix.....

Graeme, I do believe that the electronic certs are acceptable. For internal only labs, I certainly consider electronic only certs acceptable, so long as all the required info under ISO/IEC 17025 section 5.10 is there.

Pangloss, I am a little hesitant to comment much on 9K2K, but under the requirements of ISO/IEC 17025, which may be where your auditor got the uncertainty concept from, testing labs are NOT required to report uncertainty....only to be able to determine the uncertainty.

As for TUR, several posts here are correct that (my paraphrase) if the TUR is less than 4:1, uncertainty must be reported by a cal lab.....at least by an accredited lab performing work under its scope. A TUR of 4:1 or higher, in the U.S., does not have to report specific uncertainties unless requested by clients. However, their AB (accrediting body) will require evidence of the TUR. I suspect that all the ABs in the U.S. will require such evidence from their cal labs. This position is allowed under ANSI/NCSL Z540-1-1994 section 10.2.b which is a carry-over from the old MIL-STD-45662A.

Hope this helps.

Hershal