# Uncertainty and Proficiency Testing

#### Marc

##### Hunkered Down for the Duration with a Mask on...
Staff member
Date: Fri, 8 Oct 1999 14:44:45 -0700
From: Jan Johansen <[email protected]>
To: Greg Gogates <[email protected]>
Subject: Uncertainty and Proficiency Testing

Dear ISO-25er’s,

We recently received the results of a proficiency test back. The test was for a 5 * digit multimeter. This test was accomplished NAPT(National Association for Proficiency Testing). They handled everything in a professional manner We passed with flying colors. However after reviewing the data from all other participants, I have some observations that I think bears some discussion.

In order to make this as easy as possible I will only deal with one data point, the 10VDC on the 20VDC range. I am going to list the values from the labs that were within +/- .0001 VDC of agreeing with my reading and their stated uncertainties.

Lab Value Uncert
1 9.9985 9e-005
2 9.9985 .00098
3 9.9984 .0018
4 9.9984 .00043
5 9.9983 .0001
6 9.9984 .00013
7 9.9983 .0009
8 9.9985 .0006
9 9.9983 .0001
10 9.9984 .0003

Reported Nominal Value for the test was 9.99839 +/- 8.8 e-005

As you can see the results are quite varied. Here are the observations.

1. The measuring instrument is a 5 * DMM. The best resolution on the 20 VDC range is .0001. How can the nominal value be reported to .00001? It is my contention that a value should never be reported past the maximum resolution of the readout.

2. The definition of uncertainty say’s ‘ … the result of a measurement , that characterizes that characterizes the dispersion of the values that could reasonable be attributed to the measurand.’

3. The device providing the measurand is the 5 * DMM. The one year specification for the DMM is +/- .0015% Rdg + 3 counts or +/- 0.0018 VDC.

4. It doesn’t matter who’s calibrator you use (Fluke, Wavetek), the majority of the uncertainty is in the DMM since it is the unit providing the measurand. Since a specification is a valid Type B uncertainty, it appears that most of the labs did not include it and only reported the uncertainty of their standards, NOT the measurement uncertainty.

5. We could have connected the DMM to our 10 V in-house reference, but it would not have improved our uncertainty by much, because of what is providing the measurement.

I would like to know if I am all wet or am I right. I am uncertain that even this forum will have consensus but it seems to be a good place to air this kind of information.

Thank you

Jan Johansen
JJ Electronics
[email protected]

#### Marc

##### Hunkered Down for the Duration with a Mask on...
Staff member
Date: Tue, 12 Oct 1999 08:11:11 -0700
From: "Nielsen, Larry E" <[email protected]>
To: 'Greg Gogates' <[email protected]>
Subject: RE: Uncertainty and Proficiency Testing

Jan,

Prior to working for Southern California Edison I worked with the Navy METCAL program as a contractor in evaluating calibration lab performance via a proficiency testing program. In order to evaluate the competence of a laboratory using proficiency testing, the measurement "problem" or experimental design must be properly posed, and the artifacts must be up to the task. This is the function of the organizer or provider of the service, in this case NAPT.

If you set out to evaluate several laboratory's whose customer base or workload consists of calibrating 5* digit DMMs, it should be presumed that the laboratory is in possession of "standards" or a calibrator with an uncertainty at least 25% smaller than the intended work. Therefore, in order to pass judgement on the measuring process, the uncertainty of the artifact must be equal to or less than that of the calibrator. In this case a stable 6* digit DMM (preferably a pair of them) would have been a better choice for the artifact.

Based on your brief description of the testing, I don't presume to know all the facts, and we must acknowledge that maintenance of ever increasing test advantage is not always possible. However, it appears to me that you wasted your time and money on this one. Some good guidance on this subject may be found in NCSL RP-15.

Larry E. Nielsen
So. Cal. Edison - Metrology
7300 Fenwick Lane
Westminster, CA 92683
(714) 895-0489; fax (714) 895-0686
e-mail: [email protected]

#### Marc

##### Hunkered Down for the Duration with a Mask on...
Staff member
Date: Wed, 13 Oct 1999 10:36:45 -0500
From: TSmith <[email protected]>
To: Greg Gogates <[email protected]>
Subject: Re: Uncertainty and Proficiency Testing

Greetings Jan,

As a participant of 9 proficiency tests in 1999, I was taken back by the fact that you chose to ignore the basic rules of Proficiency Testing : To publish proficiency testing data without the written consent of NAPT and use the information for advertising (stating your colors were flying.) I can not attempt to understand your agenda nor could I expect any other metrologist that understands proficiency testing or guide 43. I thought one of the conditions of participation was not to use the results for advertisement.

After reading one of my reports issued by NAPT, I noticed that on the report is states that the information contained in this report can not be reproduced expect in full, I am sure that a lot of folks are going to be confused by the limited amount of information that you shared with them. When you only give bits and pieces of the puzzle how can you expect anyone to put it together. (I have written permission to publish this paragraph)

It also states on the report that the values contained in the report were reported to NAPT, I was led to believe that NAPT was an administrator of proficiency testing. Not a calibration provider. All values are reported to the association and the data is then reduced by the association. Isn't this what we want from an third party administer of proficiency tests?

The values reported by all the participants is what should be reviewed by us. After all this was one of the reasons for participating in the test was to help all labs better understand where they are in relationship to other labs making the same type of measurement. I agree that some of the uncertainties reported by other participants seem a little out of whack, now that they have seen how they compare to other they can re-evaluate their measurement process. We all know that labs making measurements have a long way to go to properly understand and report their uncertainties. Let's work on solving that problems together.

I spoke with the folks at NAPT and they shared with me how they arrived at the nominal value for this PT. The nominal value was given to them by the primary lab, which inputted the value into the meter to achieve the desired set point. In this case the standard providing the source was greater than the resolution of the meter, hence the input value required to achieve the reading on the meter under test.

Jan, are you going to also tell us those tests that you dont fly your colors so brightly. I can only say that I feel somewhat confused as to your reason for ignoring the basic rules for this test and to air your results. You may want to reconsider your future actions if you want to participate in any PT's that follow guide 43 and are administered by a third party in this country. Talk to NAPT. Obtain information. Grind your ax somewhere else.

Tom Smith
Maintaining self control.

#### Marc

##### Hunkered Down for the Duration with a Mask on...
Staff member
Date: Wed, 13 Oct 1999 21:01:00 EDT
From: [email protected]
To: [email protected]
Subject: Re: Uncertainty and Proficiency Testing

Jan,

You are right in at least one point. This Group doesn't agree on the answer to your question.

I don't know much about multimeters, but if it were a digital micrometer that read to the nearest 0.0001 unit, I think it would not be unreasonable to report an uncertainty less than 0.0001.

All digital micrometers that I have seen recently round the last place rather than truncate. Thus, a reading of 0.9985 means that the measurement is greater than 0.99845 and less than 0.99855.

Of course the uncertainty of a measurement might be many times this number. Perhaps the repeatability is five time the resolution. Perhaps we are measuring a rubber ball.

But if the resolution were the only, or dominate, factor in the error budget, we would compute the uncertainty by dividing the resolution by two (for an instrument that rounds the last digit), assume a rectangular distribution, divide by the square root of three, multiply by two (k=2), and arrive at an expanded uncertainty of +/-0.00006 unit.

It appears to me that rounding the +/-0.00006 to +/-0.0001 would be throwing away useful information.

Ralph Veale
ISO Guide 25 Assessor

CMM Measurement Uncertainty - Proficiency Testing for 3D on CMM Measurement Uncertainty (MU) 3
In-tolerance Probability - Level of Confidence - Uncertainty and Proficiency Measurement Uncertainty (MU) 0
Measurement Uncertainty in Optical Microscopy Measurement Uncertainty (MU) 1
Uncertainty calculation for an average Measurement Uncertainty (MU) 1
Determining Uncertainty from Gage R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
IEC 60601-2-10 - Accuracy of Pulse Parameters - Required Measurement Uncertainty IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
Missing value of Uncertainty B Measurement Uncertainty (MU) 10
Issues in Uncertainty Estimation Measurement Uncertainty (MU) 5
Operator Manual and Type B Uncertainty Measurement Uncertainty (MU) 3
Learning Uncertainty Budget for Novice - Fluke 5520A Measurement Uncertainty (MU) 1
Does pitch/increment/resolution of a ruled scale apply to measurement uncertainty as line item? Measurement Uncertainty (MU) 10
Uncertainty in measurement larger than tolerance Measurement Uncertainty (MU) 2
Uncertainty Budgets - Phase Noise (SSB), Harmonics Measurement Uncertainty (MU) 2
ISO 17025:2017 clause 7.6.2 - Performing calibration of its own equipment shall evaluate the measurement uncertainty ISO 17025 related Discussions 6
Informational US FDA Final Guidance – Consideration of Uncertainty in Making Benefit-Risk Determinations in Medical Device Premarket Approvals, De Novo Classificati Medical Device and FDA Regulations and Standards News 0
Uncertainty Budget - Externally produced sensor module Measurement Uncertainty (MU) 1
Discussion between co-worker on tolerance and uncertainty and how to apply it. Thoughts? 17025 ISO 17025 related Discussions 1
Assigning a calibration tolerance - An x-y coordinate machine - Uncertainty as my verification tolerance General Measurement Device and Calibration Topics 4
3 Operator Std. Dev. and uncertainty calculation Measurement Uncertainty (MU) 2
Reporting measurement uncertainty for custom items Measurement Uncertainty (MU) 2
L Gage R&R for test precision or uncertainty prediction? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
Uncertainty of staging a short line scale standard on longer measuring machine? Measurement Uncertainty (MU) 4
K Measurement Uncertainty Budget (MU) of for Ultrasonic Thickness gauge Measurement Uncertainty (MU) 1
NIST SOP 29 - Assignment of Uncertainty - Question General Measurement Device and Calibration Topics 0
Medical Device News FDA news - 05-09-18 - Draft - Uncertainty in Benefit-Risk Determinations Other US Medical Device Regulations 0
Can Measurement Machine Bias be part of Uncertainty? General Measurement Device and Calibration Topics 1
How to include machine error in uncertainty calculations? Measurement Uncertainty (MU) 5
Difference between uncertainty and expanded uncertainty of measurement General Measurement Device and Calibration Topics 1
M Calibration certificates avowing accuracy beyond the uncertainty limits? Measurement Uncertainty (MU) 7
ANAB ISO 17025 Accreditation - Uncertainty and systematic error? ISO 17025 related Discussions 6
How many Uncertainty Measurements for Micrometers and Calipers? Measurement Uncertainty (MU) 3
What degrees of freedom means in uncertainty budget and how do I set the value Measurement Uncertainty (MU) 2
Uncertainty Experts - ISO 17025 - If the Distribution is Rectangular Measurement Uncertainty (MU) 2
Uncertainty Budget Examples for Caliper, Micrometer and Dial Gauge Measurement Uncertainty (MU) 3
P Uncertainty and Significantly Different Means - Buffer Solutions Measurement Uncertainty (MU) 4
Pressure Gauge Calibration Uncertainty Calibration and Metrology Software and Hardware 5
E Uncertainty of Mean - Test rig that measures water temperatures Measurement Uncertainty (MU) 3
Internal Laboratory Scope and Measurement Uncertainty (MU) IATF 16949 - Automotive Quality Systems Standard 1
Manual Procedure with Uncertainty Budgeting for either a Caliper or Micrometer General Measurement Device and Calibration Topics 6
MU (Measurement Uncertainty) in an Internal Calibration Laboratory Measurement Uncertainty (MU) 1
N How to Convert Micro Volts into Temperature Uncertainty Measurement Uncertainty (MU) 1
N Include homogeneity uncertainty into budget table for thermocouples Measurement Uncertainty (MU) 1
J CMM Uncertainty Calculation Question - Deviations Measurement Uncertainty (MU) 9
M Uncertainty Budget for a Micrometer Measurement Uncertainty (MU) 5
C Uncertainty Trainer in Texas? Measurement Uncertainty (MU) 3
P Purpose of calculating Uncertainty value in calibration study Measurement Uncertainty (MU) 5
Uncertainty Estimation for Spark Optical Emission Device Measurement Uncertainty (MU) 1
W Dimensional Measurement of 10% accuracy and 25% uncertainty standard General Measurement Device and Calibration Topics 6
Measurement Uncertainty using Minitab Measurement Uncertainty (MU) 2
J Uncertainty budget for IR thermometry at high T - 1000 - 1500?C Measurement Uncertainty (MU) 1