New ILAC Requirements Regarding Repeatability - Your thoughts on this please

Jerry Eldred

Forum Moderator
Super Moderator
#1
Okay, I have precious little time to spend on this topic, and I must confess that I am not an MU expert. I am reading the new ILAC document (P14:12/2010) regarding new clarified requirements for uncertainty.

On the one hand, it requires including repeatability and suggests reproducibility. But on the other hand, it says to exclude significant contributions from the unit under test (UUT). In things such as digital multi-meters, when calculating uncertainties using best available UUT, and lets say for example the accredited standard is a Fluke 5720A, and a best available UUT might be something like an Agilent 34401A (not quite 4:1 on everything).

And let me throw in the caviat that we have Agilent 3458A's and Fluke 8508A's. But since the 5720A is not good enough to calibrate a 3458A or 8508A, it doesn't seem reasonable to use them for doing repeatability experiements.

All that said, if I then use a 34401A for the repeatability tests, I believe the 34401A may contribute the majority of the repeatability (??). So if I have what ever amount of non-repeatability... lets say for example I do a set of connect/disconnect/STBY/OPER at 10 VDC with the 5720A connected to the 34401A using good 5440 cables. I do a set of 20 readings, and for the sake of discussion (I'm making up numbers for this discussion) I have a 2 Sigma value of 0.00003 VDC. So if this number is likely instability/non-repeatability of the 34401A multi-meter, it should not be included, and should be so stated in the uncertainty budget.

In many DC/AC/RESISTANCE parameters in the context of DMM's and similar, this does not apply because the non-repeatability is the UUT. And in the case of calibrating thermocouple thermometers (such as Omega or Fluke handhelds, etc.) using standards such as Fluke 5520A thermocouple output function, it would be a similar situation (as it is not temperature being calibrated, but electrical simulation of millivolts).

I'd like some opinions on these circumstances in electrical instruments where the likely major contributor is the UUT, and in the new ILAC P14 requirements, do you enter N/A and a note that the UUT is the major contributor.

I'd be interested in some knowledgable opinions on this.
 
Elsmar Forum Sponsor

Hershal

Metrologist-Auditor
Trusted Information Resource
#4
Hi Jerry, P14 is a bit all over the map. That is why ILAC backed off full implementation at least until a meeting in April to hopefully fix the issues. Instead ILAC states use 17025, primarily as P14 seemed to remove the ability to refer to an accepted metrological specification (e.g., 4:1 TUR).

However, the meeting in April does also need to settle on the definition of uncertainty, as there currently seem to be two. One requires the UUT to be included, one allows the UUT to be excluded. The US ABs have pretty much taken the position that the UUT needs to be included in total, and that while the 4:1 TUR can be stated, if an "accredited" cert (with AB logo) is issued it must state the uncertainty.

If this confuses you, don't stress, you are not close to being alone!
 

adamt

Involved In Discussions
#5
I'm also not an expert at MU. I do have some comments though. We are accredited to 17205 and also find the process of interpreting the multitude of opinions and rules head spinning.

I find it dis-concerning that P14 has been implemented and there are so many questions. Definition of uncertainty seems to be something there should not be confusion on. I have always had an opinion that the UUT should not be the included in the uncertainty listed on the calibration certificate.

Reasons I believe this is on electrical instruments is that the cal lab calibrates the UUT under ideal conditions. Things like skilled operator, environment, test leads and drift of the UUT that the end user can only know are not something I can account for. Torque wrench--- the person that is using the torque wrench is the largest source of error in tightening a bolt up in the field. (they do not use a loader). The caliper---the end user can push the jaws together to make the reading vary all over.

One of many examples--
I received two different uncertainties on the same 6 in caliper resolution of .0005in both from accredited labs both under the same accreditation body. Lab 1 listed uncertainty of 0.00056 in and Lab 2 listed 0.0016 in. As customer I would be confused. Both labs listed uncertainties smaller than there scopes but lab two listed uncertainty larger than the accuracy of the caliper. Lab one gave more digits than the UUT has in the display. These are other points of disagreement that seems to have lots of opinions. What was included what was not. Why such a large difference?

So the uncertainty (in some cases) listed on the certifications is really misleading.

So my vote and I'm sure I do not have one, is to have the uncertainty listed on the certification paper work given to a customer include the accuracy of the standard(s) combined with the uncertainty given on there certification RSS with the statement that the end user will have to add any errors that may raise the uncertainty of the finial measurement that they will be making.

That would be the true uncertainty of the measurement.

Would this not make things a lot simpler?
 

Hershal

Metrologist-Auditor
Trusted Information Resource
#7
Not sure of the current status of hopeful fixes to P14. I will post if I hear anything about the P14 discussions.

With respect to the accreditation scope, the CMCs should include the UUT used for the demonstration. However, the UUT is supposed to be - effectively - a golden standard. Then you give sacrifices to the gods, light candles...you get the idea.

Now, the customer will never get the CMC as their uncertainty as their item is almost certainly worse than your golden standard.

Uncertainty reported to the customer must include their item. Hence the uncertainty may be significantly higher for the customer.

In truth, customers should not really look at uncertainties when deciding between two labs, except maybe as a tie-breaker, in my opinion. Having said that, they do, because they may believe the lab with lower uncertainty is "better". Uncertainties can be driven by any number of things, and the customer may be very much unaware of that.
 

adamt

Involved In Discussions
#8
Hi Hershal,

The God you speak about is it Kirot Prime? Do the candles have to be scented? :D Thanks! Hope to here any changes that affect Accreditation.
 
S

stefanhg

#9
Hello adamt,

Regarding the measurement uncertainty of calibration of vernier caliper:
- The uncertainty of the calibration cannot be less than resolution of the caliper.
- The uncertainty of the calibration is reported normally with two significant digits; this is not the measurement uncertainty when you use the caliper in the workshop.
- The uncertainty of the first lab is quite small, but possible - see the first example below.
- The uncertainty of the second lab is too large - they cannot make any conformity assessment.
- If the MPE (maximum permissible error) of the vernier caliper is 0.0015, for the first lab TUR=1:2.7, for the second lab TUR=1:0.9

Method 1: use of indicator for the applied measurement force, gauge block grade 1 with MPE = 0.000008in. acc. to NIST
Uncertainty contributions:
- Gauge block - rectangular distribution with half-width of limits =0,000008in; standard uncertainty = 0.0000046in
- Resolution of the caliper - rectangular distribution with half-width of limits =0,00025in; standard uncertainty = 0.00014in
- Calibration method (Mechanical effects, temperature differences) - normal distribution with pooled standard uncertainty = 0.00025in
Combined standard uncertainty: u=0.00029
Expanded uncertainty with normal distribution: U=2*u = 0.00057in
Expanded uncertainty with trapezoidal distribution (Formfactor 0.3): U=1.9*u = 0.00054in

Method 2: without indicator of the measurement force, gauge block grade 3 with MPE = 0.000048in.
Uncertainty contributions:
- Gauge block - rectangular distribution with half-width of limits =0,000048in; standard uncertainty = 0.0000277in
- Resolution of the caliper - rectangular distribution with half-width of limits =0,00025in; standard uncertainty = 0.00014in
- Calibration method (Mechanical effects, temperature differences) - normal distribution with pooled standard uncertainty = 0.0005in
Combined standard uncertainty: u=0.00052
Expanded uncertainty with normal distribution: U=2*u = 0.0010in
 
D

dv8shane

#10
B
On the one hand, it requires including repeatability and suggests reproducibility. But on the other hand, it says to exclude significant contributions from the unit under test (UUT). In things such as digital multi-meters, when calculating uncertainties using best available UUT, and lets say for example the accredited standard is a Fluke 5720A, and a best available UUT might be something like an Agilent 34401A (not quite 4:1 on everything).

And let me throw in the caviat that we have Agilent 3458A's and Fluke 8508A's. But since the 5720A is not good enough to calibrate a 3458A or 8508A, it doesn't seem reasonable to use them for doing repeatability experiements.

All that said, if I then use a 34401A for the repeatability tests, I believe the 34401A may contribute the majority of the repeatability (??). So if I have what ever amount of non-repeatability... lets say for example I do a set of connect/disconnect/STBY/OPER at 10 VDC with the 5720A connected to the 34401A using good 5440 cables. I do a set of 20 readings, and for the sake of discussion (I'm making up numbers for this discussion) I have a 2 Sigma value of 0.00003 VDC. So if this number is likely instability/non-repeatability of the 34401A multi-meter, it should not be included, and should be so stated in the uncertainty budget.
Jerry

In my opinion not only do you include the repeatability but the UUT resolution as well. The 3458A or 8508A should be used even though you can not issue a conformance statement under the 4:1 premise. The reporting will just be the measurement and the uncertainty of the entire measuring system instead. The results in your budget should be better than using a 34401A

Shane
 
Last edited by a moderator:
Thread starter Similar threads Forum Replies Date
H Regarding the validity of EMC & Safety test reports without NABL & ILAC-MRA mark. CE Marking (Conformité Européene) / CB Scheme 2
Sidney Vianna Informational IAF and ILAC Seek Contractor for Establishment of a Single International Organization for Accreditation ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 2
dinakim_zv CB report or ILAC-MRA report ISO 17025 related Discussions 11
Sidney Vianna Informational IAF and ILAC to merge. A Global Accreditation mammoth? ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 1
M ILAC Regulations - Expression of Units on the Calibration Certificate General Measurement Device and Calibration Topics 6
Hershal ILAC is preparing to start the possible revision to ISO/IEC 17025 ISO 17025 related Discussions 2
4 Understanding ILAC policy P14:12/2010 6.3 part a) General Measurement Device and Calibration Topics 28
S Peer Review using ILAC-G11 as Criteria ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 4
K Any Information on April ILAC 2012 meeting? Measurement Uncertainty (MU) 2
DietCokeofEvil ILAC-g8:03/2009 - Reporting Results with Measurement Uncertainty General Measurement Device and Calibration Topics 7
M ILAC p14:12/2010 - Accredited Calibration Certificate Uncertainty General Measurement Device and Calibration Topics 2
Hershal ILAC P14 and its implications General Measurement Device and Calibration Topics 0
Sidney Vianna IAF and ILAC World Accreditation Day ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 0
C BMU=CMC for every Calibration? ILAC Policy Clarification Measurement Uncertainty (MU) 3
Sidney Vianna ILAC & IAF 2009 meeting - 11 days long! ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 3
A ILAC explanation - IEC60601-1 testing certified by an A2LA laboratory IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
Hershal Congratulations L-A-B and PJLA - Signatory status - ILAC Resolution GA12.03 General Measurement Device and Calibration Topics 0
Marc Definition ILAC - the International Laboratory Accreditation Cooperation Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 0
Marc International Laboratory Accreditation Cooperation (ILAC) General Measurement Device and Calibration Topics 0
D Accreditation Scheme:relationships between ANSI, NIST, NCSL, ILAC, APLAC, NCCLS, etc. General Measurement Device and Calibration Topics 3
M MDR Legal requirements EU Medical Device Regulations 6
M Labeling Placement Requirements according to the 60601-1 IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
Judy Abbott Biocompatibility requirements for submission to Chile Other Medical Device Related Standards 0
S Additional registration requirements for different EU-states on top of standard CE-registration? EU Medical Device Regulations 3
Q What are Quality Requirements for Design? Quality Management System (QMS) Manuals 8
J Old procedure review and training requirements Document Control Systems, Procedures, Forms and Templates 2
Q AS9100D Approved Supplier List requirements and how to apply Supplier Quality Assurance and other Supplier Issues 6
D Halal - Returnable IBC Cleaning Requirements Food Safety - ISO 22000, HACCP (21 CFR 120) 0
P Requirements for Exporting Medical Devices to Libya Other Medical Device Regulations World-Wide 2
S Functionality of software in countries with different legal requirements IEC 62304 - Medical Device Software Life Cycle Processes 2
G MDR Translation Requirements in Member States EU Medical Device Regulations 8
C CALIBRATION CERTIFICATE REQUIREMENTS General Measurement Device and Calibration Topics 8
Sidney Vianna IAQG News AS9163 - Certificate of Conformance Requirements AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 0
N FDA class 2 Device QS Requirements US Food and Drug Administration (FDA) 2
D Class I Shelf-life requirements EU Medical Device Regulations 5
Judy Abbott 10993-18 requirements - Well known materials Medical Device and FDA Regulations and Standards News 8
T FDA labeling requirements US Food and Drug Administration (FDA) 2
B SPOC 001/SPOC 002 AS9100 FAIR REQUIREMENTS AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
V Software license key regulatory requirements Medical Information Technology, Medical Software and Health Informatics 2
L Documenting internal audit of customer specific requirements IATF 16949 - Automotive Quality Systems Standard 7
V Sterility samples requirements for AAT Manufacturing and Related Processes 8
D Shopfloor drawings & manufacturing requirements Document Control Systems, Procedures, Forms and Templates 2
Sidney Vianna ISO 37301 - Compliance management systems – Requirements with guidance for use Other ISO and International Standards and European Regulations 2
W Insulation requirements of HV pulse circuits IEC 60601 - Medical Electrical Equipment Safety Standards Series 8
C I need some clearing on requirements of Asuring the validity of results ISO 17025 related Discussions 2
S Drawing Requirements for Catalog Items (Class II Medical Device) US Food and Drug Administration (FDA) 3
Sidney Vianna IAF PR 7:2022 - Requirements for Producing IAF Mandatory Documents on Transitions ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 1
L TGA Requirements Other Medical Device Regulations World-Wide 1
F GM BIQS level 4 requirements Customer and Company Specific Requirements 1
Aliken Registration veterinary medical device (laser) requirements Medical Device and FDA Regulations and Standards News 2

Similar threads

Top Bottom