Is Uncertainty Calculation Mandatory

[BradM]

Further, you might as well jump in the water, so to speak, and consider building an uncertainty budget.

1. Start with vendors and associates in the industry. I have found it very helpful to benchmark on uncertainty budgets.

2. Make a post here of your activities, and what you are doing. There are a lot of pretty knowledgeable folks here that will be more than happy to help you as much as they can.

3. To me, the best way to learn, is just to do it. So, feel free to use the Cove as your resource.

 

[dv8shane]

If you are claiming traceability of your calibrations (to the SI units of measure through a national metrology institute), accounting for measurement uncertainty is necessary.

So tell me how do Fluke, Agilent etc keep on getting away with this?

Their non accredited certificates state they are traceable to NIST, NRC etc. What if we suggest to A2LA they tell their customers they can't issue non accredited certificates because of the definition of traceability, and then lets tell all all those who operate under 9001 etc they must have accredited calibration and be able to perform accredited calibration for their in house requirements.

I don't think this would go over well, but it might be worth trying just to get a reaction!

Shane

 

[George Weiss]

I have another grab at the "NIST Traceable" thing:
.
In another thread the traceability concept was discussed at length with no real end. From the evolution of the calibration documentation and expressed requirements it seems the phrase "NIST Traceable" has also evolved? It seems that Nist-traceable once meant an unbroken chain of standards linked to NIST. The varying postures and views seem to suggest a high degree of qualified uncertainty evaluation in the current "NIST Traceable" definition. It might be that NIST-Traceable(ISO17025) , is not identical to NIST-Traceable(MIL-STD45662), and etc. It might also be that computerized statistical wiz-bang applications have been applied excessively to the old 4:1 TAR. Just because we can, do we need to?

 

[dv8shane]

It might also be that computerized statistical wiz-bang applications have been applied excessively to the old 4:1 TAR. Just because we can, do we need to?

Depends on your QMS and business applications. If a customer requires uncertainty to fulfill their definition of traceability then yes it must be provided. For those that have had a look at military T.K.O.'s or NAVAIR or T.B. procedures there is no reporting of uncertainty. It is TAR driven.
Example company X is the subcontractor that provides calibration for a department of the US DND. They are accredited and the primary standards lab. I would challenge any one to say their non accredited calibrations are non conforming.

 

[raghu_1968]

Hello Raghu!

Now... there are still a lot of questions unanswered here. Including, what level of calibration you are performing, what industry, what standard, etc.

But.... if you are ABC company, and you are calibrating your own micrometers/calipers and such, you don't need to perform uncertainty calculations so that your work is considered a legitimate calibration. However, there are multiple considerations you must take into account, such as acceptable accuracy ratios, environment, etc. And... once you're done with all that, it's just as easy to throw it into a budget and calculate uncertainty anyway.

But.... honestly... I wouldn't feel comfortable one way or another, unless you might be at liberty to share a little more about your industry and the requirements. Whoever made this observation in your lab might have an awfully good reason for suggesting it.

Brad,

At present I am working for the API (Active Pharmaceutical Ingredient) industry.

The available calibration facility at our site is very small and it is not accredited to any of the certifying bodies (like NABL, ISO 17025).

We are maintaining standards and the standards are calibrated outside at the NABL accredited lab. The lab that performed the calibration of our standards provides the uncertainty value.

The measuring device that we calibrate in-house are pressure gauge, vacuum gauge, temperature indicators, weighing scales, weighing stones etc.,

For weighing stones we are using secondary reference standard for calibration.

The observation came from the internal audit during self inspection.

 

[BradM]

Brad,

At present I am working for the API (Active Pharmaceutical Ingredient) industry.

The available calibration facility at our site is very small and it is not accredited to any of the certifying bodies (like NABL, ISO 17025).

We are maintaining standards and the standards are calibrated outside at the NABL accredited lab. The lab that performed the calibration of our standards provides the uncertainty value.

The measuring device that we calibrate in-house are pressure gauge, vacuum gauge, temperature indicators, weighing scales, weighing stones etc.,

For weighing stones we are using secondary reference standard for calibration.

The observation came from the internal audit during self inspection.

Ok, set aside the argument for uncertainty. Have you performed the following:

1. Have you performed an assessment of the measurement system?
2. Have you assured (and documented) sufficient accuracy ratios?
3. Have you accounted for all the variables that could influence/affect your measurement process?
4. Are the uncertainties of your standards sufficient?

I would be intrigued to know what basis the internal auditor mentioned uncertainties.

Raghu, as a metrologist in the community, I agree with Hershel and the others, that performing uncertainty is pretty much a requirement. The process gives you several benefits:

1. It derives an estimation of contributing error for any/all influences in the measurement process.
2. It delivers a quantifiable amount of true potential error; accuracy ratios do not do that.
3. It gives a true indication of how good the measurement really is.

Having said that , for your operation, if there is not a clearly defined requirement to perform uncertainty, it may be something you should plan to accomplish. But to say it's an absolute immediate requirement (unless it is) may cause confusion, frustration, and create a bunch of unnecessary work.

You should be able to demonstrate to the internal auditor (or any auditor) that you have adequate control of the process, and that your measurement process is sufficient for its purpose.

 

[George Weiss]

This is an interesting situation. A small calibration facility, which is likely ISO 9001, but not Z540.3 or 17025, and not believed to be 13485 either, and is receiving 17025 calibrated standards. In the company’s focus, Active Pharmaceutical Ingredient, is a clue. There is either an over-kill in the calibration process, or likely a need to elevate internal processes to par with as yet not fully developed expected or actual requirements. A company in the API drug ingredient chain is or will be going cGMP.
.
FDA has long recognized that the cGMP requirements in the good manufacturing practice regulations for finished pharmaceuticals (21 CFR Parts 210 and 211) are valid and applicable in concept to Active Pharmaceutical Ingredient (API) manufacturing. FDA expects API manufacturers to apply cGMPs to the API process.
The API and FDA reference @
https://www.fda.gov/downloads/ICECI/ComplianceManuals/ComplianceProgramManual/ucm125420.pdf

 

[Graeme]

So tell me how do Fluke, Agilent etc keep on getting away with this?

Their non accredited certificates state they are traceable to NIST, NRC etc. What if we suggest to A2LA they tell their customers they can't issue non accredited certificates because of the definition of traceability, and then lets tell all all those who operate under 9001 etc they must have accredited calibration and be able to perform accredited calibration for their in house requirements.

I don't think this would go over well, but it might be worth trying just to get a reaction!

Shane

Documented measurement uncertainty is necessary to be able to substantiate a claim of traceability of measurements. However, the conditions for reporting the uncertainty vary.

If an accredited laboratory performs a calibration that is within their scope, and the customer requested an accredited calibration, then the laboratory must report the measurment uncertainty in one of the acceptable ways.

If an accredited laboratory performs a calibration that is within their scope, and the customer has not requested an accredited calibration, then the laboratory must calculate and retain the measurment uncertainty, but need not report it.

If the laboratory is not accredited, or if it is but the requested work is outside their scope, then the laboratory should calculate and retain the measurment uncertainty, and may or may not report it.

Calculating and recording measurement uncertainty is always good professional practice even if there is not a "must" or "shall" attached to it. If the work is not accreidted, then customer requirements can override laboratory standard practice for what goes on the calibration certificate.

 

[Graeme]

I have another grab at the "NIST Traceable" thing:
.
In another thread the traceability concept was discussed at length with no real end. From the evolution of the calibration documentation and expressed requirements it seems the phrase "NIST Traceable" has also evolved? It seems that Nist-traceable once meant an unbroken chain of standards linked to NIST. The varying postures and views seem to suggest a high degree of qualified uncertainty evaluation in the current "NIST Traceable" definition. It might be that NIST-Traceable(ISO17025) , is not identical to NIST-Traceable(MIL-STD45662), and etc. It might also be that computerized statistical wiz-bang applications have been applied excessively to the old 4:1 TAR. Just because we can, do we need to?

Measurement traceability is always to the relevant units of the International System of Units (the SI), generally through standards maintained by a national metrology institute (NMI). For example, if you are calibrating a laser interferometer, your measurements (including their uncertainty) need to be traceable to the meter and the second. If you are in the USA, the national standards for those units are maintained by NIST, which is the NMI for the United States. Other countries have their own NMI's.

ISO/IEC 17025 requires measurement traceability to the SI (not to NIST).

MIL-STD-45662A is an obsolete standard. It was replaced in 1994 by ANSI/NCSL Z540-1, which has subsequently been replaced by ISO 9001 and ANSI/NCSL Z540.3. As 45662 was a United States Department of Defense standard for calibration systems for the US armed forces and their contractors, it naturally referenced the United States' NMI. (Other US government agencies still do the same thing, but a few are starting to recognize that other NMIs exist, and even that they can be compared using the BIPM Key Comparison Database.)

TAR (test accuracy ratio) is an oblsolete concept. It is permissible to use test uncertainty ratio (TUR), where the uncertainty of the measurement system is compared to the specification of the item being measured. Of course, to properly compute measurement uncertainty, or to call the result of the process a calibration, you need to have traceable measurements.

 

[Graeme]

... For those that have had a look at military T.K.O.'s or NAVAIR or T.B. procedures there is no reporting of uncertainty. It is TAR driven. ...

This is correct as far as it goes, but is is not the full story. A military calibration procedure (Navy NAVAIR, Air Force TO, Army TB) is written by that service's primary standards laboratory for the needs and operating environment of that service. (Those probably are not the same as your customer's needs, which is why all procedures of external origin must be reviewed and validated before use. A military procedure is NOT necessarily the best available for your customer!) Also, now it is TUR driven, not TAR.

Measurement uncertainty is taken into account when the procedure is developed. It is reflected in the choice of measurement standards and in the test limits for each measurement. They are also aware of the knowledge, skills and abilities of the people in the labs that will be using the procedure, and the operating environment of the labs. As examples for the operating environment, a Navy calibration facility may be on a ship at sea; an Army, Air Force or Marine Corps lab may be deployed to Afghanistan.) Since the military laboratories are not generally accredited (and there is no requirement for accreditation), there is no need for measurement uncertainty to be reported to the end user.

Graeme
Ex-USAF, Ex-USN civilian, but a Marine forever!