In-tolerance Probability - Level of Confidence - Uncertainty and Proficiency

[Marc]

Date: Thu, 14 Oct 1999 11:36:54 -0400
From: "James D. Jenkins"
To: Greg Gogates
Subject: RE: Uncertainty and Proficiency RE8

Dick,

Pertaining to the level of confidence being the most important thing, you are absolutely correct and while the GUM annex G doesn't use the term "In-tolerance Probability", it uses "Level of Confidence". And when talking about the "level of confidence" of a device in which the specifications are utilized as error containment of the measuring parameter bias, we are talking about the confidence in the error of the readings being contained within those specifications, that is "In-Tolerance Probability". As you pointed out, the confidence level is really the most important thing because of the large effect it has on the uncertainty value. That is why the concerns expressed in the GUM annex G must be dealt with relative to the category B estimates. And of course to deal with them we should use the methods of statistics given for such a task to maintain conformance to the GUM and common sense. You should also read my response in "RE: Uncertainty and Proficiency RE4".

I have found two techniques for determining the "In-tolerance Probability" for a particular devices parameter, with method number one being the most accurate. The mathematical methodology for doing this can be found at http://www.isgmax.com under the section for the Freeware software program "Category B Degrees of Freedom Estimator". You can also download this program and methodology for you own evaluation and use. Dr. Howard Castrup developed this method for estimating the "delta u" variable in Equation G.3 of the GUM so that the degrees of freedom can be computed for Category B uncertainties. I am sure he realized as you do that this "Level of Confidence" is extremely important and deserves some attention.

The "Category B Degrees of Freedom Estimator" gives us 3 options for formulating our findings in a statistical statement with the resultant being that "x lies within +/- L with C% confidence".

Options
1. Containment Limits and Probability.
2. Containment Limits and History of Observations.
3. Containment Limits and Number of Cases.

The two data collection techniques for determining the "In-tolerance Probability" that I have used:

1. Using historical information from previous calibration performance tests we can formulate the "In-Tolerance Probability" pertaining to the parameter in question with actual values. Now you may find that you lack enough historical information to be of statistical value either for the particular unit even when combined with others of the same make/model being serviced under the same conditions. In that case you could contact either the service which provides the calibration for you or the manufacturer and request information pertaining to the in-tolerance probability of the devices parameter in question for the given assigned calibration interval. I have found that technicians with years of experience with a particular make/model already know this probability answer. But hard data is always preferred and with a little digging in the archives they should be able to give you enough data to support your analysis. Enter this information into the "Category B Degrees of Freedom Estimator" or for those math lovers use the formula and a sharp pencil.

2. Not preferred, but in a crunch, an estimate from someone with experience with the device in question is better than not addressing the issue at all. Even if all we have is the data from the current calibration and hopefully the uncertainty of that data, we can still estimate a reasonable close value of "In-Tolerance Probability"(Level of Confidence). If you use a software database for your analyses like the one included with UncertaintyAnalyzer, you can tag and revisit your analysis for improvement once you acquire a reasonable amount of data to support a statistical estimate. Use this estimate as the "Confidence Level" in your equation.

There are numerous advantages to using software in computing and storing all the details pertaining to the analysis, although it is extremely important for those using software to still understand the concepts as well as have an expectation of the resultants value. I find that using software is much more efficient with a higher probability of getting the correct mathematical answer. As a Metrologist, I want to invest my time identifying and dealing with the physical components of the measurement and let my computer perform the redundant math.

I hope this answers your questions. I find that demonstrating this process with the software (as I do in the measurement uncertainty classes I teach) is somewhat easier than trying to explain this in writing where I am unable to sense the comprehension or deal with live questions. Further information pertaining to the software I use and the classes I teach can be found at http://measurementuncertainty.com .

Sincerely,
James D. Jenkins
Quametec Corporation