Bias & Linearity Study and the Uncertainty of Standards

[gaspiston]

I have a question about bias and linearity studies which I hope you guys can help me with.

I recently performed a Bias and Linearity studies on a tool. We are measuring a film thickness on a substrate. The NIST traceable standard used for the Bias study is 45A +/- 4A (A=Angstroms).

When I performed the study the tool is measuring on average 42.2A. Which gives me a bias of -2.8A. According to the statistics (t-test) -2.8A is significantly different than zero at 95% confidence level. My question is as follows:

1) If the NIST traceble substrate that I am measuring to perform the Bias study is 45A +/- 4A (41A to 49A). Is it fair to say that I do not have a bias problem? Because bias is much small than the uncertainty of the traceable standard.
2) There a way to mathematically insert the uncertainty of the standard into the calculations?

One of the methods that we use to see if there is a significant difference between the measured values and the known value is to compare the zero to the upper and lower confidence interval of the bias. If zero is between the two values and you can conclude that the bias is not significant.

Results:
refer value = 45A +/-4
measurments =30
df=29
tcrit=2.04523
average=42.20
stdev=.23
sterr=.04
Avg Bias=-2.79633
t=66.62243
lower 95% Conf Int of Bias = -2.88
upper 95% Conf Int of Bias = -2.71

Would it be statistical valid to add the uncertainty of the standard to upper and lower conf intervals?
lower 95% Conf Int of Bias = -2.88+4=1.12
upper 95% Conf Int of Bias = -2.71-4=-6.71

Your Expert Advice is greatly appriciated.

Thanks,
GP

 

[Stijloor]

A Quick Bump!

Can someone help?

Thank you very much!!

 

[gaspiston]

Any thoughts?

 

[Miner]

Have you ever sent the standard out to a calibration service that has given a more precise statement of the actual value?

 

[gaspiston]

No, not possible. What I am trying to figure out is:

How to account for the uncertainty of the calibration standard in your bias and linerity calculations?

All calibration standards are stated as the value +/- the uncertainty. Being that the case how do you perform a bias and linerity study? If your tools measures the calibration standard within the uncertainty of the standard technically speaking there will be bias. It is our policy no to adjust a tool if the measurements are within the uncertainty of the standard.

Example:
Standard 1:
45A +/- 4A (41A-49A). My tool measure the standard at 42.2A. Bias = -2.78 which is significantly different than zero.
Standard 2:
1023A +/- 3A. (1020A-1026A). My tool measure the standard 1022.53A. Bias = -.475 which is significantly different than zero.

So is there a mathematical way to include the uncertainty into the bias calculations? I want to aviod an argument with an auditor that does not understand metrology and looks at the results of the required bias analysis and mistakenly thinks that there is a problem.

See the attached screen shots of our Bias Analysis. One is of the Bias study using the value stated 45A the other is using the average of the measured values. Same data just different reference points.

Another approach that I am thinking about is to have the speadsheet use the average measured value if it is within the uncertainty of the standard.

-GP

 

[debyang]

A valid reference value for bias analysis should have better resolution or more precise than the analyzed measurement system has, thus any value within the confident interval can be used as reference.

Your study case for reference value 45 has poorer precision(1.96xσ=4) than the study result (EV=0.23) and that makes the bias study invalid.

Trying to obtain more accountable reference value is essential to do this bias study, however, the EV=0.23 value must be examined if too big or not before proceeding the bias t test.

 

[JuneFoo]

I would like to know - do I need to conduct bias & Linearity for a measurement tape, which used to measure length of a part, the tolerance is ? 4?

 

[dgriffith]

At the risk of resurrecting the dead, I'd like to chime in.

....2) There a way to mathematically insert the uncertainty of the standard into the calculations?

Yes, but not with the app you used. You could do it in addition to, using the stats from the app.

Would it be statistical valid to add the uncertainty of the standard to upper and lower conf intervals?
lower 95% Conf Int of Bias = -2.88+4=1.12
upper 95% Conf Int of Bias = -2.71-4=-6.71

No, they cannot be combined in that way in this case. They are uncorrelated, so the standard artifact uncertainty (a probability distribution) and your 30 sample measurement statistics must be combined statistically.

Your Expert Advice is greatly appriciated.

Thanks,
GP

Well, not sure about that. I definitely need to know more about MSA.
However, not knowing anything at all about your instrument's specifications (an optical comparator or such, considering the dimensions involved), a very superficial uncertainty analysis gives me the following; I assumed k=2 @95% for the artifact:
Reference Value: 45 A ?4 A
Computed Mean Value(n=30): 42.204 A
Computed Mean Dev: -2.796 A ?3.946 A with 95% Conf.

The sample mean was within the expanded uncertainty of the standard to begin with, so the evidence could suggest that your instrument is ok on that alone. The uncertainty analysis suggests your instrument's true value could be as low as 38.25 and as high as 46.15, not including other much-needed instrument parameters, with 95% conf.

 

[Alexandre Bonatto]

I believe that (in the future revisions of the manual) they should change the bias (and linearity) study in order to include the standard value uncertainty: in many situations even the best standard value one can generate still has an uncertainty that should not be neglected. Then, the study would be very similar to a standard hypothesis test to compare two distributions (one for the standard value, another for the measurements with the measurement system under evaluation).