Measurement Uncertainty effects on Process Capability and Tolerances

[dfirka]

Hello Everybody!

This seems to me a very simple question, so I'm frustrated because I couldn’t find a clear answer so far.

Introduction:

As you may know, every measurement has some uncertainty associated with it, usually described as a standard deviation (u) that produces a "confidence interval" around the value measured.

On the other side, the concept of Process Capability refers to the total variability of a certain process, calculated from a group of measurements, and how this variability relates to the tolerance of the product. Usually this relationship is expressed as the Quality Index called Cp:

Cp=(Tolerance) / (6 sigma )

The denominator is six times the value of estimated total standard deviation, representing the "process width".

Question:

It is argued that as the "sigma" in the denominator contains the uncertainty of the measurements, that are exogenous to the process, "inflating" therefore the process variability with a variance component u^2. Thus, in order to find the true "process variability", without the influence of the uncertainty in the measurements, we should "deflate" sigma in the denominator, using sqr(sigma^2 - u^2)

1) So according to this the <process width> should be considered "shorter" due to the effect of uncertainty in measurements.

(This is the position of AIAG MSA Manual, 3ed , Appendix B)

but...

If I look at the histogram, made up with the individual results of the data set, each one of the extreme values should be considered V +/- 3u, so actually the histogram representing the process will be wider (3u on each side).
Widening the histogram this way, could intuitively represent the confidence interval of what I can consider the process variability.

2) So according to this the <process width> should be considered "wider" due to the effect of uncertainty.

(1) and (2) are contradictory, but if I look at the arguments, I can't find the flaw that makes one of them the valid answer. Can you help me?

Thanks
Daniel

[Statistical Steven]

Originally Posted by dfirka

Hello Everybody!

This seems to me a very simple question, so I'm frustrated because I couldn’t find a clear answer so far.

Introduction:

As you may know, every measurement has some uncertainty associated with it, usually described as a standard deviation (u) that produces a "confidence interval" around the value measured.

On the other side, the concept of Process Capability refers to the total variability of a certain process, calculated from a group of measurements, and how this variability relates to the tolerance of the product. Usually this relationship is expressed as the Quality Index called Cp:

Cp=(Tolerance) / (6 sigma )

The denominator is six times the value of estimated total standard deviation, representing the "process width".

Question:

It is argued that as the "sigma" in the denominator contains the uncertainty of the measurements, that are exogenous to the process, "inflating" therefore the process variability with a variance component u^2. Thus, in order to find the true "process variability", without the influence of the uncertainty in the measurements, we should "deflate" sigma in the denominator, using sqr(sigma^2 - u^2)

1) So according to this the <process width> should be considered "shorter" due to the effect of uncertainty in measurements.

(This is the position of AIAG MSA Manual, 3ed , Appendix B)

but...

If I look at the histogram, made up with the individual results of the data set, each one of the extreme values should be considered V +/- 3u, so actually the histogram representing the process will be wider (3u on each side).
Widening the histogram this way, could intuitively represent the confidence interval of what I can consider the process variability.

2) So according to this the <process width> should be considered "wider" due to the effect of uncertainty.

(1) and (2) are contradictory, but if I look at the arguments, I can't find the flaw that makes one of them the valid answer. Can you help me?

Thanks
Daniel

The contradiction comes from the fact that the histogram values have the uncertainity of the measurement error. In (1) you are removing the variance of the measurement error from your overall variance (this makes sense). But in (2) you cannot remove this source of error from the "actual value observed".

I have stated many times that indices such as Cp are susceptable to inflation or deflation of the process error based on the data used to calculate the error. Sometimes people include measurement error, and sometimes they remove it. There is no consistent application of the method to estimate the process variability.

[dfirka]

Thanks Steven,

So if you have to standardize the calculation, will you use (1)?

Let me give you an exaggerated example for the sake of expressiveness, because I'm still a little bit confused:

I have only three values, 5, 6 and 7. If I consider the uncertainty on those measurements, (u=0.5) I can create an "expanded uncertainty" (GUM) of 1.5 (using coverage k=3), so for me 5 is the interval [3.5, 6.5], and 7 is [5.5,8.5]

If I use (1), I will calculate the overall variance using 5,6 and 7, and then reduce it because those value where inflated with "measurement variance", is that right?

But if you ask me, I really don’t know whether my process is wandering between 3.5 and 8.5, instead of between 5 and 7. That was the argument for using (2) in the capability calculation.

Of course, the argument looses strength as we increase the S/N ratio, but I want to clearly understand the title of this post: "Measurement Uncertainty effects on Capability", and maybe define a "de facto" standard (i.e. agreed) way to calculate it with its pros and cons.

DF

[Statistical Steven]

Originally Posted by dfirka

Thanks Steven,

So if you have to standardize the calculation, will you use (1)?

Let me give you an exaggerated example for the sake of expressiveness, because I'm still a little bit confused:

I have only three values, 5, 6 and 7. If I consider the uncertainty on those measurements, (u=0.5) I can create an "expanded uncertainty" (GUM) of 1.5 (using coverage k=3), so for me 5 is the interval [3.5, 6.5], and 7 is [5.5,8.5]

If I use (1), I will calculate the overall variance using 5,6 and 7, and then reduce it because those value where inflated with "measurement variance", is that right?

But if you ask me, I really don’t know whether my process is wandering between 3.5 and 8.5, instead of between 5 and 7. That was the argument for using (2) in the capability calculation.

Of course, the argument looses strength as we increase the S/N ratio, but I want to clearly understand the title of this post: "Measurement Uncertainty effects on Capability", and maybe define a "de facto" standard (i.e. agreed) way to calculate it with its pros and cons.

DF

I think you are on target (no pun intended). The issue is complex in that your estimate of measurement error is estimated from measuring the same part multiple times to quantify the variance. Any individual value is assumed to be 100% accurate, but we can use the measurement variability to build some confidence interval on the individual value. In your example, the true value, yet unknown value is +/- 1.5 of the observed value. Your process includes this source of variability, so for a capability index, you would include all sources of variability since you cannot control the measurement error.

I think as long as your measurement error is less than 10% of your total process error, it is not an issue. Once the measurement error is too large or S/N is too small, the process capability index is just a pseudo-measurement capability index.

I am not a fan of using capability indices, but rather designing studies to assess sources of variability, and using pareto principles to reduce the larger sources of variability. De facto, it will improve capability.

[Tim Folkerts]

DF,

This is an interesting challenge that I hadn't really thought about before. I think I got a little carried away, but here goes...

Let me follow up on your example:
Data: 5,6,7
mean = 6,
standard deviation = 1

For explicitness, I will use

s(m) = st dev inherent in the measurement system
s(p) = st dev inherent in the process
s(d) = st dev actually observed in some set of data

Then
s(d)^2 = s(m)^2 + s(p)^2

• If you already know the measurement system is very accurate i.e. s(m)=0, then all of the variation is in the actual process and s(p) = 1.
  In this case, the value "5" is really 5 to a high degree of accuracy.
  Also, the process is wandering between 6 +/- 3 (using your value of k=3 and my value of s(p) = 1)
• If you already know the measurement system is poor i.e. s(m) = 1, then none of the variation is attributable to the process, so the best estimate is s(p) = 0.
  In this case, the value "5" is really 6 to high degree of accuracy!
  Also, the process is wandering between 6 +/- 0
  You don't want to say "Well that 5 could really be between 2 and 8 because the measurement system is so poor". Really, you should say "The 5 must really be 6 becaue I know that the only reason it was as bad as 5 to begin with is because the mesaurement system is so bad."
• If we split the uncertainty evenly, then s(m) = s(p) = 0.707
  In this case, the value "5" is probably from a part that is near 5.5 but the measurement error also added some error.
  Also, the process is wandering between 6 +/- 3*0.707 = 6 +/- 2.1

It looks like inflating the individual values is never a good idea. The process variation should simply be based on the estimated parameters for the process as a whole: the process mean and s(p) = [ s(d)^2 - s(m)^2 ]^0.5

I think that all makes sense.


NOTE1: The exact results probably depend on an assumption of normality, but I expect the gist is correct in any case.
NOTE2: The estimates of sigma themselves are subject to considerable uncertainty, especially for a sample size of just 3! That will muddle the precise results, but the general conclusion should stand.

Tim F

[Statistical Steven]

Originally Posted by Tim Folkerts

DF,

This is an interesting challenge that I hadn't really thought about before. I think I got a little carried away, but here goes...

Let me follow up on your example:
Data: 5,6,7
mean = 6,
standard deviation = 1

For explicitness, I will use

s(m) = st dev inherent in the measurement system
s(p) = st dev inherent in the process
s(d) = st dev actually observed in some set of data

Then
s(d)^2 = s(m)^2 + s(p)^2

• If you already know the measurement system is very accurate i.e. s(m)=0, then all of the variation is in the actual process and s(p) = 1.
  In this case, the value "5" is really 5 to a high degree of accuracy.
  Also, the process is wandering between 6 +/- 3 (using your value of k=3 and my value of s(p) = 1)
• If you already know the measurement system is poor i.e. s(m) = 1, then none of the variation is attributable to the process, so the best estimate is s(p) = 0.
  In this case, the value "5" is really 6 to high degree of accuracy!
  Also, the process is wandering between 6 +/- 0
  You don't want to say "Well that 5 could really be between 2 and 8 because the measurement system is so poor". Really, you should say "The 5 must really be 6 becaue I know that the only reason it was as bad as 5 to begin with is because the mesaurement system is so bad."
• If we split the uncertainty evenly, then s(m) = s(p) = 0.707
  In this case, the value "5" is probably from a part that is near 5.5 but the measurement error also added some error.
  Also, the process is wandering between 6 +/- 3*0.707 = 6 +/- 2.1

It looks like inflating the individual values is never a good idea. The process variation should simply be based on the estimated parameters for the process as a whole: the process mean and s(p) = [ s(d)^2 - s(m)^2 ]^0.5

I think that all makes sense.


NOTE1: The exact results probably depend on an assumption of normality, but I expect the gist is correct in any case.
NOTE2: The estimates of sigma themselves are subject to considerable uncertainty, especially for a sample size of just 3! That will muddle the precise results, but the general conclusion should stand.

Tim F

Tim -
Excellent analysis. I still contend that the measurement system is part of the process, and therefore should not be removed from the estimate of sigma used in calculating the Cpk. You can never seperate out the measurement system from process variation, because in many cases s(d)^2 is not equal to s(p)^2 + s(m)^2, but rather you need to subtract off some covariance between the process and measurement system since the measurement system is not perfectly linear and measurement error might be a function of process location. It gets really muddled.

[Hershal]

It is a simple answer.....

Two sources have been mentioned.....uncertainty of measurement under GUM (Guide to the Expression of Uncertainty in Measurement), and analysis of the measurement process under AIAG MSA.

They are not imcompatible, but are different angles of the same picture I believe. The MSA looks at the measurement process to account for variances in the result. Measurement uncertainty (MU) also looks at the measurement process to acount for the variances in the result. However, MU is typically seen in its normal form coming out of the accredited calibration lab where MSA is at the customer location (e.g. manufacturer). Each does the analysis a bit differently but accomplishes the same basic goal, with a few exceptions.....

MU is described in a single sided approach based on the bell curve. That is, confidence levels are k=1. k=2, or k=3 (or 1, 2, or 3 sigma if you prefer), approximately 67%, 95%, or 99%. That of course, is once standard uncertainty has been calculated per GUM.

For MU, once you have standard uncertainty, simply multiply by 2 or 3 to get expanded uncertainty per the GUM. Done.

First, however, one has to get to standard uncertainty. I will proceed from the MU approach since that was the topic of the original question.

MU has two basic components, Type A or random, and Type B or systemic. Type A simply put includes the reading taken during the efforts to determine MU. For example, if calipers are used to measure a part, three operators, three days, three times per day, the result is 27 sets of readings. If there are five different readings involved in the part, then there are five sets of 27 readings. Take the standard deviation for each set of 27 readings. Much of the time for MU these readings may well be taken as the population, but that is not a hard and fast requirement, as it may also be taken as sample.....for this we will consider it the population. Take each standard deviation, divide by square root n and that is the Type A uncertainty for that set of readings. The Type A is now ready for the final formula.

Type B is much trickier. First, list ALL potential influences that could affect the measurement. For example, heat, RH, resolution, parallax, or other sources of error. Take the MU reported by the accredited calibration lab, divide each of those numbers by 2 for normal distribution (or 3 if the calibration is a NMI like NIST), and those numbers are ready for the final formula. The other influences must be quantified. The quantity is then divided by square root of 3 (approximately 1.732) for rectangular distribution.

Take each of the results, square each one, add them all together, take the square root. Now you are at standard uncertainty. This is where you multiply by 2 to get to expanded uncertainty approximating 95% confidence.

If the discussion is MU then keep it as short and simple as possible.....

Hope this helps.

Hershal

[dfirka]

Thanks Tim, Hershal & Steven!

Yesterday I posted the same question in math.stat group, and got an answer pointing also to (1):

Newsgroups: sci.stat.math
From: "Scott Brogan" <s.d.brogan...@cantab.net> - Find messages by this author
Date: 18 Jan 2006 14:32:17 -0800

(1) is correctly reasoned & intuition is at fault in (2). If I
understand you right I think I know why. Measurement error already
forms part of the results. When you take a data point at random, then
nearly all the time its real value will lie +/- 3u on each side of the
measured value : when you take 100 data points at random and pick out
the maximum, then more often its real value will lie below the measured
value-rarely will the maximum have a negative measurement error;
commonly it will be the maximum precisely because it has a large
positive one. It's picking out extreme points & considering them as
typical that misleads intuition. If I haven't explained this well look
up regression towards the mean or do a little simulation.

Address of thread: http://groups.google.com/group/sci.s...8a8dd0e70b0315

DF