NotMe
Quite Involved in Discussions
Whether your measurement result is numerical or categorical, in both cases you have to make a critical assumption: You need to assume that the three parts are representative of the population. If you are willing to do so, you could argue that the repeated measurements are independent from each another. Thus, you could use Fleiss kappa to estimate the reliability of the measurement device.
You will have to use your estimated probabilities that each part is measured to be within the specification. For the following simulation I assumed
p.withinSpec = (2%, 99%, 3%) and measured each part nRepeat=50 times. Calculating Fleiss kappa and then repeating the simulation many times yields the power of this procedure. Using the above stated number yields the following result:
The red line represents the average value (kappa=91.4%) and the blue lines represent the 2.5% percentile (kappa=81%) and the 97.5% percentile (kappa=1). Thus, using these numbers you obtain a kappa >= 81% with a probability of 97.5%. In contrast, if we change the success probability to p.withinSpec = (5%, 97%, 5%), we will need to increase nRepeat to 400 to obtain kappa>=80% with a probability of 80%.
From what you stated above I understand that you know the numerical values of the uncertainty of your measurement device. I would take this value into account and proceed with the statistical study only if you are convinced that the measurement system is good enough. E.g. are you able to use the manufacturing specification [LSL+2Sigma, USL-2Sigma] for evaluation -- instead of the product specification [LSL, USL]? Using guard banding is useful to ensure quality.
You will have to use your estimated probabilities that each part is measured to be within the specification. For the following simulation I assumed
p.withinSpec = (2%, 99%, 3%) and measured each part nRepeat=50 times. Calculating Fleiss kappa and then repeating the simulation many times yields the power of this procedure. Using the above stated number yields the following result:
The red line represents the average value (kappa=91.4%) and the blue lines represent the 2.5% percentile (kappa=81%) and the 97.5% percentile (kappa=1). Thus, using these numbers you obtain a kappa >= 81% with a probability of 97.5%. In contrast, if we change the success probability to p.withinSpec = (5%, 97%, 5%), we will need to increase nRepeat to 400 to obtain kappa>=80% with a probability of 80%.
From what you stated above I understand that you know the numerical values of the uncertainty of your measurement device. I would take this value into account and proceed with the statistical study only if you are convinced that the measurement system is good enough. E.g. are you able to use the manufacturing specification [LSL+2Sigma, USL-2Sigma] for evaluation -- instead of the product specification [LSL, USL]? Using guard banding is useful to ensure quality.
