Confidence & Reliability - Reference to a 90/95 Confidence & Reliability Level

[jameswalsh]

I have a related quesiton.
Is it correct to make a statement that a process (for example a welding process) is qualified to 95% reliability with 95% confidence?
I would normally make statements regarding an individual inspection or test in a validation. For example I might state that the pull strength of a product is qualified to 95% reliability with 95% confidence. This would be established using the tolerance interval (k factors) analysis (for variable data). Or if 59 samples are inspected (attribute i.e. pass/fail data) and no failures are found then a statement of 95% reliability with 95% confidence may be made about the individual inspection that was carried out.
However I have never made such a statement about the overall process being validated. Does anyone have any thoughts or experience of this? Your opinions would be greatly valued.

 

[Miner]

First, I need to make a distinction between reliability and quality. Reliability is ALWAYS used with a time or time equivalent (e.g., cycles, miles, etc.) element, while Quality is not. Therefore, anytime you make a statement about a given reliability it must be given with an accompanying time element as well as the confidence level. For example, the reliability of a car at 12,000 miles is quite different from its reliability at 200,000 miles.

Also, it is not clear whether you mean product quality, product reliability or process reliability.

If you mean quality, use rolled throughput yield. If you mean reliability, the reliability of the overall process could be viewed the same as a system reliability. In a series system, the overall reliability is the product of the individual reliabilities. for parallel systems, the math is slightly more complex.

 

[Bev D]

I have heard this terminology used for process validations, when using a sampling plan that comes from the reliability world as described by Miner. The samplign plan is actually referring to the 'allowable defect rate' at some confidence level. (there is no time element involved). some individuals use the term 'reliability' to indicate the 'allowable defect rate' or in tolerance interval lingo the 'coverage'.

technically it is incorrect to use the term reliability in this context but your - or your Customer's - definition of the term is what matters as long as everyone is clear about the meaning. of course that's the difficulty when we arbitrarily assign different meanings to common words - we are NEVER clear about their meaning.

our words have many meanings...

 

[jameswalsh]

Thanks for the replies guys. I think Bev D, you are correct the term reliability is actually the allowable defect rate, so if we say 95% reliability with 95% confidence, we really mean we have 95% confidence that the defect rate will be no more than 5%.

My question I suppose really relates to making an overall statement such as this about a process. We currently make these statements in our validation reports about individual tests and inspections carried out on samples that have been manufactured be the process being validated. We don?t typically say that the process as a whole is validated to a certain reliability and confidence level.

To give an example if we are validating a pouch sealing process. We may carry out certain tests or inspections on samples that have been sealed. We may perform pouch seal pull tests and maybe visual inspection of the pouch seal. If we get the results we are looking for we can then make statements such as ?the pouch seal pull test achieved 99% reliability with 95% confidence, while the pouch seal visual inspection achieved 95% reliability with 95% confidence.? Would it be correct to make a statement that the process (pouch sealing) as a whole is validated to 99% reliability with 95% confidence? Or would you have to go with the lower reliability of 95% (with 95% confidence)?

 

[bcoolnow]

All of this discussion so far looks like it is concerning Attribute sample sizes. I understand that the sample sizes for Variable data would be smaller. If this is correct, how is this figured? We have a lot of machines and it is not feasible for us, being a smaller shop, to run these kind of sample sizes for all of this equipment.
Please reply

 

[Bev D]

All of this discussion so far looks like it is concerning Attribute sample sizes. I understand that the sample sizes for Variable data would be smaller. If this is correct, how is this figured? We have a lot of machines and it is not feasible for us, being a smaller shop, to run these kind of sample sizes for all of this equipment.
Please reply

Bcoolnow: can you elaborate on your concern? This thread is from a couple of years ago so it will be helpful to understand your predicament a bit better. For example, are you concerned with lot acceptance sampling or are you concerned with validating a new machine or perhaps a new product?

 

[bcoolnow]

Bcoolnow: can you elaborate on your concern? This thread is from a couple of years ago so it will be helpful to understand your predicament a bit better. For example, are you concerned with lot acceptance sampling or are you concerned with validating a new machine or perhaps a new product?

I am talking about validating existing machines, which would be the same as new. I just want to run enough samples for accuracy but not more than required for good results. Also, how do you determine the minimum Cpk required based on confidence and reliability? i have seen a table with this but can't find it.

 

[joezilla86]

Unfortunately for variables data, the calculation is not as straightforward because it is influenced heavily by the standard deviation of the data.

The method I prefer in this case is using a tolerance interval. It essentially is completely backwards from the attributes method. With attributes, you can calculate the sample size required to validate a particular confidence/reliability. With variables, you select the number of samples you want to run, and then you can calculate the cutoff value at which you achieve the predetermined confidence/reliability and compare that cutoff to your specification limit.

For instance, let's say I am validating a glue joint and I want it to be over 1 lb with 90%/90% confidence and reliability. If I want to do an attributes test, I need 22 samples with all achieving over 1 lb. With variables, I can choose to do say 10 samples.

Let's say my data had a mean of 2.1 lb and standard deviation of 0.5 lb. Using the tolerance interval formula (which is pretty complex, but Minitab spits it right out), we get a lower bound of 1.067. Since this is higher than our spec, the process is validated using only 10 samples. Yay!

There is, however, a corollary to this method. Your sample data must be normally distributed. Otherwise you can use nonparametric analysis, but this basically matches the attribute sample sizes. In our example, the achieved confidence at 10 samples for non-parametric data is only 65% and it only reaches 90% at 22 samples, the same as the attributes test.

The other alternative is using k tables for tolerance intervals, which give you a factor K to apply to the formula X_bar - K * s to determine the cutoff value. It's just a simplified version of the above, but I prefer the exact method from Minitab because I have it.

As for your Cpk question, I'm not aware of a relationship between conf/rel and Cpk so someone else will have to chime in there.

 

[bcoolnow]

That information does help quite a bit and i may have found the table i was referring to, although it was put together by one of our customers so not exactly sure if it is official.
I have just run into another situation where I hope someone has been involved in. It has to do with validating a bead blast process. Is this possible to do, considering the fact that the outcome would be subjective with no definitive way to verify the results.

 

[howste]

I have just run into another situation where I hope someone has been involved in. It has to do with validating a bead blast process. Is this possible to do, considering the fact that the outcome would be subjective with no definitive way to verify the results.

Bead blasting processes are generally validated using Almen strips. This method gives you variable data and shouldn't be subjective.