Population failure rate based on attribute data

[Michel Saad]

Hi.

We make several hundred of thousands of parts a week (between 500K to 900K). We have a go/no go test that is performed on 1 part every 800 parts. We never see a failure. Some are interpreting this as the entire population is error free. How do I get a population failure rate based on an sampling that never sees a failure?

Michel.

 

[Bev D]

inspect every part. seriously. that is the only way.

there are ways to estimate the worst case error rate that would result in a sample having zero defects and you could treat all inspections for the last year as a single sample for purposes of the estimate. You would most need to use the exact binomial and calculate the upper confidence interval.
for example lets say that you made 900k parts last week. That is 1125 units tested with 0 defects found. that gives us an upper confidence interval (95%) of .00327 or .33%. The larger the sample size the smaller that upper bound is going to get so if you use 2 weeks at 900k the upper limit is: 00.163....since you've NEVER found a defect this number will get vanishingly small.

are there defects out there? maybe, maybe not. but it is highly unlikely that you would have tested so many times and not found any defects if there were some number out there...which leaves you with 2 approaches that will not result in debating the existence of something you haven't found: test them all for some extended period of time or rely on your customer to tell you when you they find the defects... Have your customers complained?

 

[Michel Saad]

Interesting. Bev D.: Can you explain to me how you calculated the .33%?

Testing 100% is not possible. This is our sample because that is the limitation on how many we can get through our test system.

Customer does not complain because customer does not measure it.

I am simply trying to have an intelligent conversation with our executives so they understand that what they were told is not quite true. Because we see no failure internally does not mean that it is zero in the entire population. I then get the question: Based on our sample, what is the estimate of the defects in the population ? since we have no defect rate internally, I don't know how to answer this question.

 

[Bev D]

see the attached spreadsheet, tab=Exact Binomial" it will calculate the upper confidence interval for you.

but I will reiterate what I said given the long track record of zero defects found, the statistical answer is that the defect rate in the field is vanishingly small.

the fact that you have no complaints from your customer(s) is with further proof of the very very low defect rate OR that the defect doesn't cause any failures and therefore the defect criteria is meaningless (not uncommon) and the whole debate is irrelevant.

 

[hogheavenfarm]

Even though your customer has not reported any defects, this still does not signify a low defect rate. The defect could not affect your customer (assembly, or finishing) but may affect end users, who would have difficulty getting complaints to go anywhere. If your customer is the end user, then you would have a better case. Also the customer may not see the area you focus on as Critical to Quality, even though it may seem that way to you. Obviously we do not know enough here to comment with any precision, just throwing out some possibilities. I frequently see cases where the manufacturer sees a certain set of characteristics as important, and the customer sees an entirely different set.