Population failure rate based on attribute data


Michel Saad


We make several hundred of thousands of parts a week (between 500K to 900K). We have a go/no go test that is performed on 1 part every 800 parts. We never see a failure. Some are interpreting this as the entire population is error free. How do I get a population failure rate based on an sampling that never sees a failure?

Elsmar Forum Sponsor

Bev D

Heretical Statistician
Staff member
Super Moderator
inspect every part. seriously. that is the only way.

there are ways to estimate the worst case error rate that would result in a sample having zero defects and you could treat all inspections for the last year as a single sample for purposes of the estimate. You would most need to use the exact binomial and calculate the upper confidence interval.
for example lets say that you made 900k parts last week. That is 1125 units tested with 0 defects found. that gives us an upper confidence interval (95%) of .00327 or .33%. The larger the sample size the smaller that upper bound is going to get so if you use 2 weeks at 900k the upper limit is: 00.163....since you've NEVER found a defect this number will get vanishingly small.

are there defects out there? maybe, maybe not. but it is highly unlikely that you would have tested so many times and not found any defects if there were some number out there...which leaves you with 2 approaches that will not result in debating the existence of something you haven't found: test them all for some extended period of time or rely on your customer to tell you when you they find the defects... Have your customers complained?

Michel Saad

Interesting. Bev D.: Can you explain to me how you calculated the .33%?

Testing 100% is not possible. This is our sample because that is the limitation on how many we can get through our test system.

Customer does not complain because customer does not measure it.

I am simply trying to have an intelligent conversation with our executives so they understand that what they were told is not quite true. Because we see no failure internally does not mean that it is zero in the entire population. I then get the question: Based on our sample, what is the estimate of the defects in the population ? since we have no defect rate internally, I don't know how to answer this question.

Bev D

Heretical Statistician
Staff member
Super Moderator
see the attached spreadsheet, tab=Exact Binomial" it will calculate the upper confidence interval for you.

but I will reiterate what I said given the long track record of zero defects found, the statistical answer is that the defect rate in the field is vanishingly small.

the fact that you have no complaints from your customer(s) is with further proof of the very very low defect rate OR that the defect doesn't cause any failures and therefore the defect criteria is meaningless (not uncommon) and the whole debate is irrelevant.



Quite Involved in Discussions
Even though your customer has not reported any defects, this still does not signify a low defect rate. The defect could not affect your customer (assembly, or finishing) but may affect end users, who would have difficulty getting complaints to go anywhere. If your customer is the end user, then you would have a better case. Also the customer may not see the area you focus on as Critical to Quality, even though it may seem that way to you. Obviously we do not know enough here to comment with any precision, just throwing out some possibilities. I frequently see cases where the manufacturer sees a certain set of characteristics as important, and the customer sees an entirely different set.
Thread starter Similar threads Forum Replies Date
L Help me to decide which test to use - If each proportion belongs to the same population Using Minitab Software 7
A Generating samples from a population in Minitab Using Minitab Software 0
Ajit Basrur Six countries that make up half the world?s population Coffee Break and Water Cooler Discussions 8
R Sample vs. Population (Data Analysis) Statistical Analysis Tools, Techniques and SPC 11
B Hypothesis Tests with nonparametric dependent population samples Using Minitab Software 5
T Calculation of Confidence Interval for the Population Variance Statistical Analysis Tools, Techniques and SPC 2
R Research involving a very small population (100 approx) - Help with Sampling Statistical Analysis Tools, Techniques and SPC 3
M Sampling and Sample Size to reflect the Population Inspection, Prints (Drawings), Testing, Sampling and Related Topics 14
C World Population Clock as Web Part in SharePoint Document Control Systems, Procedures, Forms and Templates 1
1 How can you tell that Sample Data represents the Population Six Sigma 23
F Capability Calculation on 1 Population Multiple Tolerances Capability, Accuracy and Stability - Processes, Machines, etc. 5
J SPC in 100% Population - Monitoring Overtime Expenses - Financial Application Statistical Analysis Tools, Techniques and SPC 3
T Process Instructions via BOM for Subcontractor? (PCB Population) Manufacturing and Related Processes 2
samer US and World Population - 6,500,000,000 people inhabiting the world - October 2006 Coffee Break and Water Cooler Discussions 5
M Probability Question - Population of x number of 'different' items Statistical Analysis Tools, Techniques and SPC 15
J Sample size for fraction defective of a population Six Sigma 1
A How to figure the total number of defects for a population - Statistical question Statistical Analysis Tools, Techniques and SPC 19
J When to use Ppk/Pp and Cpk/Cp - Difference between population and sample Capability, Accuracy and Stability - Processes, Machines, etc. 22
Tim Folkerts Confidence Interval Question - The calculations ASSUME you know the true population Statistical Analysis Tools, Techniques and SPC 8
L Cp/Cpk vs Pp/Ppk (Short term using population sigma) - Formulas to use? Capability, Accuracy and Stability - Processes, Machines, etc. 21
S How to measure severity if my product is designed for emergency use and failure would result in death? ISO 14971 - Medical Device Risk Management 6
C Corrective action for failure in documents control ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
T Root Cause Failure Analysis - Not following Customer packaging Specification Problem Solving, Root Cause Fault and Failure Analysis 5
A Exact terms for a plating failure and difference between rejection rate and failure rate Manufacturing and Related Processes 9
JoCam Failure to test Class I medical device to IEC 60601-1-11 IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
C Kaizen Events - Factors Affecting Failure Lean in Manufacturing and Service Industries 13
C Failure nets - Same level effects FMEA and Control Plans 0
C Failure mode at 0h, vs. 1 year vs. 5 year... FMEA and Control Plans 8
R How to properly use Survival Analysis to predict time to machine failure Reliability Analysis - Predictions, Testing and Standards 0
D Testing to failure for design verification Reliability Analysis - Predictions, Testing and Standards 11
M Informational US FDA – Ethicon Recalls Circular Staplers for Insufficient Firing and Failure to Completely Form Staples Medical Device and FDA Regulations and Standards News 0
C Detection Action for Failure Effects - AIAG 4th Edition Layout FMEA and Control Plans 5
Marc NASA Says Oregon Company Metals Fraud Caused $700 Million Satellite Failure - 2019 World News 16
C PCBA Hardware Component different failure types - How to rate detection? FMEA and Control Plans 8
N PFMEA repetitive failure modes in subsecuent stations. FMEA and Control Plans 1
P Detection rating for a user (surgeon) related failure mode in DFMEA APQP and PPAP 3
J How to look up Failure Modes on FDA website FMEA and Control Plans 4
P Electronic component failure analysis turnaround time Reliability Analysis - Predictions, Testing and Standards 0
C Design for Assembly in DFMEA - Failure Effect of Sub-System(s) FMEA and Control Plans 5
Marc Server RAM Failure - 7 Dec 2018 Forum News and General Information 3
K Spot weld - Peel test failure Manufacturing and Related Processes 3
C Electrical Power Plant Supply failure in PFMEAs FMEA and Control Plans 10
kisxena Memo to File - Failure to Validate Supply Chain Security Management Systems 1
Y IEC 62304 Section 4.3(a) - 100% probability of failure IEC 62304 - Medical Device Software Life Cycle Processes 3
S FMEA column column ''therblig motion'' for a given potential failure FMEA and Control Plans 12
J What does "factional and failure dependencies“ mean? FMEA and Control Plans 1
N PFMEA Potential Failure Mode for Weight FMEA and Control Plans 5
P Root Cause for a Process Validation Batch Failure 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
M Infusion Pump Power Failure Alarm IEC 60601-2-24 IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
K Determining Effect of Failure without a DFMEA (Design FMEA) FMEA and Control Plans 1

Similar threads

Top Bottom