Sampling Level for Defective Units with a confidence level of ~95%

M

Maven

#1
I am posting this somewhat off-centered question to see if anyone else has dealt with this. We are analyzing returned units (retail consumer products) for the purpose of determining failure modes and relaying that information back up to the manufacturing process.

My question is: what sample size would be appropriate to have a sufficient confidence level (~95%) that the defects found represent the population? Can I use the same methodology as acceptace sampling of good product?

Thanks in advance for any comments!
 
Elsmar Forum Sponsor

Bev D

Heretical Statistician
Staff member
Super Moderator
#2
Re: Sampling Level for Defective Units with a onfidence level of ~95%

not really...remember that acceptance sampling plans are typically designed to reject a lot when 1 defect (or at most 2-3) are found.

but let me ask why you think you need to be confident that the occurence rate of each failure mode needs to be known with such great confidence?

If the intent of determining what the failure modes are is to feed this information to manufacturing in order to determine casue and prevent reoccurence, why woudln't a simple pareto tally of say 100 or more parts be sufficient to give manufacturing guidance? Since this is one my primary job responsibilities I've found that the only reason for a precise estimate is to debate and delay taking action.

shouldn't the goal be: find failures; correct cause, repeat?
 
Last edited:
M

Maven

#3
Re: Sampling Level for Defective Units with a onfidence level of ~95%

Thanks, Bev. I appreciate your perspective and you raise some really good points. We are relaying these results back to our supplier (manufacturer) and want to be confident we have sampled enough from the return population to say that the defects we found are representative.

Yes, it is for root cause analysis and prevention, but also for us to be able to charge back the supplier for the defects we have found since we were charged from our customer. So, we need some level of confidence of "how much is enough". We don't want to sample more than we need to because resources are valuable. Make sense?
 
D

daamor

#4
Re: Sampling Level for Defective Units with a onfidence level of ~95%

Your defect rate in this instance should be based on total units shipped.

defective/ # units shipped built with the same specifications

post-market data should not use acceptance sampling plans
 

rickpaul01

Involved in HankyPanky
#5
I think there is some confusion regarding your question.
I think you are taking a sampling of the returned items. So, in my opinion, yes, use a sampling plan.
 

Steve Prevette

Deming Disciple
Staff member
Super Moderator
#6
You can use the binomial confidence interval for seeing if two populations have the same defect rate or not.

You can either calculate the interval exactly, or more commonly, use the normal approximation. The normal approximation for the binomial is that the standard deviation is sqrt(avg*(1-avg)/n)

The 95% two sided interval for the defect rate is plus or minus two standard deviations (well, 1.96). So, if I measure a 10% defect rate by sampling 50 units, the defect rate is

10% plus or minus 2 * sqrt(.1 * (1-.1) / 50)

Or, if you want to determine the failure rate plus or minus 3%, and you think the failure rate is about 10%, you initial estimate of the sample size is .03 = 2 * sqrt(.1 * (1-.1) / n)

Now solve for n.
 
M

Murphys Law

#7
Why a 95% confidence level?

The sample size will also be determined by size of the problem you are chasing. The smaller the % or even ppm, will determine the sample size needed to detect it.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#8
Take a small sample. How small, you ask? Well it depends on the expected number of defects for any defect type. The lower the defect rate, the higher the sample size needed. As a general rule of thumb, it's probably between 50 and 100.

Then calculate the defect rate as the number of defects divided by that small sample size. You can do this for just the defect type that has the smallest rate as it will have the largest sample size.

Then you can calculate the sample size needed for the final sample size:

n = [4*defect proportion*(1-defect proportion)]/delta^2
this will give you sample size with 95%.
the delta is the critical decision: delta represents the amount of 'precision' you are looking for in the estimate. (think of it as the width of the confidence interval). it aslo has to be stated as a proportion and not a percentage.

so for example, if the defect proportion for defect type C is 0.10 (10% of the returns) and you wanted to be 'precise' within say + 1%, you would state delta as + .01.
then n = [4*.1(.9)]/.01^2 = 3600

Now this is where we begin to negotiate. 3600 is probably too high for you. you can reduce the sample size by increasing the delta (reducing the precision). so for a 10% defect rate you may change the delta to:
Delta = .02, n= 900
delta = .05, n = 144

good luck!
 
M

Maven

#9
Thanks, Bev. I think you are correct, but also it is what I was afraid of. We probably need significantly larger samples sizes if we want to be able to say with any level of confidence that we have "X" percent of returns with a particular defect (i.e. dead battery, failed motor, etc.). There is testing/analysis involved with looking at each of these returns and so going above 100 pieces, for example, will probably be too time consuming.

Thanks to Bev and all who replied on this. Much appreciated!
 

Steve Prevette

Deming Disciple
Staff member
Super Moderator
#10
Keep in mind, on the positive side, if you accumulate your data over a long period of time (say years) you may build up your confidence levels. Keep a p-chart control chart going though to see if there is a detectable shift over the time period so you know it is valid to cumulate the data.
 
Thread starter Similar threads Forum Replies Date
I Sampling processes - Who must define the AQL level? AQL - Acceptable Quality Level 9
A What is the purpose of using AQL (Acceptable Quality Level) for Sampling? AQL - Acceptable Quality Level 3
N AQL (Acceptable Quality Level) Sampling Plan for Small Lots AQL - Acceptable Quality Level 8
N AQL - Sampling plan (variable, inspection level ii, normal inspection S-method) AQL - Acceptable Quality Level 2
A Level 2 Sampling Plan SOP (Standard Operating Procedure) Request Inspection, Prints (Drawings), Testing, Sampling and Related Topics 9
Y What is the confidence level for a given AQL in zero sampling plan AQL - Acceptable Quality Level 4
S Using AQL to get true random sampling using General Inspection Level II AQL - Acceptable Quality Level 4
J IVD sampling sizes for buyer Reliability Analysis - Predictions, Testing and Standards 0
S Which Sampling Plan(s) Should I Use? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
C Sampling - ISO 16269-6 vs ISO 2859 Other ISO and International Standards and European Regulations 0
L Sampling Plan Risks AQL - Acceptable Quality Level 6
A Sampling plan for in-process QC (medical devices) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
M Minitab Capability of the Population (no sampling) Using Minitab Software 11
W LTPD, AQL, Ppk and Cpk validation sampling plan table Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
Q Sampling method for Distributor verification activities EU Medical Device Regulations 1
E Change in control plan - Do I have to do sampling? IATF 16949 - Automotive Quality Systems Standard 1
S Is using ANSI/ASQ Z1.4-2008 the correct sampling plan to determine Pass/Fail of Apparel measurements? AQL - Acceptable Quality Level 4
N Sampling Plan for Internal Audits - ISO 2859 or 3951 - Or Neither? Internal Auditing 6
F Product audit sampling plans IATF 16949 - Automotive Quality Systems Standard 3
R Reduced sampling plan for sterial products APQP and PPAP 0
A ISO 2859-1 - 9.3.3.2 Switching score - Sampling Plans and PPM AQL - Acceptable Quality Level 0
R BS EN 12060:1998 - Zinc and zinc alloys Method of sampling Specifications Manufacturing and Related Processes 4
S High voltage testing - ISO 17025 - 7.2.2 Validation of methods and 7.3 Sampling ISO 17025 related Discussions 3
8 Activity Sampling Exercise Lean in Manufacturing and Service Industries 11
T Defining sampling plan for different AQL AQL - Acceptable Quality Level 3
B AS9100D Sampling Plan AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
R Design verification for interim design outputs - sampling rationale ISO 13485:2016 - Medical Device Quality Management Systems 2
Nicole Desouza Sampling plan for a customer who wants AQL 1.0 (per ANSI Z1.4) AQL - Acceptable Quality Level 5
V How to interpret AQL sampling tables AQL - Acceptable Quality Level 5
A AQL sampling bags of parts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
C Applying Weibayes to Attribute sampling Reliability Analysis - Predictions, Testing and Standards 0
E Sampling for capability studies for variables and attributes Capability, Accuracy and Stability - Processes, Machines, etc. 4
O Statistical justification of sampling size in V&V tests ISO 13485:2016 - Medical Device Quality Management Systems 5
M Sampling Plan for Alumin High Pressure Die Castings Manufacturing and Related Processes 0
E Sampling plan for orthopedic implant - Process Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
S Inspection/sampling economics Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
S How to make a Sampling Plan - Conveyor belts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
V Statistical basis and justification while comparing / changing sampling plans Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
J AQL for Effectiveness Check - What AQL tiers to use on this type of sampling? Nonconformance and Corrective Action 9
N The question asks about sampling plans and AQL Reliability Analysis - Predictions, Testing and Standards 5
S Help develop sampling plan for prototype samples (medical devices) Statistical Analysis Tools, Techniques and SPC 7
A ISO 2859 Single Sampling - Clarification regarding the sampling table Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
C Are zero defect sampling plans relevant to AS9120B Distributor AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
R DCMA Sampling of CSI Parts AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
D Sampling method that is representative of the devices supplied by that distributor EU Medical Device Regulations 5
T ISO 13485: 2016 Internal Audit - Is sampling on projects allowed? ISO 13485:2016 - Medical Device Quality Management Systems 6
M Zero Defect Sampling Plan Standards Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
L ISO 2859-2 LQL sampling plan linked to %confidence/%reliability Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
K Sampling Plan for transparent tubular plastic parts from 32 different cavities Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
M ISO / ASTM standards for Sampling Plans for Medical Devices Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1

Similar threads

Top Bottom