Sample Size Attribute DOE (Design of Experiments)

S

Sophomere

#1
Hi people,
I am doing a six sigma project and needed some help on the following problem.

There is an operation that can go wrong due to three reasons A, B and C.
The outcome of operation is binary, and whether the result is pass/fail is decided by visual inspection. What has contributed to the operation failure can also be known (A/B/C ?).

There are 4 machines which carry out this operation. I need to find out if there is/are machine(s) more/less efficient that the others, and if that is the case, which of A/B/C should i focus upon.

How can i decide the sample size for attribute DOE? And how do i f
 
Elsmar Forum Sponsor

Bev D

Heretical Statistician
Staff member
Super Moderator
#3
Re: Sample Size Attribute DOE

*I* use the basic formula for categorical data sample sizes. You need to know the current failure rates. then you need to know how much difference you think you will see betweent the different factors.

the generic formula for sample size determination (given teh defect rate is > ~5%) is:

n = [z_alpha/2*pbar(1-pbar)]/Delta^2

z-alpha/2 is the multiplier based on your alpha risk. for a basic first stage experiment an alpha of 5% is jsut fine. so z_alpha/2 = z_.025 = 1.96

Delta is the amount of difference you expect to see if one of the factors is actually a primary contributor to the results...don't make it too big! thsi will drive a very small sample size which will work if the difference is in fact very large, but if it isn't you will run into resolution issues.

I typically plot the results for each main effect and interaction usign the resulting defect rate and it's 95% confidence intervals. there are other statistical aproaches such as Chi-square and ANOM for proportional data. But for a first step experiment you don't need statistical prescision or sophistication.

depending on the nature of the process, you might need to have 1-2 replicates to the first set of samples. (in other words, if your failure rate is relatively high you might get an n=30 for each trial. IF you have non-homogenous variation over time you woudl be well advised to replicate the n=30 for each trial at least once.

It is always helpful if you have the data to plot the current failure rate for each lot, hour or day as appropriate for each machine over some reasonable period of time to understand the nature of the variation. A basic p chart will do.

If you can provide some more info on the process and any data (p charts) you may have we can provide more help
 

Statistical Steven

Statistician
Staff member
Super Moderator
#4
Re: Sample Size Attribute DOE

Bev, how do you determine the power of that approach? I struggle with that often. Also, you can use logistic regression to get odds ratios for the binary response. Just another analysis idea.
 

MasterBB

Involved In Discussions
#5
Re: Sample Size Attribute DOE

You can simply use a DOE to do this. How many factors do you have?
What is the allocated funds for this DOE? This will be your strategy to determine the right DOE/sample size.
 

Statistical Steven

Statistician
Staff member
Super Moderator
#6
Re: Sample Size Attribute DOE

You can simply use a DOE to do this. How many factors do you have?
What is the allocated funds for this DOE? This will be your strategy to determine the right DOE/sample size.
Budget does not justify sample size! You can design a DOE that meets your allocated funds be significantly under powered to identify differences if they exist, therefore wasting the money because the study will not reach it goal.
 

MasterBB

Involved In Discussions
#7
Re: Sample Size Attribute DOE

Steven,
I agree with your comments, but it all depends oh how much information is to be collected from the DOE & hence decide on the right resolution.
 

Statistical Steven

Statistician
Staff member
Super Moderator
#8
Re: Sample Size Attribute DOE

Steven,
I agree with your comments, but it all depends oh how much information is to be collected from the DOE & hence decide on the right resolution.
Doing a DOE to collect information is one approach, but once you decide to do statistical tests including a F-test, the Type II error is usually quite large for most studies. This is the real issue with DOE, we claim no statistical difference when in reality the study was SO underpowered, the effect would never be seen even if present. I think we are saying the same thing in different ways.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#9
Re: Sample Size Attribute DOE

I think the OP is planning on a "DOE" quote = "How can i decide the sample size for attribute DOE?"

Of course he hasn't said whether it will be a full factorial or fractional factorial, but his question was about sample size nto DOE structure...
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#10
Re: Sample Size Attribute DOE

Bev, how do you determine the power of that approach? I struggle with that often. Also, you can use logistic regression to get odds ratios for the binary response. Just another analysis idea.
You can of course use the categorical equation for sample size that includes the Beta risk BUT in most problem solving investigations, power is usually not a consideration. Unlike in Design optimization, V&V or Process characterization, we are not concerned about small effects.

IF we understand the baseline variation of the process AND the failure rate is not really small AND the levels of the factors are appropriately set, we will be looking to see a relatively large difference the levels of the causal factor (or interaction). so we are looking for large differnces not subtle or small ones. That is why I prefer the relatively simple and visual methods of confidence intervals or Proportional ANOM. You can see the magnitude of the difference and the statistical significance in one glance. It is also much more compelling and easier to understand for managers and the mathematically challenged...

In these cases power can actually be detrimental since it can (statistically) detect small differences that are either 'noise' from a poor experimental design or a real effect of a minor causal mechanism. (we must always answer the question "but does it matter"?) Due to the typical pareto distribution of the effect of each causal mechanism and the "square root of the sum of the squares" effect of each of the causal mechanisms on the final output, fixing a small effect cause will not result in any big - or potentially even detectable - improvement.

Of course some problems require power but that will become evident either in a baseline check or when the well structured DOE fails to return an affirmative result....
 
Last edited:
Thread starter Similar threads Forum Replies Date
B Bayes Success-Run on Attribute Data - Determining Sample Size Statistical Analysis Tools, Techniques and SPC 2
optomist1 Sample Size for Attribute Binary Data Inspection, Prints (Drawings), Testing, Sampling and Related Topics 10
L Correct Sample Size for Attribute Gage R&R (Good / Bad parts) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
V Sample Size for Attribute Data - Large Production Lots Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
W Sampling Plan problem - Determining Sample Size for both Attribute and Variables Data ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
S Sample Size for Attribute MSA (Measurement System Analysis) in 4th Edition Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
C Attribute Data - What is the required sample size (medical field)? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 14
N Finding Sample Size - Data Type is Attribute Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
N Determining Sample Size for Attribute Data Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
A Clinical assessment sample size - Medical device Class IIb implantable (93/42 directive) EU Medical Device Regulations 2
P Sample Size for Distribution Simulation Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
C Stress / Challenge Conditions for Design Verification Testing to Reduce Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 11
K How can Acceptance Number and Reject Number be larger than Sample Size on Z1.4 Table? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
J Validation Sample Size for Tray Seal Qualification and Validation (including 21 CFR Part 11) 1
D Design Verification Sample Size vs Repeats Statistical Analysis Tools, Techniques and SPC 9
M Minimum sample size - Guidance and statistical rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
P Unrealistic Packaging Validation Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 39
J Control chart for huge sample size Statistical Analysis Tools, Techniques and SPC 9
S Is there a standard for sample size during R&D phase Other ISO and International Standards and European Regulations 18
J Sample size definition in an Automotive SMT pilot lot run Misc. Quality Assurance and Business Systems Related Topics 1
E Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
F Bayes Success run Theorem for sample size during OQ&PQ Qualification and Validation (including 21 CFR Part 11) 4
I Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
J Sample size for creating a data base as a reference to a tested variable Other Medical Device and Orthopedic Related Topics 6
T Sample Size for Design Validation Design and Development of Products and Processes 4
C AS9138 Sample Size Determination AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
V Acceptable maximum RSD (relative standard deviation) for an sample size Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
E Sample size for design verification of variable in single use device Design and Development of Products and Processes 20
R Sample size for clinical validation/investigation EU Medical Device Regulations 4
R Process Validation sample size selection Statistical Analysis Tools, Techniques and SPC 0
J Interesting Discussion Sample Size Determination for Medical Device Other Medical Device and Orthopedic Related Topics 19
P Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
F Sample Size for Biocompatibility Tests Other Medical Device Related Standards 4
V Firmware Verification Testing & Sample Size Software Quality Assurance 1
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
H Sample Size Calculation (Confidence Interval and Reliability) - Medical Devices Reliability Analysis - Predictions, Testing and Standards 6
P Cleaning and Disinfection Validation Sample Size of 3? Other Medical Device Related Standards 2
V Sample Size Calculation for Image Analysis - Microscopy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
G Sample Size, Significant Figures, Scale General Measurement Device and Calibration Topics 3
L Heated Sealed Packages - Sample Size for OQ (Operational Qualification) and PQ Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
P Frequency and Sample Size Requirements for an MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
V When and How to apply the Bayes Success Run approach for Sample Size Determination Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
N Sample Size Justification for Medical Device Shelf-Life Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
C Process Qualification Sample size Document Control Systems, Procedures, Forms and Templates 6
G Sample size for IEC 60601-1-11 environmental tests IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
G IEC 60601-1-2 Sample Size issue Other Medical Device Related Standards 1
A How to calculate Mean and standard deviation without sample size in Minitab Using Minitab Software 5

Similar threads

Top Bottom