Sample Size - I want to compare 2 different processes

B

bhsiao

#1
Suppose I want to compare 2 different processes applied to a product, such as a wheel that is quenched in water after casting and a wheel that is NOT quenched after casting (we are looking at flange thickness variation). Seems like a 2 sample t-test is what I need, please correct me if I am wrong.

So the next question is, what should my sample size be?

Can I use the 2-sample t function located under the "power and sample size" in Minitab? Problem with this is that it requires me to insert the std deviation of the population (which is unknown). Unless I can insert the std deviation of a previous sample here... confused...

Is there another way to calculate what the correct sample size should be if I do not know the std deviation of the population?
 
Elsmar Forum Sponsor

Miner

Forum Moderator
Staff member
Admin
#2
If you are truly looking for flange thickness variation, you should be performing a Test for Equal Variances, not the 2-sample t-Test. The t-Test is used to detect differences in the mean, not the variance.

Since you are only testing two levels, Minitab will perform the F test for a normal distribution and Levene's test for a non-normal distribution. First, decide how big of a ratio difference in variation do you want to detect, as well as the degree of confidence. Then go into an F table or use Minitab's equivalent and find the sample sizes that match this ratio.

I will warn you that the number may be quite large (~50 - 100 for each) if you want to detect small ratios (e.g., 1.4:1) at a high degree of confidence.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#3
The other consideration in this case would be the difference in within piece variation and piece to piece variation for both wheel types.

And until we actually see the variation, we cannot know which statistical test you really need. The best statistical analysis we have is to plot the data (the raw data, not the averages and standard deviations).

Statistical analysis is rarely successful as a single event: it is an iterative process.

My suggestion would be to measure 3 pieces for each of the two wheel types, randomly crated if possible (it is also helpful to understand the process) and post your data...
 
D

Duke Okes

#4
Assuming that one of the processes you want to compare is the currently used process, you should have some data already available from which you can estimate the population std. deviation.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#5
If you don't have historical data you can do one of the following
  1. divide the applicable tolerance spread by 5
  2. take a small sample and don't wory about the size. 10-20 will typically suffice. you will most likely underestimate the SD
  3. guess This isn't as radical as it seems. What is the SD of similar processes and tolerance ranges? What seems reasonable?

Remember, knowledge is iterative and there is no such thing as the perfect single experiment.
 
B

bhsiao

#6
If you are truly looking for flange thickness variation, you should be performing a Test for Equal Variances, not the 2-sample t-Test. The t-Test is used to detect differences in the mean, not the variance.

Since you are only testing two levels, Minitab will perform the F test for a normal distribution and Levene's test for a non-normal distribution. First, decide how big of a ratio difference in variation do you want to detect, as well as the degree of confidence. Then go into an F table or use Minitab's equivalent and find the sample sizes that match this ratio.

I will warn you that the number may be quite large (~50 - 100 for each) if you want to detect small ratios (e.g., 1.4:1) at a high degree of confidence.
In my mind, I am thinking that the mean of the two different processes are different, which is why I am thinking the 2 sample t-test would be sufficient. Granted each process will vary, they should have a normal distribution... BUT the mean of each of the processes will be different, so one should be shifted either to the left or the right of the other mean - which is why I want to prove that the means are different (don't really care so much about the actual variation within the same process as it should be normal).

In this case, does the 2 sample t-test make sense?


Thanks!!!
 
D

Duke Okes

#7
2 sample t-test is fine. It is different from paired t-test, which would not be applicable.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#8
Just be careful that you aren't blindly applying the t test. and ignoring other changes that might occur that weren't considered.
Dont' ignore the assumptions that make the t test valid.
Remember that a low p value only tells you that there is a statistically significant different, it doesn't tell you how big the difference is or if it matters. Plot your raw data points not just the average and SD...
 
B

bhsiao

#9
Just be careful that you aren't blindly applying the t test. and ignoring other changes that might occur that weren't considered.
Dont' ignore the assumptions that make the t test valid.
Remember that a low p value only tells you that there is a statistically significant different, it doesn't tell you how big the difference is or if it matters. Plot your raw data points not just the average and SD...

Thanks guys...

However, what should my sample size be?
 

Miner

Forum Moderator
Staff member
Admin
#10
Can I use the 2-sample t function located under the "power and sample size" in Minitab? Problem with this is that it requires me to insert the std deviation of the population (which is unknown). Unless I can insert the std deviation of a previous sample here... confused...
Yes, you can use the standard deviation of a previous sample. You should also decide how big of a difference is of practical importance to you. Then use the Power and sample size function.
 
Thread starter Similar threads Forum Replies Date
A Clinical assessment sample size - Medical device Class IIb implantable (93/42 directive) EU Medical Device Regulations 2
P Sample Size for Distribution Simulation Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
C Stress / Challenge Conditions for Design Verification Testing to Reduce Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 11
K How can Acceptance Number and Reject Number be larger than Sample Size on Z1.4 Table? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
J Validation Sample Size for Tray Seal Qualification and Validation (including 21 CFR Part 11) 2
D Design Verification Sample Size vs Repeats Statistical Analysis Tools, Techniques and SPC 9
M Minimum sample size - Guidance and statistical rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
P Unrealistic Packaging Validation Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 39
J Control chart for huge sample size Statistical Analysis Tools, Techniques and SPC 9
S Is there a standard for sample size during R&D phase Other ISO and International Standards and European Regulations 18
J Sample size definition in an Automotive SMT pilot lot run Misc. Quality Assurance and Business Systems Related Topics 1
E Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
F Bayes Success run Theorem for sample size during OQ&PQ Qualification and Validation (including 21 CFR Part 11) 4
I Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
J Sample size for creating a data base as a reference to a tested variable Other Medical Device and Orthopedic Related Topics 6
T Sample Size for Design Validation Design and Development of Products and Processes 4
C AS9138 Sample Size Determination AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
V Acceptable maximum RSD (relative standard deviation) for an sample size Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
E Sample size for design verification of variable in single use device Design and Development of Products and Processes 20
R Sample size for clinical validation/investigation EU Medical Device Regulations 4
R Process Validation sample size selection Statistical Analysis Tools, Techniques and SPC 0
J Interesting Discussion Sample Size Determination for Medical Device Other Medical Device and Orthopedic Related Topics 19
P Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
F Sample Size for Biocompatibility Tests Other Medical Device Related Standards 4
V Firmware Verification Testing & Sample Size Software Quality Assurance 1
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
H Sample Size Calculation (Confidence Interval and Reliability) - Medical Devices Reliability Analysis - Predictions, Testing and Standards 6
P Cleaning and Disinfection Validation Sample Size of 3? Other Medical Device Related Standards 2
V Sample Size Calculation for Image Analysis - Microscopy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
G Sample Size, Significant Figures, Scale General Measurement Device and Calibration Topics 3
L Heated Sealed Packages - Sample Size for OQ (Operational Qualification) and PQ Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
P Frequency and Sample Size Requirements for an MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
V When and How to apply the Bayes Success Run approach for Sample Size Determination Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
N Sample Size Justification for Medical Device Shelf-Life Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
C Process Qualification Sample size Document Control Systems, Procedures, Forms and Templates 6
G Sample size for IEC 60601-1-11 environmental tests IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
G IEC 60601-1-2 Sample Size issue Other Medical Device Related Standards 1
A How to calculate Mean and standard deviation without sample size in Minitab Using Minitab Software 5
D What is the minimum Sample Size for Weibull Analysis Reliability Analysis - Predictions, Testing and Standards 12
C Sample Size for Stability Testing Other Medical Device and Orthopedic Related Topics 2
M Calculating Cpk when sample size equals to 1 Capability, Accuracy and Stability - Processes, Machines, etc. 12
F Power and sample size for factorial design Using Minitab Software 4
S Sample Subgroup Size for NP/P chart - nPbar>=5 Statistical Analysis Tools, Techniques and SPC 4
U Sample size to prove parts are good with 99.73% confidence Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
F Sample Size to get Statistically Valid Data Measurement Uncertainty (MU) 6
S Low Sample Size for Gage R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
S SPC Card for One Sample Size - Customer requires for all critical dimensions Statistical Analysis Tools, Techniques and SPC 8

Similar threads

Top Bottom