# Sample Size to get Statistically Valid Data

F

#### farasm5

We are looking for stabilization time during the leak test to check the amount of decay. (Pressure is the same for all stabilization times.)

According to some unofficial testing, 4 stabilization times (10second, 15s, 20s & 30s) were selected and 5 times were tested per stabilization time. It was confirmed that the amount of decay is smaller when the stabilization times were increased and the variations of each tests (5 times per stabilization time) were similar.

Stabilization time 15s looks good to us. But only 5 times were tested for this time (15s).
My question is how many times we have to test with 15s to get statistically valid data for the amount of decay?

I look forward to hearing from you.

Thanks.

Peter Kim

#### Steve Prevette

##### Deming Disciple
Staff member
Super Moderator
It depends. In your original scenario, did you run the data through ANOVA and find out the the decay rates were significantly different between the stabilization times? This would give you some confidence you are on the correct track.

When you say you want to get "statistically valid data for the amount of decay" what does that mean to you? That indeed 15 seconds is the best stabilization time? That you want to know what the decay rate is within a certain error band? That you want to declare this process to be "in control" (that is, stable and predictable)? You will get different answers for sample size depending upon what your need is.

#### dgriffith

##### Quite Involved in Discussions
We are looking for stabilization time during the leak test to check the amount of decay. (Pressure is the same for all stabilization times.)

According to some unofficial testing, 4 stabilization times (10second, 15s, 20s & 30s) were selected and 5 times were tested per stabilization time. It was confirmed that the amount of decay is smaller when the stabilization times were increased and the variations of each tests (5 times per stabilization time) were similar.

Stabilization time 15s looks good to us. But only 5 times were tested for this time (15s).
My question is how many times we have to test with 15s to get statistically valid data for the amount of decay?

I look forward to hearing from you.

Thanks.

Peter Kim
*Forward: At what point will you decide there is no more stabilization needed and how many iterations will you need to find the pressure value that best represents the remaining stable pressure in the system? I recommend a visual approach first, rather than jumping in with statistics and then wash rinse repeat. Data visualization will tell you more in one iteration. Done properly, you can record the results and perform analysis on them while plotting. Monitor the pressure with a good transducer and plot the results in real time--about one data point per minute to start--and you will be able to see how much stabilization time you need.*

Perhaps you already know that abiabatic's are the culprit here. The apparent gain or loss of pressure due to temperature change within the system (heat of compression) without external heat added or removed. So, yes, the longer the stabilization time, the less impact it will have on any apparent leak.

The volume of the system may determine how long the stabilization need be. So will the delta pressure and delta time. So will the precision of the measurement--do you care about ten-thousandths or just tenths.

For a small system of less than a ft.^3, our wait time is 5 minutes, then leak check timing of either 1 or 5 minutes. This is based on two things--manufacturers leak check procedure, and visual observation--you can actually see the displayed pressure change its rate during the wait time. I would have to know more about your system to have any opinion other than 15 seconds is too short.

But we have never tried to statistically characterize ours, perhaps because we have the luxury of waiting until we're satisfied, with no time constraint. I would be interested in your findings.
I would add a reproducibility component to your testing. Perhaps repeating a few tests after dis-assembly and re-assembly of your plumbing connections.

Last edited:
F

#### farasm5

Hi, Steve & Dgriffith

I appreciate for your response. Let me explain in details.
We did not run through ANOVA. The product we are testing has volume, 8? (L) x 6?(W) X ?3?(H). It is not big.

According to the collected data ( 5 times per stabilization time), when the stabilization time was 15s, the amount of decay was max. 0.023psi, min. 0.018 psi, average 0.02psi and standard deviation 0.0019.

For example, I will set up a 0.035 psi as a trigger point. After the stabilization time (15 s), if the leak tester detects the amount of decay is greater than 0.035psi within certain time, the tested product will be rejected.

My concern is that alarm is falsely triggered not because of a leak but because the leak tester is not stabilized within 15 seconds yet.
I want to be confident that the amount of decay doesn?t exceed 0.035 psi after 15 stabilization time unless there is a leak. In order for me to be confident for this, how many times I have to test the amount of decay with 15s stabilization time?

Thanks.
Peter

#### Steve Prevette

##### Deming Disciple
Staff member
Super Moderator
For example, I will set up a 0.035 psi as a trigger point. After the stabilization time (15 s), if the leak tester detects the amount of decay is greater than 0.035psi within certain time, the tested product will be rejected.

My concern is that alarm is falsely triggered not because of a leak but because the leak tester is not stabilized within 15 seconds yet.
I want to be confident that the amount of decay doesn’t exceed 0.035 psi after 15 stabilization time unless there is a leak. In order for me to be confident for this, how many times I have to test the amount of decay with 15s stabilization time?

Thanks.
Peter
We are still faced with what is your definition of "stabilized". If you can express that as less than a certain rate of pressure change per time interval, and if you indeed know your standard deviation then I'd probably define it as measuring a rate of change less than that value, taking into account the standard deviation of the measurement error. This will relate to sample size in that the standard deviation of the average leakage rate can be better estimated the more samples you have - the standard deviation of the average is the standard deviation of the individuals divided by the square root of the sample size.

#### dgriffith

##### Quite Involved in Discussions
Steve, my post explained a little of it as far as 'stabilized' goes. Pressure change in a closed system, where one would perform a leak test (like the air conditioning system on your car-it's not allowed to leak into the environment), the measureable pressure is influenced by adiabatic effects. As the temperature of the system returns to its quiescent point, the pressure also changes, and that can be seen as a system leak, which it is not. Stabilization is the point at which adiabatic effects are no longer present, and the pressure only changes (ideally) if the system leaks. He's trying to find that point and prove it statistically, I believe.

#### Steve Prevette

##### Deming Disciple
Staff member
Super Moderator
However, without a numerical explanation of what "stabilized" means, there is no way to numerically calculate or justify a specific sample size ahead of time.

I think what you are going to be faced with is collecting data in small increments, analyze the cumulative data to that point and subjectively decide "is that good enough". If the answer is no, collect another set of data. Basically you will have to keep going until you can definitively say - the test passes or the test fails.

Unrealistic Packaging Validation Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 13
Control chart for huge sample size Statistical Analysis Tools, Techniques and SPC 9
Is there a standard for sample size during R&D phase Other ISO and International Standards and European Regulations 18
Sample size definition in an Automotive SMT pilot lot run Misc. Quality Assurance and Business Systems Related Topics 1
Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 3
Bayes Success run Theorem for sample size during OQ&PQ Qualification and Validation (including 21 CFR Part 11) 4
Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
Sample size for creating a data base as a reference to a tested variable Other Medical Device and Orthopedic Related Topics 6
Sample Size for Design Validation Design and Development of Products and Processes 4
AS9138 Sample Size Determination AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
Acceptable maximum RSD (relative standard deviation) for an sample size Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
Sample size for design verification of variable in single use device Design and Development of Products and Processes 19
Sample size for clinical validation/investigation EU Medical Device Regulations 4
Process Validation sample size selection Statistical Analysis Tools, Techniques and SPC 0
Interesting Discussion Sample Size Determination for Medical Device Other Medical Device and Orthopedic Related Topics 9
Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Sample Size for Biocompatibility Tests Other Medical Device Related Standards 4
Firmware Verification Testing & Sample Size Software Quality Assurance 1
Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
Sample Size Calculation (Confidence Interval and Reliability) - Medical Devices Reliability Analysis - Predictions, Testing and Standards 6
Cleaning and Disinfection Validation Sample Size of 3? Other Medical Device Related Standards 2
Sample Size Calculation for Image Analysis - Microscopy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
G Sample Size, Significant Figures, Scale General Measurement Device and Calibration Topics 3
L Heated Sealed Packages - Sample Size for OQ (Operational Qualification) and PQ Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
P Frequency and Sample Size Requirements for an MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
When and How to apply the Bayes Success Run approach for Sample Size Determination Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
Sample Size Justification for Medical Device Shelf-Life Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
Process Qualification Sample size Document Control Systems, Procedures, Forms and Templates 6
Sample size for IEC 60601-1-11 environmental tests IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
IEC 60601-1-2 Sample Size issue Other Medical Device Related Standards 1
A How to calculate Mean and standard deviation without sample size in Minitab Using Minitab Software 5
What is the minimum Sample Size for Weibull Analysis Reliability Analysis - Predictions, Testing and Standards 12
C Sample Size for Stability Testing Other Medical Device and Orthopedic Related Topics 2
M Calculating Cpk when sample size equals to 1 Capability, Accuracy and Stability - Processes, Machines, etc. 12
F Power and sample size for factorial design Using Minitab Software 4
S Sample Subgroup Size for NP/P chart - nPbar>=5 Statistical Analysis Tools, Techniques and SPC 4
U Sample size to prove parts are good with 99.73% confidence Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
S Low Sample Size for Gage R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
SPC Card for One Sample Size - Customer requires for all critical dimensions Statistical Analysis Tools, Techniques and SPC 8
S Determining sample size for inspection to achieve x% confidence re defects Misc. Quality Assurance and Business Systems Related Topics 10
S Determining Sample Size - AQL & LQ AQL - Acceptable Quality Level 10
Bayes Success-Run on Attribute Data - Determining Sample Size Statistical Analysis Tools, Techniques and SPC 2
J Flammability: Testing Burn Rate with Improper Sample Size fmvss 302 Various Other Specifications, Standards, and related Requirements 1
H Factory Dictated Sample Size Justification Statistical Analysis Tools, Techniques and SPC 9
M Design Verification: "Difference" in Sample Size Calculations? Design and Development of Products and Processes 1
M Calculating Adequate Receiving Inspection Sample Size Statistical Analysis Tools, Techniques and SPC 2
S Salmonella Test Sample Size Food Safety - ISO 22000, HACCP (21 CFR 120) 3