Sample Size Justification for Medical Device Shelf-Life

NicoleinFlorida

Inactive Registered Visitor
#1
Hello everyone,

I have a question about how to determine the sample size for medical device shelf-life test.Our device is Class II medical device, and expected shelf-life is 3 years.

Shelf-life test will be conducted with five stages:
1. Time 0 No aging
2. 6-Month accelerated aging (for 510k)
3. 36-Month accelerated aging
4. 6-Month real-time aging
5. 36-Month real-time aging

During each stage, package tests (integrity, seal strength, and 100% visual inspection) and device performance tests will be conducted.

In our protocol, we used the AQL Level I that required 32 samples for each test. I am considering whether we can reduce this number with a rational statistical justification.

According to the thread about V&V sample size, I am considering to use R=0.95, C=0.95 which gives a N=59. So does this "N" means the number of devices for each tests? Or if we have three tests for the package, then can N be divided by 3, which means that we can use 20 samples?

I will really appreciate your responses.
 
Elsmar Forum Sponsor

Bev D

Heretical Statistician
Staff member
Super Moderator
#2
it really depends on what you are trying to do. Although the AQL sampling plans are NOT appropriate for these types of tests.

If you only want to demonstrate that some percentage of your product will survive beyond some specified period of time, the confidence and reliability plan will suffice. the sample size applies to EACH of the tests you specified. time zero through 36 month real time...you may not need to perform each of those but regardless the sample size applies to each.

If you are trying to model the stability to use continuous data to predict the failure rate over time the sample size will be based on the part to part variation at time zero and you will need to use continuous sample size calculations for this approach. and again the sample size will apply to each time period tested.

Some people will select smaller sample sizes based on convenience but they are statistically justifiable. depending on your reviewer and the nature of your product this may be acceptable or it may not.
 

NicoleinFlorida

Inactive Registered Visitor
#3
Hello Bev D
Thank you for the response.

The objective of this test is to establish the shelf-life of our devices. Therefore, your first recommendation of confidence and reliability plan fits the best.

The materials used on our devices are mainly PC and stainless steels, whose shelf-life will be much longer than the expected 3 years. Therefore, the sterile barrier system (package) will be the most critical for the establishment of the shelf-life. In this way, could we use smaller sample sizes for the devices?

For the package, because it is the most critical part for shelf-life testing, we would like to adopt the confidence and reliability plan.

Thank you again.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#4
It doesn't matter what you are testing for shelf life. The sample size remains the same. You may be able to test only the packaging without using the PC inside of it - but this isn't a statistical question it's a physics question and a question for your reviewer...does the shape and size of what's going to be inside the packaging effect the shelf life of the packaging?
 

Statistical Steven

Statistician
Staff member
Super Moderator
#5
Nicole

I have a few questions for you to get a better understanding of your issues.

  1. Does 6 months accelerated aging equate to 36 months real time based on your Arrenhius model?
  2. Why are you doing 36 months accelerated aging if you are doing 36 month real time testing?
  3. What are you only testing 6 and 36 months real time? For stability studies it is beneficial to have more time points than more samples per time point, especially if you have a failure at 36 months, as your label claim would only be 6 months based on your data.
  4. If you are doing go/no testing at each time point, sample sizes will be large since as Bev mentioned you can only make confidence and reliability statements (could be the rationale for accelerated conditions). Are there any continuous type tests that can be done?


Hello everyone,

I have a question about how to determine the sample size for medical device shelf-life test.Our device is Class II medical device, and expected shelf-life is 3 years.

Shelf-life test will be conducted with five stages:
1. Time 0 No aging
2. 6-Month accelerated aging (for 510k)
3. 36-Month accelerated aging
4. 6-Month real-time aging
5. 36-Month real-time aging

During each stage, package tests (integrity, seal strength, and 100% visual inspection) and device performance tests will be conducted.

In our protocol, we used the AQL Level I that required 32 samples for each test. I am considering whether we can reduce this number with a rational statistical justification.

According to the thread about V&V sample size, I am considering to use R=0.95, C=0.95 which gives a N=59. So does this "N" means the number of devices for each tests? Or if we have three tests for the package, then can N be divided by 3, which means that we can use 20 samples?

I will really appreciate your responses.
 

NicoleinFlorida

Inactive Registered Visitor
#6
Hello Steven,
To answer your questions:
1. No.6 month accelerated aging is for 510k application.
2. We would like to label our product with 3 year shelf-life with the accelerated aging data. And the 3 year real-time aging is to validate the accelerated aging.
3. Do you mean that we can have more time points with smaller sample size? For instance, in our original plan, we will test 32 samples at 6 month and another 32 samples at 36 month. So your suggestion is to test maybe 10 samples at 6 month, 12 month, 18 month, 24 month, 30 month and 36 month? According to Bev's comment, I have to ensure the sample size is large enough for each test at each stage. So if I would like to do more tests throughout the 36 months, I have to increase the sample size dramatically?
4. Our device is a single use device, so I am not sure whether we can have continuous test data for the shelf-life. The objective of the tests after the accelerated aging is to test whether the device is functioning as intended. Basically, it will be a go/no-go test. Could you provide some examples of the continuous test data?

In general, the device is not cheap and we would like to reduce the sample size for the shelf-life testing but with rational justification. Could you give me more advises?

I really appreciate your response.
 

Statistical Steven

Statistician
Staff member
Super Moderator
#7
See my comments in RED
Hello Steven,
To answer your questions:
1. No.6 month accelerated aging is for 510k application.
Assuming a Q=2 that is typical for medical devices. No issues there.
2. We would like to label our product with 3 year shelf-life with the accelerated aging data. And the 3 year real-time aging is to validate the accelerated aging.
Again pretty standard for a medical device (assume it's injection molded).
3. Do you mean that we can have more time points with smaller sample size? For instance, in our original plan, we will test 32 samples at 6 month and another 32 samples at 36 month. So your suggestion is to test maybe 10 samples at 6 month, 12 month, 18 month, 24 month, 30 month and 36 month? According to Bev's comment, I have to ensure the sample size is large enough for each test at each stage. So if I would like to do more tests throughout the 36 months, I have to increase the sample size dramatically?
My point is that at 6 and 36 you can test 32 samples for your 95/90 at each time point. I would do some sampling between 6 and 36 months to get some ensure that the product still functions. Maybe n=10 for a 95/75 assurance at 12 and 24 months.
4. Our device is a single use device, so I am not sure whether we can have continuous test data for the shelf-life. The objective of the tests after the accelerated aging is to test whether the device is functioning as intended. Basically, it will be a go/no-go test. Could you provide some examples of the continuous test data?
If the test is just a go/no-go test then stability is not really what you are testing per se. Continuous testing would be measuring the force values, or tensile strength or other measurement. I am always weary of functioning as intended testing, as it is a very subjective criteria that does not allow for an assessment of degradation over time. I will give you an example form a past life. We made a connector for tubing that was accelerated life tested over six months. When connected it needed to function as intended, meaning no leaks. We connected them to tubing, and testing showing no leaks. So the test passed. We never tested if the membrane was more or less brittle or other physical characteristics of the product. Just my :2cents:
In general, the device is not cheap and we would like to reduce the sample size for the shelf-life testing but with rational justification. Could you give me more advises?

I really appreciate your response.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#8
a few other clarifications:

'continuous test data' means measuring a continuous variable as opposed to an attribute or categorical variable. continuous variables being measureable such as force, tensile strength, permeability, etc. and categorical data being count data such as the number of passing or failing parts.

categorical data will almost always result in larger sample sizes than continuous data.

if you are highly confident that the 36 month test will pass, you only need to test at that time period. but as Steven pointed out, you may wan tot play it safe and test some parts at interim time points just case they fail earlier than you might expect. in this case sample size doesn't have to be 'statistically' justified as you are doing it to protect yourself from an early failure that you otherwise might not detect until 36 months. of course smaller sample sizes at interim time points increases the risk of missing a slight degradation before 36 months. however this is your business risk to determine...

sample size at interim time points only matters if you intend to 'model' the degradation. While you can do this for categorical data the sample sizes would be very large to get a meaningful model and it doesn't sound like this would be helpful to at this time.
 

Statistical Steven

Statistician
Staff member
Super Moderator
#9
Bev

My suggest sample size of 10 at interim time points is to find a catastrophic issue. Truth is that a sample size at t=0 and t=36 months with a c=0 sampling plan really has no statistical power. If it intended to find large issues with "stability". I tend not to use such small sample sizes for performance based testing, but for package integrity testing where either all the samples will usually pass or a large percent will fail (greater than 10%). A sample size of 32 is about 95/90 and even if you pool the samples to get a sample size of 64, that still approximately 95/95, so 5% defective rate can still pass 95% of the time :).

Just my :2cents:

Thanks for your continued contributions!


a few other clarifications:

'continuous test data' means measuring a continuous variable as opposed to an attribute or categorical variable. continuous variables being measureable such as force, tensile strength, permeability, etc. and categorical data being count data such as the number of passing or failing parts.

categorical data will almost always result in larger sample sizes than continuous data.

if you are highly confident that the 36 month test will pass, you only need to test at that time period. but as Steven pointed out, you may wan tot play it safe and test some parts at interim time points just case they fail earlier than you might expect. in this case sample size doesn't have to be 'statistically' justified as you are doing it to protect yourself from an early failure that you otherwise might not detect until 36 months. of course smaller sample sizes at interim time points increases the risk of missing a slight degradation before 36 months. however this is your business risk to determine...

sample size at interim time points only matters if you intend to 'model' the degradation. While you can do this for categorical data the sample sizes would be very large to get a meaningful model and it doesn't sound like this would be helpful to at this time.
 

NicoleinFlorida

Inactive Registered Visitor
#10
Hello Ben and Steven,

Thank you for all the clarification.

The materials of our devices will not degrade a lot throughout shelf-life, especially the inner mechanism which is made of stainless steel. The integrity of the package actually will mainly determine the shelf-life. However, we need to have finished devices inside the package to perform the package integrity test, shipping simulation test and device performance test in order to establish the shelf-life.

I agree with you that adding more interim time points can provide more protection for us, and may identify failures early.

For the stability testing, we assess the performance of our devices, results in pass/fail categorical data. In our case, unfortunately, we cannot use continuous data to reduce the sample size :(
 
Thread starter Similar threads Forum Replies Date
H Factory Dictated Sample Size Justification Statistical Analysis Tools, Techniques and SPC 9
Q Sample Size justification for IEC 60601-1 Testing IEC 60601 - Medical Electrical Equipment Safety Standards Series 4
G Sampling Plan - Sample Size Justification for a Bottling Cleaning Process Validation Qualification and Validation (including 21 CFR Part 11) 2
J Validation Sample Size for Tray Seal Qualification and Validation (including 21 CFR Part 11) 1
D Design Verification Sample Size vs Repeats Statistical Analysis Tools, Techniques and SPC 9
M Minimum sample size - Guidance and statistical rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
P Unrealistic Packaging Validation Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 13
J Control chart for huge sample size Statistical Analysis Tools, Techniques and SPC 9
S Is there a standard for sample size during R&D phase Other ISO and International Standards and European Regulations 18
J Sample size definition in an Automotive SMT pilot lot run Misc. Quality Assurance and Business Systems Related Topics 1
E Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
F Bayes Success run Theorem for sample size during OQ&PQ Qualification and Validation (including 21 CFR Part 11) 4
I Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
J Sample size for creating a data base as a reference to a tested variable Other Medical Device and Orthopedic Related Topics 6
T Sample Size for Design Validation Design and Development of Products and Processes 4
C AS9138 Sample Size Determination AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
V Acceptable maximum RSD (relative standard deviation) for an sample size Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
E Sample size for design verification of variable in single use device Design and Development of Products and Processes 19
R Sample size for clinical validation/investigation EU Medical Device Regulations 4
R Process Validation sample size selection Statistical Analysis Tools, Techniques and SPC 0
J Interesting Discussion Sample Size Determination for Medical Device Other Medical Device and Orthopedic Related Topics 9
P Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
F Sample Size for Biocompatibility Tests Other Medical Device Related Standards 4
V Firmware Verification Testing & Sample Size Software Quality Assurance 1
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
H Sample Size Calculation (Confidence Interval and Reliability) - Medical Devices Reliability Analysis - Predictions, Testing and Standards 6
P Cleaning and Disinfection Validation Sample Size of 3? Other Medical Device Related Standards 2
V Sample Size Calculation for Image Analysis - Microscopy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
G Sample Size, Significant Figures, Scale General Measurement Device and Calibration Topics 3
L Heated Sealed Packages - Sample Size for OQ (Operational Qualification) and PQ Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
P Frequency and Sample Size Requirements for an MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
V When and How to apply the Bayes Success Run approach for Sample Size Determination Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
C Process Qualification Sample size Document Control Systems, Procedures, Forms and Templates 6
G Sample size for IEC 60601-1-11 environmental tests IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
G IEC 60601-1-2 Sample Size issue Other Medical Device Related Standards 1
A How to calculate Mean and standard deviation without sample size in Minitab Using Minitab Software 5
D What is the minimum Sample Size for Weibull Analysis Reliability Analysis - Predictions, Testing and Standards 12
C Sample Size for Stability Testing Other Medical Device and Orthopedic Related Topics 2
M Calculating Cpk when sample size equals to 1 Capability, Accuracy and Stability - Processes, Machines, etc. 12
F Power and sample size for factorial design Using Minitab Software 4
S Sample Subgroup Size for NP/P chart - nPbar>=5 Statistical Analysis Tools, Techniques and SPC 4
U Sample size to prove parts are good with 99.73% confidence Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
F Sample Size to get Statistically Valid Data Measurement Uncertainty (MU) 6
S Low Sample Size for Gage R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
S SPC Card for One Sample Size - Customer requires for all critical dimensions Statistical Analysis Tools, Techniques and SPC 8
S Determining sample size for inspection to achieve x% confidence re defects Misc. Quality Assurance and Business Systems Related Topics 10
S Determining Sample Size - AQL & LQ AQL - Acceptable Quality Level 10

Similar threads

Top Bottom