# Sample Size Calculation (Confidence Interval and Reliability) - Medical Devices

#### Harini17

##### Involved In Discussions
Dear Statisticians and regulatory experts,

Kindly help me to solve this problem for one of the medical device project. I asked many statisticians, but none were able to give convincing answer from the regulatory standpoint as you know FDA is very strict with the evidences and rationals. I found the similar problem on this forum but couldn't find the end answer.

Problem: The manufacturer of medical endoscope device equipment (pretty expensive prototypes) and they are trying to figure out justification for using 5, 7 or 10 samples with sufficient confidence level and reliability to proceed w/ manufacturing.
Testing such as bend cycling and environmental testing etc which they can do in house, for these tests sample sizes need to be determined (for each test). *These sample sizes can be calculated based on confidence and reliability levels, but a precedent should be set for the confidence/reliability that is needed for these devices and testing based on the risk level of the testing and the expense of running these tests.

At least with the examples of similar kind with formula and method!

I have very limited statistical knowledge. Kindly help me to solve this problem with the solutions from the regulatory standpoint.

Thank you!

Last edited:

#### Harini17

##### Involved In Discussions
Thank you Paulag.
I would like to know the reliability and confidence interval percentages that we can to use for these tests, so that FDA accepts. I would like to know the FDA requirements for these tests.

Thank you!

#### Bev D

##### Heretical Statistician
Staff member
Super Moderator
I am afraid you are asking for something that cannot be answered to your satisfaction. Sample sizes that will 'satisfy the FDA' are not easy or straightforward. You cannot simply plug numbers into a formula. We can give you the formula (I believe that it is in the attachment in the above post) but YOU must come up with the numbers that go into the formula based on your organizations assessment of the risk (hazard) of each characteristic that you will be testing.

The formula that incorporates Reliability with a specified confidence level is:

n = LN(1-C)/LN(R)

n= sample size
LN is the natural log
1-C is 1- confidence level (traditionally this is 95% or 99%. a higher confidence level is selected if the hazard is high)
R is the 'Reliability'. This is 1-defect rate that you can tolerate. again a severe hazard would require a pretty low defect rate

This formula requires that NO DEFECTS be found in the sample in order to accept the validation results.

This formula is based on using only pass/fail categorical results. you can get smaller sample sizes if you have continuous data where you can make predictions regarding the distribution vs. specification limits. These typically require a statistician or at least someone who is well versed in these types of statistics.

Severe hazards require large sample sizes per this Reliability/Confidence formula - although this typically results in the lowest sample sizes possible. Often companies will balk at high sample sizes and so the FDA provides the ability to provide a rationale based on balancing cost of testing vs. cost of the hazard. you can plug various numbers into the formula and see the effect for yourself.

Beyond this you have the complication of understanding that sample size applies to both the number of instruments/products and the number of USES of the product. The reliability/confidence formula typically applies to the number of instruments and the number of uses is based on the either the specified number of maximum uses or the maximum expected number of uses.

#### Statistical Steven

##### Statistician
Staff member
Super Moderator
Thank you for sharing my presentation. I have updated it slightly, but the ideas are the same. There is no "Standard" confidence and reliability. You need to tie it back to your risk management program so that your sample sizes (confidence and reliability) are commensurate with the risk to the patient (reliability component).

#### Harini17

##### Involved In Discussions
Thank you, Bev D and Statistical Steven...I have read all your threads addressing these questions thank so much. You both are doing good work in solving these regulatory obstacles. Special thanks to Bev D for such detailed explanation!

Last edited:

#### v9991

Trusted Information Resource
Often companies will balk at high sample sizes and so the FDA provides the ability to provide a rationale based on balancing cost of testing vs. cost of the hazard. you can plug various numbers into the formula and see the effect for yourself.

while FDA is providing enough direction, there is not even an outline/framework or guidance from agency for same. got certain references on risk from PDA & ISPE ; but certainly not ones which reference cost of testvshazard.!

1. point is, agency has ultimate power (veto to say the system/practice is inadequate, non-compliant) but what it says is ...all theory, no reference/ example, guidance, framework; (I remember an section on risk benefit analysis on clinical trial data evaluation!., but no such thing on validations/sample size.)

2. is this one reason, why (especially) pharma is way behind in adopting these tools and techniques. (its only recently that 'statistical' rationale/analysis gaining releac env

#### MeganG

##### Registered
Thanks all for this, which has been incredibly informative for determining sample size.

In Paulag's attachment, the example for continuous data states "Unknown Standard Deviation and Delta, but want to detect a 0.5 standard deviation shift." How would one decide the shift that is necessary to detect?

#### Bev D

##### Heretical Statistician
Staff member
Super Moderator
What is the difference you can tolerate? That is how you determine it.

#### MeganG

##### Registered
Thanks Bev D. I'm curious what would determine when to use the "sample size for continuous data" formula vs. the k-value formula.

Assuming the same specification as the example, for 95% confidence and 95% reliability, I would get a sample size 52 using the continuous data formula vs sample size 21 using the k-value formula.

Is it only existing information (mean, standard deviation) that justifies using the lower sample size?

Thread starter Similar threads Forum Replies Date
Sample Size Calculation for Image Analysis - Microscopy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
J Minitab Sample Size Calculation: 1 Sample t vs Minimum Sample Size for Means Using Minitab Software 3
P Sample Size Calculation help needed Reliability Analysis - Predictions, Testing and Standards 4
C Sample Size Calculation Formula for Process Capability Statistical Analysis Tools, Techniques and SPC 25
P Sample Size for Capability Calculation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 33
D Validate 99.5% reliability 90% confidence - Sample size calculation Reliability Analysis - Predictions, Testing and Standards 4
U The Importance of Correct Sample Size Calculation - Greek Alphabet Soup Book, Video, Blog and Web Site Reviews and Recommendations 7
T Sample Size Calculation for PPM (Parts Per Million) Prediction Statistical Analysis Tools, Techniques and SPC 6
Clinical assessment sample size - Medical device Class IIb implantable (93/42 directive) EU Medical Device Regulations 2
Sample Size for Distribution Simulation Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
Stress / Challenge Conditions for Design Verification Testing to Reduce Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 11
How can Acceptance Number and Reject Number be larger than Sample Size on Z1.4 Table? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
Validation Sample Size for Tray Seal Qualification and Validation (including 21 CFR Part 11) 3
Design Verification Sample Size vs Repeats Statistical Analysis Tools, Techniques and SPC 9
Minimum sample size - Guidance and statistical rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Unrealistic Packaging Validation Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 41
Control chart for huge sample size Statistical Analysis Tools, Techniques and SPC 9
Is there a standard for sample size during R&D phase Other ISO and International Standards and European Regulations 18
Sample size definition in an Automotive SMT pilot lot run Misc. Quality Assurance and Business Systems Related Topics 1
Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
Bayes Success run Theorem for sample size during OQ&PQ Qualification and Validation (including 21 CFR Part 11) 4
Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
Sample size for creating a data base as a reference to a tested variable Other Medical Device and Orthopedic Related Topics 6
T Sample Size for Design Validation Design and Development of Products and Processes 4
C AS9138 Sample Size Determination AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
Acceptable maximum RSD (relative standard deviation) for an sample size Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
E Sample size for design verification of variable in single use device Design and Development of Products and Processes 20
R Sample size for clinical validation/investigation EU Medical Device Regulations 4
Process Validation sample size selection Statistical Analysis Tools, Techniques and SPC 0
J Interesting Discussion Sample Size Determination for Medical Device Other Medical Device and Orthopedic Related Topics 19
P Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Sample Size for Biocompatibility Tests Other Medical Device Related Standards 4
V Firmware Verification Testing & Sample Size Software Quality Assurance 1
Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
P Cleaning and Disinfection Validation Sample Size of 3? Other Medical Device Related Standards 2
G Sample Size, Significant Figures, Scale General Measurement Device and Calibration Topics 3
L Heated Sealed Packages - Sample Size for OQ (Operational Qualification) and PQ Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
P Frequency and Sample Size Requirements for an MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
When and How to apply the Bayes Success Run approach for Sample Size Determination Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
N Sample Size Justification for Medical Device Shelf-Life Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
Process Qualification Sample size Document Control Systems, Procedures, Forms and Templates 6
Sample size for IEC 60601-1-11 environmental tests IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
IEC 60601-1-2 Sample Size issue Other Medical Device Related Standards 1
A How to calculate Mean and standard deviation without sample size in Minitab Using Minitab Software 5
D What is the minimum Sample Size for Weibull Analysis Reliability Analysis - Predictions, Testing and Standards 12
C Sample Size for Stability Testing Other Medical Device and Orthopedic Related Topics 2
M Calculating Cpk when sample size equals to 1 Capability, Accuracy and Stability - Processes, Machines, etc. 12