Interesting Discussion Sample Size Determination for Medical Device

J
#1
We will perform design verification on medical device (portable tabletop) intended for diagnostic purposes. Looking for sample size rational to perform the functional/performance characteristic challenges for design verification purposes. Below is a statistical rational I'm proposing. Looking for feedback if this rational is justifiable. If not, what would appropriate statistical rational.
Is this sample size rational valid for reusable capital equipment?
Can I divide number of trials or samples into multiple devices?

"Using the Binomial distribution to calculate the sample size at 95% confidence the probability to detect no more than zero defect present out of 95% of the population would require at-least 59 number of trials or samples. Since the device is reusable capital equipment the number of trials will be divided into three devices where each device will perform 20 trials with total of 60 trials. This will discover any inherit risk present in the device. The device gets 100% functionally tested during manufacturing."
 
Elsmar Forum Sponsor

yodon

Staff member
Super Moderator
#2
What is the intended user / patient population? Can different users / patients (age, weight, sex, race, ...) have any effect? What kind of tests are these? Mechanical, performance, etc.? That might have a bearing. Are you saying that only 3 devices will be used, with each undergoing 20 tests?

Sorry I can't answer your question but maybe this will help stimulate discussion.
 
J
#4
What is the intended user / patient population? Can different users / patients (age, weight, sex, race, ...) have any effect? What kind of tests are these? Mechanical, performance, etc.? That might have a bearing. Are you saying that only 3 devices will be used, with each undergoing 20 tests?

Sorry I can't answer your question but maybe this will help stimulate discussion.
Hi Yodon, yes this would be 3 devices each undergoing 20 tests vs. producing 60 units to collect data from 1 trial with each device. Do you think it would be valid to perform multiple trials across 3 units to collect the 60 trials worth of data for design verification testing (mechanical and performance testing)?
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#5
In my organization we manufacture diagnostic instruments so we a similar dilemma.
we calculate the required sample size as the number of runs necessary on a single instrument.
and because performance isn't homogenous across instruments we use run that sample size on multiple instruments.

we find that this is much more informative
 
Last edited:

Mark Meer

Trusted Information Resource
#6
I'm not certain I'm following the rationale here. We're talking design verification, correct?

Suppose you have 60 requirements to verify and, for simplicity, have developed 60 corresponding verification tests. It seems as though the discussion is between either:
(a) Using a single device and do all 60 tests; or
(b) Using 3 devices, and dividing the tests among them (so each unit is subject to 20 tests).

It's unclear how you justify selecting one or the other. If you acknowledge that performance isn't homogeneous across devices, then shouldn't you be performing all 60 tests on multiple units?
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#7
If the performance wasn’t homogenous across devices (and it often isn’t, so we always assume it isn’t until we demonstrate that it is) we would test all characteristics across multiple instruments with multiple runs. Often the number of runs will range into the 50-100 range or more depending on what we are looking for. For our 1 time use devices we will test up to 100 or more devices from 3 independent lots.

for characteristics that are sensitive to specific use conditions (protocol or specimen conditions) we can manage the sample size by directed testing: we test under worst case conditions to ensure that we have margin.
 

Mark Meer

Trusted Information Resource
#8
If the performance wasn’t homogenous across devices (and it often isn’t, so we always assume it isn’t until we demonstrate that it is) we would test all characteristics across multiple instruments with multiple runs. Often the number of runs will range into the 50-100 range or more depending on what we are looking for. For our 1 time use devices we will test up to 100 or more devices from 3 independent lots.
Thank for the clarification Bev. But I'm afraid (as is often the case) I still don't think I'm grasping this - at least not how your approach can be applied in practice...

Again, we are talking design verification, correct?
If so, how is it that you have so many units, and multiple independent lots before the design has even been verified? I presume you're working with a product that can be easily (and inexpensively) produced in quantity?
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#9
I’m not an expert in hearing aids (my only exposure is that my wife wears them and I’m in charge of replacing the batteries).



Usually these things are not about some statistical calculation of the sample size, although that is important. It’s about understanding what constitutes the ‘population’.



In design verify, a re-usable device such as a hearing aid would have a smaller sample size of the device than a single use device. Sample size will still depend on what characteristics you need to verify.


Although some characteristics are deterministic - either it works or it doesn’t - some variation will have to be included. For example battery fit. Batteries have a specified size range and your design has a specified size range. At a minimum I would expect that you would need to ensure that the minimum and maximum battery size fits in the minimum and maximum specified sizes of your hearing aids. So the sample size is determined by counting the conditions to be verified.



Another example: I’m sure you have requirements regarding the wear of the battery compartment. How many battery changes are you targeting? that and your confidence level as well as the margin you need for your requirement will tell you the sample size – how many insertions do you need? Of course you could go to directed testing around stress vs strength that would not require actual insertions until validation with ‘production’ level parts.


Some characteristics are subject to inherent variation from use conditions: for example, I can imagine that you need to prove that the hearing aid meets output requirements given a range of sound volume and frequency as well as ‘fit’ in the ear? In this case your sample size is more dependent on the range of each condition than anything else. The number of devices is less important than the number of conditions. Do you just test at the extremes and the nominal? Or across the range in a distribution of conditions that is representative of the conditions that exist in actual use? It is also dependent on whether or not the hearing aid is consistent in its performance across the range. In this case of design verify you can usually have a small number of devices tested across a full range of conditions. I typically would recommend 3 devices just to be safe. The ‘number of conditions tested on each device’ is then the real sample size.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#10
Thank for the clarification Bev. But I'm afraid (as is often the case) I still don't think I'm grasping this - at least not how your approach can be applied in practice...

Again, we are talking design verification, correct?
If so, how is it that you have so many units, and multiple independent lots before the design has even been verified? I presume you're working with a product that can be easily (and inexpensively) produced in quantity?
it might depend on what you use 'design verify' for. there are many active working definitions of this. In our case we do design verify with prototypes. One of our product types is a one time use device that is a diagnostic assay. we always demonstrate at the prototype stage that our design can meet our targeted sensitivity and specificity claims. this requires more than 100 blood samples to be tested in a matched pair fashion. Each sample must be tested on the gold standard and a prototype device. since each device is a single use, we use over hundred devices. we also build 3 prototype lots because that we know that there is a difference between lots; it is the nature of biologicals. this isn't what I call cheap and we must do it in design verify to verify that we are ready for to move to the validation stage when we submit to the USDA for licensure. We have the same approach for larger instruments that are multiple use. we do make a small number of them but we runs hundreds of runs on each...
 
Thread starter Similar threads Forum Replies Date
C AS9138 Sample Size Determination AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
V When and How to apply the Bayes Success Run approach for Sample Size Determination Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
Statistical Steven Continuous Data Analysis Sample Size Determination Six Sigma 3
A Destructive Test Sample Size Determination Statistical Analysis Tools, Techniques and SPC 4
P Repetition (sample size) for measuring equipment accuracy determination General Measurement Device and Calibration Topics 1
K Sample Size Determination - Medical Devices Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
S Determination of sample size for product audit IATF 16949 - Automotive Quality Systems Standard 1
shimonv Frequency and sample size for lot release of EtO sterilized products EU Medical Device Regulations 2
A Clinical assessment sample size - Medical device Class IIb implantable (93/42 directive) EU Medical Device Regulations 2
P Sample Size for Distribution Simulation Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
C Stress / Challenge Conditions for Design Verification Testing to Reduce Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 11
K How can Acceptance Number and Reject Number be larger than Sample Size on Z1.4 Table? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
J Validation Sample Size for Tray Seal Qualification and Validation (including 21 CFR Part 11) 4
D Design Verification Sample Size vs Repeats Statistical Analysis Tools, Techniques and SPC 9
M Minimum sample size - Guidance and statistical rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
P Unrealistic Packaging Validation Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 41
J Control chart for huge sample size Statistical Analysis Tools, Techniques and SPC 9
S Is there a standard for sample size during R&D phase Other ISO and International Standards and European Regulations 18
J Sample size definition in an Automotive SMT pilot lot run Misc. Quality Assurance and Business Systems Related Topics 1
E Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
F Bayes Success run Theorem for sample size during OQ&PQ Qualification and Validation (including 21 CFR Part 11) 4
I Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
J Sample size for creating a data base as a reference to a tested variable Other Medical Device and Orthopedic Related Topics 6
T Sample Size for Design Validation Design and Development of Products and Processes 4
V Acceptable maximum RSD (relative standard deviation) for an sample size Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
E Sample size for design verification of variable in single use device Design and Development of Products and Processes 20
R Sample size for clinical validation/investigation EU Medical Device Regulations 4
R Process Validation sample size selection Statistical Analysis Tools, Techniques and SPC 0
P Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
F Sample Size for Biocompatibility Tests Other Medical Device Related Standards 4
V Firmware Verification Testing & Sample Size Software Quality Assurance 1
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
H Sample Size Calculation (Confidence Interval and Reliability) - Medical Devices Reliability Analysis - Predictions, Testing and Standards 9
P Cleaning and Disinfection Validation Sample Size of 3? Other Medical Device Related Standards 2
V Sample Size Calculation for Image Analysis - Microscopy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
G Sample Size, Significant Figures, Scale General Measurement Device and Calibration Topics 3
L Heated Sealed Packages - Sample Size for OQ (Operational Qualification) and PQ Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
P Frequency and Sample Size Requirements for an MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
N Sample Size Justification for Medical Device Shelf-Life Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
C Process Qualification Sample size Document Control Systems, Procedures, Forms and Templates 6
G Sample size for IEC 60601-1-11 environmental tests IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
G IEC 60601-1-2 Sample Size issue Other Medical Device Related Standards 1
A How to calculate Mean and standard deviation without sample size in Minitab Using Minitab Software 5
D What is the minimum Sample Size for Weibull Analysis Reliability Analysis - Predictions, Testing and Standards 12
C Sample Size for Stability Testing Other Medical Device and Orthopedic Related Topics 2
M Calculating Cpk when sample size equals to 1 Capability, Accuracy and Stability - Processes, Machines, etc. 12
F Power and sample size for factorial design Using Minitab Software 4

Similar threads

Top Bottom