Sample Size for Design Verification (Prototype Testing) for Medical Devices

[amadeuscurse]

I have looked through various threads @ this forum and found some information as to what I need.
I need to figure out the number of samples during design verification (prototype testing) for medical devices.
We manufacture capital equipment (read pretty expensive prototypes) and I am trying to figure out justification for using 3, 5 or 7 samples (as I have read in other threads) as having sufficient confidence level and reliability to proceed w/ manufacturing.
I did see some posts by Bev D about the same. I could not find any statistically sound argument to justify the small sample size.
Any help would be appreciated.
Thank you in advance.
-Amadeus​

 

[SteveK]

Re: Sample size for prototype testing

Hi Amadeus,

Welcome!

We had external testing for Class IIa portable electrical device based on one item. It was supposed to be the final design (rather than prototype) but needed modifications to meet EMC (EN 60601-1-2) requirements. Don’t know if this helps. It has not been questioned by our NB.

Steve

 

[amadeuscurse]

Thank you Steve.
I have been telling the developers to come up with the sample size based on the confidence interval and repeatability of testing. I am comfortable with that approach. During a recent FDA audit, we were told that we need to have some sort of statistical rationale for the sample size.
I am struggling to come up with one since there does not seem to be a whole lot of sample size calculations for capital equipment (or maybe there is and I am not aware of it)
-Amadeus

 

[Statistical Steven]

This topic has come up over and over again throughout the medical device industry. A sampling unit does not have to be a unique prototype but might be a multiple tests on the same unit.

 

[Bev D]

exactly so Steven. for medical devices or any instrument really that isn't single use, it's the number of runs or uses that constitute sample size.

this is particularly true in DESIGN verification as opposed to process or manufacturign validation, which woudl involve the manufcture of multiple devices to ensure that the manufacturign process is capable of meetign the requirements over the allowable range of inputs. (but even in these cases the number of actual devices need not be large)

 

[amadeuscurse]

Thank you Bev and Steven.
Thats excatly the rationale I provided to the FDA auditor, as long as the testing meets the confidence level and reliability criteria, we need to test 'x' samples based on the complexity of the instrument.
The FDA auditor did not like that and was looking for statistically 'sound rationale'. All my effort to explain the rationale fell to deaf ears. This is how we do it now
We determine the confidence level (say 95%) and reliability (say 90%) to verify design variability. We look at the mean, st dev of the test results to justify the number of samples we test.
I was asked to put in some statistically sound rationale around this.
I looked around and could not come up with one...any help or thoughts on how to proceed will be helpful.
-Amadeus

 

[Statistical Steven]

You can use statistical sample size rationale to justify your sample size. How you divide those samples in the design verification is your decision. Look at Dimitri Kececioglu, Reliability and Life Testing Handbook Page 47 for a sample size equation based on confidence and reliability.

 

[amadeuscurse]

Thank you Steven.
I looked through page 47 of the book as you suggested. It talks about 'exact binomial confidence limits'.
Are you suggesting that I calculate the sample size based on the known values of confidence limits and reliability values?
Thanks in advance for your help.
-Amadeus

 

[Statistical Steven]

Thank you Steven.
I looked through page 47 of the book as you suggested. It talks about 'exact binomial confidence limits'.
Are you suggesting that I calculate the sample size based on the known values of confidence limits and reliability values?
Thanks in advance for your help.
-Amadeus

You can use the exact binomial interval to calculate the sample size for a given confidence and reliability value. Assume you have calculate the sample size to be 14, you can choose to test 2 instruments 7 times each.

 

[Bev D]

Steven got you headed down a good path regardign sample size determination so let me look at the issue from a slightly different angle.

It sounds like you chose a sample size (total number of runs over several devices without using any particular statistical formula? Then after teh tests were completed you calculated and applied the confidence intervals (with your standard confidence and reliability levels). The since the resutls given teh confidence interval met your acceptance criteria and you accepted the resutls? is this correct?