Sample Size for Design Validation

#1
Hi All,

I am currently working on product validation testing for a medical device (US/FDA) and we in the process of trying to figure out how many devices are needed for each test. All of the validation testing we have to do amounts to making sure one or another of our products properties (e.g., shear strength, crush strength, etc.) is greater than our specification or in cases like number of particles released during use, less than our specification.

We have already built enough prototypes, samples, etc. that we have a very good idea of the average and “standard deviation” of these properties for our product (some properties clearly do not fall on a normal distribution). The average for each is typically well above our spec (or for particles, well below). For crush strength, the average is high enough, but the deviation is also quite high, though mostly due to some products having a very high crush strength – this was as expected due to the mechanical design, there are some interferences that can come into play to dramatically improve the crush tolerance. To be concrete, let’s say our spec for crush is >=10, the average failure is 14 and the deviation is 4.2 and our measurement precision is 0.5. (Some units can survive many times our spec, but again, none has ever failed to meet 10 – in fact the lowest shear we have encountered was on a device purposely built way out of spec and it failed at 10.8).

For other properties, such as shear strength, we again have to validate that the device survives up to at least our specified shear strength, but again we don’t care if it’s over, just if it’s under. For properties like the shear strength, our data looks like it could be a normal distribution. To give sample numbers, our shear spec is >=1.0 and our average shear at failure is 1.5 with a standard deviation of 0.25 and a measurement precision of 0.05.

Lastly, for particles shed by our device, we are well under the spec and our data does look vaguely like a normal distribution. Our spec is 100 and we measure an average of 30 with a standard deviation of 10 and a measurement precision of (about) 3.

How would I calculate the required sample size for these situations, all one-sided specs, where one is very likely not a normal distribution, one has an a lower bound but no upper bound, and the other has an upper bound, but no lower bound (down to zero)?

Our previous compliance consultants used a formula very similar to the one Bev D gave in an attachment to the thread entitled “Determining Sample Size for FDA Verification and Validation Activities,” but it’s not clear to me that this formula applies to any of these situations (I am however statistically challenged, so please feel free to set me straight).

Thanks in advance for your help,

-Tom
 
Elsmar Forum Sponsor
#3
I think there are several steps in general. It seems you done some preliminary experiments so with a few trial result available.
First, From to decide which distribution could apply better. See the link :
Then next step is to take appropriate "Operating Characteristics curve" for your distribution. As normal distribution is a kind of Poisson distribution, even with like Z transformation should be able to transform your nominal and tolerance to such distribution.
It seems we can also find OC curve available for other distributions like Lognormal distribution on web but not for publicity.
 
#4
Sounds like you are conducting design verification, not design validation.
Hi Ronan,

Thanks for the speedy reply. These tests are all portions of tests being performed on devices under one of the relevant accepted FDA guidance documents for performance testing and labeling. As part of the determination of whether or not we meet the user needs, our compliance people are having us use the simulated use requirements from the guidance before some of the testing (and also comparing it to the predicate device). The three specs I mentioned above are some of the quantitative safety endpoints our compliance people gave us to test after simulated misuse of the devices based on risk analysis of the user needs. They have been calling this testing “design validation.” Maybe calling the series of tests "design verification and validation" is more accurate. I will bring this issue (validation/verification) up at the next meeting; thank you for bringing it to my attention. If calling it validation or verification affects the number of parts we should test, that would also be very useful to know.

-Tom
 

Ronen E

Problem Solver
Staff member
Moderator
#5
Hi Ronan,

Thanks for the speedy reply. These tests are all portions of tests being performed on devices under one of the relevant accepted FDA guidance documents for performance testing and labeling. As part of the determination of whether or not we meet the user needs, our compliance people are having us use the simulated use requirements from the guidance before some of the testing (and also comparing it to the predicate device). The three specs I mentioned above are some of the quantitative safety endpoints our compliance people gave us to test after simulated misuse of the devices based on risk analysis of the user needs. They have been calling this testing “design validation.” Maybe calling the series of tests "design verification and validation" is more accurate. I will bring this issue (validation/verification) up at the next meeting; thank you for bringing it to my attention. If calling it validation or verification affects the number of parts we should test, that would also be very useful to know.

-Tom
Hi Tom,

To be honest, I'm a little confused / overwhelmed by the situation's description above (also taking into account your post #1 above). It is very important to sort out terms & definitions, roles and responsibilities, and a logical process flow, if you want to gain clarity.

Design Verification and Design Validation are normally separate activities with different goals and different characteristics. Bundling them together doesn't add clarity, though it might "solve" some org-internal difficulties (I wouldn't know).

It's also important to understand who is the process/activity owner, and according to what methodology they act. Normally it's R&D / D&D / PD / Engineering staff who own Design Verification and Design Validation, with input from "Compliance" (Regulatory Affairs? QA?) staff; and normally they'd follow the Design Control flow (e.g. ISO 13485 s. 7.3 or 21 CFR 820.30). What exact standards / guidance (or parts thereof) to apply and what exact tests to conduct should initially be part of preparing the Design Input and later revisited when preparing the Verification protocol(s) (later, Validation protocols too).

It might be useful to lay down the thought process that guides the various activities so it's easier to understand what stage it actually is and what formal requirements should apply. Currently the terms seem a little mixed-up / mis-sequenced (or at least not that clearly laid out) and I wonder how clear you all are on the process flow and how all the pieces fit together.
 
Thread starter Similar threads Forum Replies Date
E Sample size for design verification of variable in single use device Design and Development of Products and Processes 19
F Power and sample size for factorial design Using Minitab Software 4
M Design Verification: "Difference" in Sample Size Calculations? Design and Development of Products and Processes 1
A Determining Sample Size in Design V&V activities 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 44
G Sample Size for Design Verification and Validation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
A Sample Size for Design Verification (Prototype Testing) for Medical Devices Qualification and Validation (including 21 CFR Part 11) 12
S Sample Size Attribute DOE (Design of Experiments) Six Sigma 9
E Determining Sample Size for Design Verification and Design Validation Other Medical Device and Orthopedic Related Topics 20
M Sample Size in Design Verification & Process Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
K Design/Development Sample Size Needed to Qualify a Design Ploy Design and Development of Products and Processes 1
M Design Verification Sample Size - What philosophies are out there Design and Development of Products and Processes 5
P Unrealistic Packaging Validation Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 13
J Control chart for huge sample size Statistical Analysis Tools, Techniques and SPC 9
S Is there a standard for sample size during R&D phase Other ISO and International Standards and European Regulations 18
J Sample size definition in an Automotive SMT pilot lot run Misc. Quality Assurance and Business Systems Related Topics 1
E Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 3
F Bayes Success run Theorem for sample size during OQ&PQ Qualification and Validation (including 21 CFR Part 11) 4
I Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
J Sample size for creating a data base as a reference to a tested variable Other Medical Device and Orthopedic Related Topics 6
C AS9138 Sample Size Determination AS9100, IAQG 9100, Nadcap and related Aerospace Standards and Requirements 1
V Acceptable maximum RSD (relative standard deviation) for an sample size Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
R Sample size for clinical validation/investigation EU Medical Device Regulations 4
R Process Validation sample size selection Statistical Analysis Tools, Techniques and SPC 0
J Interesting Discussion Sample Size Determination for Medical Device Other Medical Device and Orthopedic Related Topics 9
P Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
F Sample Size for Biocompatibility Tests Other Medical Device Related Standards 4
V Firmware Verification Testing & Sample Size Software Quality Assurance 1
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
H Sample Size Calculation (Confidence Interval and Reliability) - Medical Devices Reliability Analysis - Predictions, Testing and Standards 6
P Cleaning and Disinfection Validation Sample Size of 3? Other Medical Device Related Standards 2
V Sample Size Calculation for Image Analysis - Microscopy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
G Sample Size, Significant Figures, Scale General Measurement Device and Calibration Topics 3
L Heated Sealed Packages - Sample Size for OQ (Operational Qualification) and PQ Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
P Frequency and Sample Size Requirements for an MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
V When and How to apply the Bayes Success Run approach for Sample Size Determination Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
N Sample Size Justification for Medical Device Shelf-Life Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
C Process Qualification Sample size Document Control Systems, Procedures, Forms and Templates 6
G Sample size for IEC 60601-1-11 environmental tests IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
G IEC 60601-1-2 Sample Size issue Other Medical Device Related Standards 1
A How to calculate Mean and standard deviation without sample size in Minitab Using Minitab Software 5
D What is the minimum Sample Size for Weibull Analysis Reliability Analysis - Predictions, Testing and Standards 12
C Sample Size for Stability Testing Other Medical Device and Orthopedic Related Topics 2
M Calculating Cpk when sample size equals to 1 Capability, Accuracy and Stability - Processes, Machines, etc. 12
S Sample Subgroup Size for NP/P chart - nPbar>=5 Statistical Analysis Tools, Techniques and SPC 4
U Sample size to prove parts are good with 99.73% confidence Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
F Sample Size to get Statistically Valid Data Measurement Uncertainty (MU) 6
S Low Sample Size for Gage R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
Similar threads


















































Top Bottom