# Sample Size for Design Validation

T

#### TSwilson

Hi All,

I am currently working on product validation testing for a medical device (US/FDA) and we in the process of trying to figure out how many devices are needed for each test. All of the validation testing we have to do amounts to making sure one or another of our products properties (e.g., shear strength, crush strength, etc.) is greater than our specification or in cases like number of particles released during use, less than our specification.

We have already built enough prototypes, samples, etc. that we have a very good idea of the average and “standard deviation” of these properties for our product (some properties clearly do not fall on a normal distribution). The average for each is typically well above our spec (or for particles, well below). For crush strength, the average is high enough, but the deviation is also quite high, though mostly due to some products having a very high crush strength – this was as expected due to the mechanical design, there are some interferences that can come into play to dramatically improve the crush tolerance. To be concrete, let’s say our spec for crush is >=10, the average failure is 14 and the deviation is 4.2 and our measurement precision is 0.5. (Some units can survive many times our spec, but again, none has ever failed to meet 10 – in fact the lowest shear we have encountered was on a device purposely built way out of spec and it failed at 10.8).

For other properties, such as shear strength, we again have to validate that the device survives up to at least our specified shear strength, but again we don’t care if it’s over, just if it’s under. For properties like the shear strength, our data looks like it could be a normal distribution. To give sample numbers, our shear spec is >=1.0 and our average shear at failure is 1.5 with a standard deviation of 0.25 and a measurement precision of 0.05.

Lastly, for particles shed by our device, we are well under the spec and our data does look vaguely like a normal distribution. Our spec is 100 and we measure an average of 30 with a standard deviation of 10 and a measurement precision of (about) 3.

How would I calculate the required sample size for these situations, all one-sided specs, where one is very likely not a normal distribution, one has an a lower bound but no upper bound, and the other has an upper bound, but no lower bound (down to zero)?

Our previous compliance consultants used a formula very similar to the one Bev D gave in an attachment to the thread entitled “Determining Sample Size for FDA Verification and Validation Activities,” but it’s not clear to me that this formula applies to any of these situations (I am however statistically challenged, so please feel free to set me straight).

Thanks in advance for your help,

-Tom

Elsmar Forum Sponsor

#### Ronen E

##### Problem Solver
Staff member
Moderator
Sounds like you are conducting design verification, not design validation.

#### michaelcwang2

##### Registered
I think there are several steps in general. It seems you done some preliminary experiments so with a few trial result available.
First, From to decide which distribution could apply better. See the link :
Then next step is to take appropriate "Operating Characteristics curve" for your distribution. As normal distribution is a kind of Poisson distribution, even with like Z transformation should be able to transform your nominal and tolerance to such distribution.
It seems we can also find OC curve available for other distributions like Lognormal distribution on web but not for publicity.

T

#### TSwilson

Sounds like you are conducting design verification, not design validation.
Hi Ronan,

Thanks for the speedy reply. These tests are all portions of tests being performed on devices under one of the relevant accepted FDA guidance documents for performance testing and labeling. As part of the determination of whether or not we meet the user needs, our compliance people are having us use the simulated use requirements from the guidance before some of the testing (and also comparing it to the predicate device). The three specs I mentioned above are some of the quantitative safety endpoints our compliance people gave us to test after simulated misuse of the devices based on risk analysis of the user needs. They have been calling this testing “design validation.” Maybe calling the series of tests "design verification and validation" is more accurate. I will bring this issue (validation/verification) up at the next meeting; thank you for bringing it to my attention. If calling it validation or verification affects the number of parts we should test, that would also be very useful to know.

-Tom

#### Ronen E

##### Problem Solver
Staff member
Moderator
Hi Ronan,

Thanks for the speedy reply. These tests are all portions of tests being performed on devices under one of the relevant accepted FDA guidance documents for performance testing and labeling. As part of the determination of whether or not we meet the user needs, our compliance people are having us use the simulated use requirements from the guidance before some of the testing (and also comparing it to the predicate device). The three specs I mentioned above are some of the quantitative safety endpoints our compliance people gave us to test after simulated misuse of the devices based on risk analysis of the user needs. They have been calling this testing “design validation.” Maybe calling the series of tests "design verification and validation" is more accurate. I will bring this issue (validation/verification) up at the next meeting; thank you for bringing it to my attention. If calling it validation or verification affects the number of parts we should test, that would also be very useful to know.

-Tom
Hi Tom,

To be honest, I'm a little confused / overwhelmed by the situation's description above (also taking into account your post #1 above). It is very important to sort out terms & definitions, roles and responsibilities, and a logical process flow, if you want to gain clarity.

Design Verification and Design Validation are normally separate activities with different goals and different characteristics. Bundling them together doesn't add clarity, though it might "solve" some org-internal difficulties (I wouldn't know).

It's also important to understand who is the process/activity owner, and according to what methodology they act. Normally it's R&D / D&D / PD / Engineering staff who own Design Verification and Design Validation, with input from "Compliance" (Regulatory Affairs? QA?) staff; and normally they'd follow the Design Control flow (e.g. ISO 13485 s. 7.3 or 21 CFR 820.30). What exact standards / guidance (or parts thereof) to apply and what exact tests to conduct should initially be part of preparing the Design Input and later revisited when preparing the Verification protocol(s) (later, Validation protocols too).

It might be useful to lay down the thought process that guides the various activities so it's easier to understand what stage it actually is and what formal requirements should apply. Currently the terms seem a little mixed-up / mis-sequenced (or at least not that clearly laid out) and I wonder how clear you all are on the process flow and how all the pieces fit together.

Thread starter Similar threads Forum Replies Date
Stress / Challenge Conditions for Design Verification Testing to Reduce Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 11
Design Verification Sample Size vs Repeats Statistical Analysis Tools, Techniques and SPC 9
E Sample size for design verification of variable in single use device Design and Development of Products and Processes 20
F Power and sample size for factorial design Using Minitab Software 4
M Design Verification: "Difference" in Sample Size Calculations? Design and Development of Products and Processes 1
Determining Sample Size in Design V&V activities 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 44
G Sample Size for Design Verification and Validation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
A Sample Size for Design Verification (Prototype Testing) for Medical Devices Qualification and Validation (including 21 CFR Part 11) 12
S Sample Size Attribute DOE (Design of Experiments) Six Sigma 9
E Determining Sample Size for Design Verification and Design Validation Other Medical Device and Orthopedic Related Topics 20
Sample Size in Design Verification & Process Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
Design/Development Sample Size Needed to Qualify a Design Ploy Design and Development of Products and Processes 1
M Design Verification Sample Size - What philosophies are out there Design and Development of Products and Processes 5
Clinical assessment sample size - Medical device Class IIb implantable (93/42 directive) EU Medical Device Regulations 2
Sample Size for Distribution Simulation Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
How can Acceptance Number and Reject Number be larger than Sample Size on Z1.4 Table? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
Validation Sample Size for Tray Seal Qualification and Validation (including 21 CFR Part 11) 1
Minimum sample size - Guidance and statistical rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Unrealistic Packaging Validation Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 25
Control chart for huge sample size Statistical Analysis Tools, Techniques and SPC 9
Is there a standard for sample size during R&D phase Other ISO and International Standards and European Regulations 18
Sample size definition in an Automotive SMT pilot lot run Misc. Quality Assurance and Business Systems Related Topics 1
Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
Bayes Success run Theorem for sample size during OQ&PQ Qualification and Validation (including 21 CFR Part 11) 4
Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
Sample size for creating a data base as a reference to a tested variable Other Medical Device and Orthopedic Related Topics 6
C AS9138 Sample Size Determination AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
Acceptable maximum RSD (relative standard deviation) for an sample size Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
R Sample size for clinical validation/investigation EU Medical Device Regulations 4
Process Validation sample size selection Statistical Analysis Tools, Techniques and SPC 0
J Interesting Discussion Sample Size Determination for Medical Device Other Medical Device and Orthopedic Related Topics 19
P Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Sample Size for Biocompatibility Tests Other Medical Device Related Standards 4
V Firmware Verification Testing & Sample Size Software Quality Assurance 1
Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
Sample Size Calculation (Confidence Interval and Reliability) - Medical Devices Reliability Analysis - Predictions, Testing and Standards 6
P Cleaning and Disinfection Validation Sample Size of 3? Other Medical Device Related Standards 2
Sample Size Calculation for Image Analysis - Microscopy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
G Sample Size, Significant Figures, Scale General Measurement Device and Calibration Topics 3
L Heated Sealed Packages - Sample Size for OQ (Operational Qualification) and PQ Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
P Frequency and Sample Size Requirements for an MSA Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
When and How to apply the Bayes Success Run approach for Sample Size Determination Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
N Sample Size Justification for Medical Device Shelf-Life Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
Process Qualification Sample size Document Control Systems, Procedures, Forms and Templates 6
Sample size for IEC 60601-1-11 environmental tests IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
IEC 60601-1-2 Sample Size issue Other Medical Device Related Standards 1
A How to calculate Mean and standard deviation without sample size in Minitab Using Minitab Software 5