Sample Size for Design Verification and Validation

G

gxzn008

#1
Recently, FDA auidted our site, they raised one question on sample size for design verification as follows:

Specifically, procedures for identifying valid statistical techniques required for verifying the acceptability of product characteristics of a new design during design verification and design validation testing have not been adequately established. For example, for the design validation testing of the new product design project, only one unit was tested. No valid statistical rationale was documented for the testing of only one unit.

So according to our practice, we only finish one prototype from Lab for design verification, so how to solve this problem, do we need manufacture more prototypes from Lab for design verification?

Can anyone help me clarify this?

Thanks.
 
Elsmar Forum Sponsor

somashekar

Staff member
Admin
#2
I have a question.

Samples scientifically represents a lot when the lot is happening in and under controlled conditions.
As you are talking about design verification and validation, of what use is the rest of the products if you take only one for the purpose ?
Hence I do not think you have any lot quantity at this design process, and I guess you are not making hundreds of products at this stage.

Kindly explain more.
 

Ronen E

Problem Solver
Staff member
Moderator
#3
Recently, FDA auidted our site, they raised one question on sample size for design verification as follows:

Specifically, procedures for identifying valid statistical techniques required for verifying the acceptability of product characteristics of a new design during design verification and design validation testing have not been adequately established. For example, for the design validation testing of the new product design project, only one unit was tested. No valid statistical rationale was documented for the testing of only one unit.

So according to our practice, we only finish one prototype from Lab for design verification, so how to solve this problem, do we need manufacture more prototypes from Lab for design verification?

Can anyone help me clarify this?

Thanks.
Hello,

Reading your post carefully, I noticed that the FDA's comment mostly related to VALIDATION, whilst your question talks about VERIFICATION. These are typically 2 separate stages. During design verification you are required to DEMONSTRATE, so a small sample may suffice (with the right justification); however, in validation you are required to PROVE, and hence the expectation for statistical rigour.

Further, please note that the FDA didn't focus on the sample size to itself, but on the lack of statistical justification thereof and/or lack of procedure(s) for identification of valid statistical techniques for determining sample size.

Had I been in your place, I would have considered the following measures:

(1) Establish a procedure(s) for identification of valid statistical techniques for determining sample size.

(2) Go through all open projects and recently completed ones and make sure all testing sample sizes are accounted for in terms of valid statistical techniques.

(3) Make sure that validation batch size and/or sampling and/or test sample size (as applicable) are representative of the expected / planned commercial production. Capture and document.

If one expects to commercially manufacture 1 machine (device) per year across 10 years, I guess sampling and statistical considerations would be irrelevant. BTW, you could use Bootstrap and other relevant techniques if you only practically have a very small sample available.

Cheers,
Ronen.
 
G

gxzn008

#4
Ronen, thanks for your post.

Actually for design validation, we can use confidence/reliability (95/90) as a basis to get the sample size (30) using the link http://elsmar.com/Forums/showthread.php?t=51211; so from your meaning, for design verfiication, we can provide the rationale for test.

Can anyone share the a procedure(s) for identification of valid statistical techniques for determining sample size?

Mike
 

Statistical Steven

Statistician
Staff member
Super Moderator
#5
Recently, FDA auidted our site, they raised one question on sample size for design verification as follows:

Specifically, procedures for identifying valid statistical techniques required for verifying the acceptability of product characteristics of a new design during design verification and design validation testing have not been adequately established. For example, for the design validation testing of the new product design project, only one unit was tested. No valid statistical rationale was documented for the testing of only one unit.

So according to our practice, we only finish one prototype from Lab for design verification, so how to solve this problem, do we need manufacture more prototypes from Lab for design verification?

Can anyone help me clarify this?

Thanks.
This is a HUGE misconception...sample size does NOT imply unique devices. If the variability within a device is equal to or greater than what I expect to see between devices I can do multiple tests on a single unit. If there is NO variability between tests (for example software) you can justify a sample size of 1 since there is NO variability between tests. Does this make things clearer?
 

Ronen E

Problem Solver
Staff member
Moderator
#6
This is a HUGE misconception...sample size does NOT imply unique devices. If the variability within a device is equal to or greater than what I expect to see between devices I can do multiple tests on a single unit. If there is NO variability between tests (for example software) you can justify a sample size of 1 since there is NO variability between tests. Does this make things clearer?
How do you PROVE the level of variability between non-software device units (e.g. machines), when you have only a few or just one?
 

Statistical Steven

Statistician
Staff member
Super Moderator
#7
How do you PROVE the level of variability between non-software device units (e.g. machines), when you have only a few or just one?
You have component level testing and other sources of testing to estimate variability. Typically it is a scientific judgement and good engineering assessment. Typically for large capital equipment many of the features are verified prior to release so the "risk" at V&V is lower.
 
Thread starter Similar threads Forum Replies Date
C Stress / Challenge Conditions for Design Verification Testing to Reduce Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 11
D Design Verification Sample Size vs Repeats Statistical Analysis Tools, Techniques and SPC 9
T Sample Size for Design Validation Design and Development of Products and Processes 4
E Sample size for design verification of variable in single use device Design and Development of Products and Processes 20
F Power and sample size for factorial design Using Minitab Software 4
M Design Verification: "Difference" in Sample Size Calculations? Design and Development of Products and Processes 1
A Determining Sample Size in Design V&V activities 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 44
A Sample Size for Design Verification (Prototype Testing) for Medical Devices Qualification and Validation (including 21 CFR Part 11) 12
S Sample Size Attribute DOE (Design of Experiments) Six Sigma 9
E Determining Sample Size for Design Verification and Design Validation Other Medical Device and Orthopedic Related Topics 20
M Sample Size in Design Verification & Process Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
K Design/Development Sample Size Needed to Qualify a Design Ploy Design and Development of Products and Processes 1
M Design Verification Sample Size - What philosophies are out there Design and Development of Products and Processes 5
Y FMEA link to Control Plan frequency and sample size FMEA and Control Plans 18
T Noob question regarding sample size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
S Help needed in choosing the method of calculating the minimum sample size Internal Auditing 12
D Estimate Sample Size for very rare event Using Minitab Software 1
S Determining Sample Size and Method Quality Tools, Improvement and Analysis 6
C Complaint Return Sample Size Question 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
shimonv Frequency and sample size for lot release of EtO sterilized products EU Medical Device Regulations 2
A Clinical assessment sample size - Medical device Class IIb implantable (93/42 directive) EU Medical Device Regulations 2
P Sample Size for Distribution Simulation Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
K How can Acceptance Number and Reject Number be larger than Sample Size on Z1.4 Table? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
J Validation Sample Size for Tray Seal Qualification and Validation (including 21 CFR Part 11) 4
M Minimum sample size - Guidance and statistical rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
P Unrealistic Packaging Validation Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 41
J Control chart for huge sample size Statistical Analysis Tools, Techniques and SPC 9
S Is there a standard for sample size during R&D phase Other ISO and International Standards and European Regulations 18
J Sample size definition in an Automotive SMT pilot lot run Misc. Quality Assurance and Business Systems Related Topics 1
E Correct way to certify hydrostatic testing when it is not 100% (and Sample Size) Various Other Specifications, Standards, and related Requirements 6
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
F Bayes Success run Theorem for sample size during OQ&PQ Qualification and Validation (including 21 CFR Part 11) 4
I Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
J Sample size for creating a data base as a reference to a tested variable Other Medical Device and Orthopedic Related Topics 6
C AS9138 Sample Size Determination AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
V Acceptable maximum RSD (relative standard deviation) for an sample size Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
R Sample size for clinical validation/investigation EU Medical Device Regulations 4
R Process Validation sample size selection Statistical Analysis Tools, Techniques and SPC 0
J Interesting Discussion Sample Size Determination for Medical Device Other Medical Device and Orthopedic Related Topics 19
P Is it possible to make an educated decision using a very very small sample size? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
F Sample Size for Biocompatibility Tests Other Medical Device Related Standards 4
V Firmware Verification Testing & Sample Size Software Quality Assurance 1
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
H Sample Size Calculation (Confidence Interval and Reliability) - Medical Devices Reliability Analysis - Predictions, Testing and Standards 9
P Cleaning and Disinfection Validation Sample Size of 3? Other Medical Device Related Standards 2
V Sample Size Calculation for Image Analysis - Microscopy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
G Sample Size, Significant Figures, Scale General Measurement Device and Calibration Topics 3
L Heated Sealed Packages - Sample Size for OQ (Operational Qualification) and PQ Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11

Similar threads

Top Bottom