The Cove Business Standards Discussion Forums
Go Back   The Elsmar Cove Business Systems and Standards Discussion Forums > > > >
Forum Username

Elsmar Cove Forum Visitor Notice(s)

Wooden Line

Sample Size for Design Verification and Validation

Monitor the Elsmar Forum
Courtesy Quick Links


Links Elsmar Cove visitors will find useful in the quest for knowledge and support:

Jennifer Kirley's
Conway Business Services


Howard's
International Quality Services


Marcelo Antunes'
SQR Consulting, and
Medical Devices Expert Forum


Bob Doering
Bob Doering's Blogs and,
Correct SPC - Precision Machining


Ajit Basrur
Claritas Consulting, LLC



International Standards Bodies - World Wide Standards Bodies

AIAG - Automotive Industry Action Group

ASQ - American Society for Quality

International Organization for Standardization - ISO Standards and Information

NIST's Engineering Statistics Handbook

IRCA - International Register of Certified Auditors

SAE - Society of Automotive Engineers

Quality Digest

IEST - Institute of Environmental Sciences and Technology


Some Related Topic Tags
design and development, design validation, design verification, sample size, samples and sampling plans, statistical analysis and studies
Reply
 
Thread Tools Search this Thread Rate Thread Content Display Modes
  Post Number #1  
Old 6th November 2011, 09:52 AM
gxzn008

 
 
Total Posts: 13
Please Help! Sample Size for Design Verification and Validation

Recently, FDA auidted our site, they raised one question on sample size for design verification as follows:

Specifically, procedures for identifying valid statistical techniques required for verifying the acceptability of product characteristics of a new design during design verification and design validation testing have not been adequately established. For example, for the design validation testing of the new product design project, only one unit was tested. No valid statistical rationale was documented for the testing of only one unit.

So according to our practice, we only finish one prototype from Lab for design verification, so how to solve this problem, do we need manufacture more prototypes from Lab for design verification?

Can anyone help me clarify this?

Thanks.

Sponsored Links
  Post Number #2  
Old 6th November 2011, 10:15 AM
somashekar's Avatar
somashekar

 
 
Total Posts: 5,371
Re: Sample size for design verification and validation

I have a question.

Samples scientifically represents a lot when the lot is happening in and under controlled conditions.
As you are talking about design verification and validation, of what use is the rest of the products if you take only one for the purpose ?
Hence I do not think you have any lot quantity at this design process, and I guess you are not making hundreds of products at this stage.

Kindly explain more.
Thanks to somashekar for your informative Post and/or Attachment!
  Post Number #3  
Old 6th November 2011, 06:15 PM
Ronen E

 
 
Total Posts: 3,736
Re: Sample Size for Design Verification and Validation

Quote:
In Reply to Parent Post by gxzn008 View Post

Recently, FDA auidted our site, they raised one question on sample size for design verification as follows:

Specifically, procedures for identifying valid statistical techniques required for verifying the acceptability of product characteristics of a new design during design verification and design validation testing have not been adequately established. For example, for the design validation testing of the new product design project, only one unit was tested. No valid statistical rationale was documented for the testing of only one unit.

So according to our practice, we only finish one prototype from Lab for design verification, so how to solve this problem, do we need manufacture more prototypes from Lab for design verification?

Can anyone help me clarify this?

Thanks.
Hello,

Reading your post carefully, I noticed that the FDA's comment mostly related to VALIDATION, whilst your question talks about VERIFICATION. These are typically 2 separate stages. During design verification you are required to DEMONSTRATE, so a small sample may suffice (with the right justification); however, in validation you are required to PROVE, and hence the expectation for statistical rigour.

Further, please note that the FDA didn't focus on the sample size to itself, but on the lack of statistical justification thereof and/or lack of procedure(s) for identification of valid statistical techniques for determining sample size.

Had I been in your place, I would have considered the following measures:

(1) Establish a procedure(s) for identification of valid statistical techniques for determining sample size.

(2) Go through all open projects and recently completed ones and make sure all testing sample sizes are accounted for in terms of valid statistical techniques.

(3) Make sure that validation batch size and/or sampling and/or test sample size (as applicable) are representative of the expected / planned commercial production. Capture and document.

If one expects to commercially manufacture 1 machine (device) per year across 10 years, I guess sampling and statistical considerations would be irrelevant. BTW, you could use Bootstrap and other relevant techniques if you only practically have a very small sample available.

Cheers,
Ronen.
Thanks to Ronen E for your informative Post and/or Attachment!
  Post Number #4  
Old 8th November 2011, 10:14 AM
gxzn008

 
 
Total Posts: 13
Re: Sample Size for Design Verification and Validation

Ronen, thanks for your post.

Actually for design validation, we can use confidence/reliability (95/90) as a basis to get the sample size (30) using the link http://elsmar.com/Forums/showthread.php?t=51211; so from your meaning, for design verfiication, we can provide the rationale for test.

Can anyone share the a procedure(s) for identification of valid statistical techniques for determining sample size?

Mike
  Post Number #5  
Old 8th November 2011, 01:05 PM
Statistical Steven's Avatar
Statistical Steven

 
 
Total Posts: 1,129
Re: Sample Size for Design Verification and Validation

Quote:
In Reply to Parent Post by gxzn008 View Post

Recently, FDA auidted our site, they raised one question on sample size for design verification as follows:

Specifically, procedures for identifying valid statistical techniques required for verifying the acceptability of product characteristics of a new design during design verification and design validation testing have not been adequately established. For example, for the design validation testing of the new product design project, only one unit was tested. No valid statistical rationale was documented for the testing of only one unit.

So according to our practice, we only finish one prototype from Lab for design verification, so how to solve this problem, do we need manufacture more prototypes from Lab for design verification?

Can anyone help me clarify this?

Thanks.
This is a HUGE misconception...sample size does NOT imply unique devices. If the variability within a device is equal to or greater than what I expect to see between devices I can do multiple tests on a single unit. If there is NO variability between tests (for example software) you can justify a sample size of 1 since there is NO variability between tests. Does this make things clearer?
Thank You to Statistical Steven for your informative Post and/or Attachment!
  Post Number #6  
Old 8th November 2011, 05:50 PM
Ronen E

 
 
Total Posts: 3,736
Re: Sample Size for Design Verification and Validation

Quote:
In Reply to Parent Post by Statistical Steven View Post

This is a HUGE misconception...sample size does NOT imply unique devices. If the variability within a device is equal to or greater than what I expect to see between devices I can do multiple tests on a single unit. If there is NO variability between tests (for example software) you can justify a sample size of 1 since there is NO variability between tests. Does this make things clearer?
How do you PROVE the level of variability between non-software device units (e.g. machines), when you have only a few or just one?
Thanks to Ronen E for your informative Post and/or Attachment!
  Post Number #7  
Old 9th November 2011, 09:17 AM
Statistical Steven's Avatar
Statistical Steven

 
 
Total Posts: 1,129
Re: Sample Size for Design Verification and Validation

Quote:
In Reply to Parent Post by Ronen E View Post

How do you PROVE the level of variability between non-software device units (e.g. machines), when you have only a few or just one?
You have component level testing and other sources of testing to estimate variability. Typically it is a scientific judgement and good engineering assessment. Typically for large capital equipment many of the features are verified prior to release so the "risk" at V&V is lower.
Thanks to Statistical Steven for your informative Post and/or Attachment!
Reply

Lower Navigation Bar
Go Back   The Elsmar Cove Business Systems and Standards Discussion Forums > > > >

Bookmarks



Visitors Currently Viewing this Thread: 1 (0 Registered Visitors (Members) and 1 Unregistered Guest Visitors)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Forum Search
Display Modes Rate Thread Content
Rate Thread Content:

Forum Posting Settings
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Emoticons are On
[IMG] code is On
HTML code is Off


Similar Discussion Threads
Discussion Thread Title Thread Starter Forum Replies Last Post or Poll Vote
Determining Sample Size for FDA Verification and Validation Activities QualityNewbie Inspection, Prints (Drawings), Testing, Sampling and Related Topics 41 11th March 2015 05:34 AM
Design Verification Sample Size - What philosophies are out there MNPIKE Design and Development of Products and Processes 5 16th February 2013 12:12 PM
Determining Sample Size for Design Verification and Design Validation edisoar Other Medical Device and Orthopedic Related Topics 20 3rd December 2012 07:32 PM
Sample Size for Design Verification (Prototype Testing) for Medical Devices amadeuscurse Qualification and Validation (including 21 CFR Part 11) 12 27th July 2011 04:56 AM
Sample Size in Design Verification & Process Validation MasterBB Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2 16th November 2009 09:00 PM



The time now is 09:27 PM. All times are GMT -4.
Your time zone can be changed in your UserCP --> Options.



Misc. Internal Links


NOTE: This forum uses "Cookies"