Determining Sample Size for Design Verification and Design Validation

E

edisoar

#1
What is the most common process for identifying sample sizes for Design Verification / Design Validation activities within the Medical Device Industry? I have seen some references to AQL which I believe is incorrect. I have a suite of testing to complete and I want to identify how many pieces I need for each test. Any help would be appreciated.
 
Elsmar Forum Sponsor
E

edisoar

#2
Re: Sample size for Design Verification / Design Validation

To further clarify what I am looking for, there is a combination of variable and attribute data that I will be collecting. One option I am considering is a sample size of 20pcs for variable data and 59pcs for attribute data as these are what are typical used to prove normality.
 

harry

Super Moderator
#3
Last edited:
E

edisoar

#4
Re: Sample size for Design Verification / Design Validation

Thanks for this information. I did not see the attachment when I reviewed the original posts. I will use this and see what the outcome is.
 
J

jscholen

#5
Re: Sample size for Design Verification / Design Validation

Here's how it's been done in some of my circles. You need to identify what is critical vs "does it work as needed" (write in your own terms)...then establish what confidence level( or other criteria) you want for what you identified, ie:
Critical 95/99
Acceptable 95/95
Adequate 95/90 or 95/80

As to variable data, I have seen it done 2 ways....treat it as attribute data (in-spec-Pass/Outta spec-Fail) or evaluate to establish your confidence interval, T tests, etc..

Typically in Development, there is a push to get a design out the door, hence treating as much data as attribute data will help, but if you are also focused on making sure you can have a process that can build what you designed, make sure you work with manufacturing to develop test plans that gives them confidence that they can produce your design at higher volumes.
 
S

someguyinback

#6
Re: Sample size for Design Verification / Design Validation

My comment concerns design verification ONLY.

What I see is that for design verification, one has, at least in our case, a limited number of "near production" devices (ten, in our case) that "work". This means they don't blow up, the software runs long enough to complete a procedure and the R&D folks believe that what is supposed to happen at the business end seems to be happening.
Now we want to Verify the design; confirming the as designed and built system has specifications (design output) that meet the design inputs.

How many do we test (there are only 10 in existence)? How many times do we test them? How do we satisfactorily justify whatever we do?

There is no body of quantifiable data, the actual specifications need to be written the burn continues and the clock is ticking.

No set of equations, handbook or forum post has to date been able to help me answer these 3 questions above.

Thanks for any assistance.
 
B

Burnett

#7
Re: Sample size for Design Verification / Design Validation

Did you ever resolve this? I'm having the exact same question.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#8
Re: Sample size for Design Verification / Design Validation

we've had several discussion regarding this in other threads (some of which are shown at the bottom of thsi page...)

the quick answer is that for verification and validation the "AQL" inspection tables are not sufficient. One must use the 'estimation' sample size formulas. The numbers used for these formulas are derived from the nature of thing being validated. If a new thing you must usually meet label claim within some delta with soem confidence. and this is often reviewed and agreed upon with your regulatory agency - if you have one - or your customer. If the thing is a revision you are usually looking for no substantial difference (layman's terms = equivalence) and this typically requires that the delta of interest and beta and alpha errors be specified...again with agreement from yoru regulatory agency.

I know that prototype sample sizes can be realtivley small but this doesn't faze the FDA.

in the case of medical devices there are several aspects to sample size: one is the number of instruments and the other is the number of samples or patients etc. typically regulatory agencies focus on the number of samples (or patients) over the number of instruments. If the instrument is a consumable the question is moot. if it is re-usable then it is highly advisable to consult your agency or customer...
 
B

Burnett

#9
Re: Sample size for Design Verification / Design Validation

Thanks Bev D. Basically I'm taking the approach of using Tolerance Intervals and k table to develop my testing. The only hiccup is how to reconcile that with the k table and other stat tables, n can be very small and much lower than the standard practice of 30pcs minimum. How is this justified statistically? Must I always use 30pcs min in spite of the table?
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#10
Re: Sample size for Design Verification / Design Validation

using tolerance intervals for V&V is not a typical approach. What are you physically trying to doin the testing? can you describe it better? as with anything the best answer depends on the specifics of your situation.

(and the old wives tale about n=30 is just that, it's a rule of thumb for estimates when you know nothing about the process and have no real 'goal' other thatn to 'get some idea' about what the average and/or SD might be. it certainly has no real statistical basis...)
 
Thread starter Similar threads Forum Replies Date
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
S Determining sample size for inspection to achieve x% confidence re defects Misc. Quality Assurance and Business Systems Related Topics 10
S Determining Sample Size - AQL & LQ AQL - Acceptable Quality Level 10
B Bayes Success-Run on Attribute Data - Determining Sample Size Statistical Analysis Tools, Techniques and SPC 2
A Determining Sample Size in Design V&V activities 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 44
G Determining Sample Size for Equipment Qualification Studies Qualification and Validation (including 21 CFR Part 11) 10
D Determining Sample Size for Capability in Six Sigma Project Six Sigma 5
H Determining the Correct Sample Size when AQL points to two Sample Sizes AQL - Acceptable Quality Level 7
L Determining Cpk based on a given Sample Size, Confidence Level, & Reliability Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
M Determining Sample Size needed for Operational Qualification (OQ) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 24
T Determining the appropriate Sample Size using an AQL Table Inspection, Prints (Drawings), Testing, Sampling and Related Topics 29
J Functional Test Sampling - Determining Sample Size to eliminate 100% Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
W Sampling Plan problem - Determining Sample Size for both Attribute and Variables Data ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
S Determining the Total Sample Size using AQL - AQL Lot Quantity question AQL - Acceptable Quality Level 8
1 Determining AQL and Sample Plan - Batch size is from 1-500 AQL - Acceptable Quality Level 24
N Auditing Proof of Delivery signed by the customer - Determining Sample Size General Auditing Discussions 6
P Determining Sample Size - Filling a bag with three different types of candy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 31
Q Determining Sample Size for FDA Verification and Validation Activities Inspection, Prints (Drawings), Testing, Sampling and Related Topics 41
D Determining sample size required based on a 95% confidence level Reliability Analysis - Predictions, Testing and Standards 7
B Determining sample size for drop test and waterproof testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
T Determining Receiving Inspection Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
N Determining Sample Size for Attribute Data Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
C Internal Audit Sampling Plans - Determining Internal Audit Sample Size Internal Auditing 45
Q Determining the Appropriate Sample Size for Process Validation/Qualification Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
T Using Cpk for the purpose of determining sample size/frequencies Capability, Accuracy and Stability - Processes, Machines, etc. 2
G Determining Sample Size and Frequency for Features on Control Plans FMEA and Control Plans 8
S Determining sample sizes for PQ Qualification and Validation (including 21 CFR Part 11) 4
R Determining Sample Sizes for Testing (Chemistry or Microbiology) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
B Selecting a Sample and Determining the Reference Value for a Bias Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
S Determining the requirements for the products and services ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
Q Determining Adverse Effects of Corrective/Preventive Actions ISO 13485:2016 - Medical Device Quality Management Systems 2
M Determining if an Insulin Pen Testing Machine is a Medical Device? EU Medical Device Regulations 4
M Determining a tolerance value for Measuring devices in-house inspection General Measurement Device and Calibration Topics 12
D Determining the the maximum number of reprocessing cycles of attachments CE Marking (Conformité Européene) / CB Scheme 2
R Determining Uncertainty from Gage R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
A IATF 16949 4.3.1 - Determining the scope of the quality management system - supplemental IATF 16949 - Automotive Quality Systems Standard 9
D ISO 9001:2015 4.3 Determining the Scope of the QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
J Determining SPC tolerance Statistical Analysis Tools, Techniques and SPC 21
B Determining SAT Offsets vs TUS Offsets per SAE AMS 2750E AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 0
R Question on determining defective units - I am not recording fixture to part rejected Statistical Analysis Tools, Techniques and SPC 5
S Clause 8.2.2 Determining the requirements for products and services ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Determining Calibration Frequency schedule for items used in production Manufacturing and Related Processes 2
C Determining if Maintenance Contractor is an External Service subject to ISO 9001 Clause 8.4 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 43
S AS9100D PEAR - Examples for organization's method for determining process results? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
I Determining Calibration Tolerance of a Measurement Device General Measurement Device and Calibration Topics 2
J ISO 17025 Documented Procedure for 6.2.5 - Determining competency ISO 17025 related Discussions 4
V Determining FDA 820 (registration) vs ISO 13485 - Supplier gives us the kit ISO 13485:2016 - Medical Device Quality Management Systems 1

Similar threads

Top Bottom