Determining Sample Size for FDA Verification and Validation Activities

Bev D

Heretical Statistician
Staff member
Super Moderator
#31
59 is correct sample for the given condition of 95% Reliability with 95% Confidence. This is NOT an AQL plan.

A Reliability of 95% means that you want to reject the lot if you have 5% defective or more. Confidence means that you want to reject that defective rate 95% of the time.

the formula is:

n= LN(1-Confidence)/LN(Reliability)

so, n = LN(1-.95)/LN(.95) = 58.40397
sample sizes should be rounded UP, so n = 59.

However this is an acceptance sampling plan and for OQ validation the sample size should be calculated differently as you are trying to estimate the actual defect rate...acceptance sampling only provides some assurance that the defect rate is probably not greater than some value for 'most' lots. That isn't a good approach for OQ validation.

you can use the acceptance sampling plan approach for your PQ validation lots.
 
Elsmar Forum Sponsor

Bev D

Heretical Statistician
Staff member
Super Moderator
#33
It really depends on what characteristics you are testing for (continuous data or defect data) and what the acceptable defect rate is. the sample size document I attached on the first page of this thread should help you get started.

in validation it's not so much the sample size but the nature of the possible defects, the testing methods and the structure of the OQ experiment that matters

If you could provide more information about the thing you are validating that would help us give you more specific advice.
I would also ask you to describe what you think the goal of OQ is about...
 

joemar

Involved In Discussions
#34
Hi Bev D,

Sorry for the delay responding, I didnt notice that the thread rolled over to a new page.

So my OQ goes though a chart of changing variables (seal times and temperatures, with a constant pressure) to identify a reasonable baseline average result for each data point. Then we use that information to set an optimal point, and an upper and lower range of seal time and temperature values that produce seals of sufficient strength and integrity to give us high confidence our products will maintain sterility through out packing, shipping and shelf life. We are fairly conservative in our upper and lower bounds, so we make sure not to produce a a bad seal in process.

Once we set our our upper and lower bounds based on the OQ data, we pick the optimal sealing temp and pressure that is safely within this range. We then confirm the values chosen for the optimal, plus the upper and lower limits found in the OQ in our PQ.

So the goal is to create a sample at each data point that is large enough (but not so large that it is overly burdensome to produce and test) to confirm that our sample at that particular point is representative of the normal deviation and unlikely to produce a significant number of outliers, so that our PQ of 59 samples at that point will accurately confirm that our a seal will not be defective.

does that make sense?
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#35
yes, this should be perfectly acceptable for PQ. Remember that n=59 doesn't' provide a very precise estimate of the actual defect rate. it can't really tell you with any certainty that the new process is any better than the old one. it can only tell you that it is not substantially worse. The plan of n=59 with zero defects determines if the lot is acceptable.
 

joemar

Involved In Discussions
#36
Thanks.

And then how would you suggest that I formulate the sample size for each of my OQ data points? In a year, we produce about 400,000 units, in lot sizes of 3000. I want to make sure that values produced at each data point are faithful representations of the in process production values that would be generated. so I know that my PQ is measuring the right thing.
 

joemar

Involved In Discussions
#37
Just to be clear, I guess im trying to figure out the minimum number of samples required at each data point to feel confident that the average of those samples is legitimately representative of the overall production data. I guess I just want enough to prove that im not using an outlier as my data point.

Any thoughts?
 

Statistical Steven

Statistician
Staff member
Super Moderator
#38
This is a running issue with me, as when performing validation (process validation), we are concerned with number of runs, but when determining the total sample size, I use c=0 sampling., which you believe in inappropriate. How can we get the total sample size when doing validation?

Are you referring to an acceptance sampling plan or validation plan?
My organization manufactures both consumables (high volume) and instruments (low volume). So I have faced the same dilemma.

ACCEPTANCE SAMPLING (aka RELEASE TESTING)
We test 100% of our instruments. the sample size applies to the number of runs and is often only 1. If you are asking about acceptance sampling I can elaborate...but since this is a validation thread:

VALIDATION TESTING:
In general, for validation we test smaller numbers of instruments, but will test many runs on those instruments. We apply the 'sample size' calculations to the number of runs (the patient sample is the sampling unit) than the number of instruments. We will typically use only a few instruments. The thought is that each instrument at new product launch is as varied as the other given the complex nature of the assembly and components. We will specify that 3 lots of consumables need to be used (in keeping with PQ principles).

When we validate a changed component we will typically use 6 instruments, randomly assigning 3 of the new components and 3 of the current version to each of 6 instruments. We will then perform a number of runs on each instrument (statistically based on the largest component of variation of the instrument system; this could be a fairly small number as we can often use continuous data statistics, but has been quite large when testing for a rather rare error code - categorical data) and then we will change out the components so that an instrument that started with a new component gets a current version component and rerun the testing. The analysis is done as a paired t-test (block = instrument) within the instruments. This provides us a lot of power while minimizing the sample size...

If you are servicing the Human medical market, your FDA reviewer and/or statistician can provide more specific guidance on the acceptable sample sizes for validation...I can only provide non-binding advice. :notme:
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#39
This is a running issue with me, as when performing validation (process validation), we are concerned with number of runs, but when determining the total sample size, I use c=0 sampling., which you believe in inappropriate. How can we get the total sample size when doing validation?
There are three issues that need to be addressed:
1. In validation (such as OQ), we are better served by dealing with continuous data whenever possible to get a better understanding of the performance of the process. Since we are trying to estimate the 'capability' (NOT a capability index) we should use the standard formulas and approaches for determining the sample size when making point estimates of the mean, standard deviation, differences in matched pairs, Se/Sp, dose response, etc.
If the characteristic of interest is an attribute and categorical data cannot be avoided, then we are still trying to estimate the defect rate and should be using the formulas for point estimates of an occurrence rate.

2.So why not use c=0 sample formulas for validation: they provide very little insight to the actual performance of the process. they are designed for acceptance sampling and not estimating a defect rate. so if I'm making a comparison of different input levels such as in process development or OQ, or a comparison of two different methods or suppliers or materials or if I am comparing a revised process to the old process, I will gain very little insight. one of the levels may be just passing while the other level is almost defect free. At zero defects in the (comparatively small) sample size I can make no judgment about the two - or more - levels. This could be setting me up for trouble later...

3. I do think that c=0 plans - or other acceptance sampling plans - are acceptable for PQ. PQ is intended as a last validation that the process (or product) performance at an acceptable level when run at 'nominal' over multiple set-ups, batches or lots (traditionally 3). so using a standard acceptance sampling plan is fine...by this time you should have proven that the process is designed to be capable.

does that make sense?
 

Statistical Steven

Statistician
Staff member
Super Moderator
#40
There are three issues that need to be addressed:
1. In validation (such as OQ), we are better served by dealing with continuous data whenever possible to get a better understanding of the performance of the process. Since we are trying to estimate the 'capability' (NOT a capability index) we should use the standard formulas and approaches for determining the sample size when making point estimates of the mean, standard deviation, differences in matched pairs, Se/Sp, dose response, etc.
If the characteristic of interest is an attribute and categorical data cannot be avoided, then we are still trying to estimate the defect rate and should be using the formulas for point estimates of an occurrence rate.

2.So why not use c=0 sample formulas for validation: they provide very little insight to the actual performance of the process. they are designed for acceptance sampling and not estimating a defect rate. so if I'm making a comparison of different input levels such as in process development or OQ, or a comparison of two different methods or suppliers or materials or if I am comparing a revised process to the old process, I will gain very little insight. one of the levels may be just passing while the other level is almost defect free. At zero defects in the (comparatively small) sample size I can make no judgment about the two - or more - levels. This could be setting me up for trouble later...

3. I do think that c=0 plans - or other acceptance sampling plans - are acceptable for PQ. PQ is intended as a last validation that the process (or product) performance at an acceptable level when run at 'nominal' over multiple set-ups, batches or lots (traditionally 3). so using a standard acceptance sampling plan is fine...by this time you should have proven that the process is designed to be capable.

does that make sense?
Oh it make sense to me...but when you have an attribute characteristic, Option 1 is not viable. I agree if comparing levels or vendors during OQ then c=0 is inappropriate. So how do you determine a sample size for a PQ when I am testing if a product can withstand a certain pressure for 30 seconds. The test is basically to apply a fixed pressure to the device for 30 seconds. If it does not break it passes. We have very few if any parts that fail. So my only recourse is a c=0 plan, but if I understand you correctly there might be a better way of generating a sample size. Any help would be appreciated.
 
Thread starter Similar threads Forum Replies Date
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
S Determining sample size for inspection to achieve x% confidence re defects Misc. Quality Assurance and Business Systems Related Topics 10
S Determining Sample Size - AQL & LQ AQL - Acceptable Quality Level 10
B Bayes Success-Run on Attribute Data - Determining Sample Size Statistical Analysis Tools, Techniques and SPC 2
A Determining Sample Size in Design V&V activities 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 44
G Determining Sample Size for Equipment Qualification Studies Qualification and Validation (including 21 CFR Part 11) 10
D Determining Sample Size for Capability in Six Sigma Project Six Sigma 5
H Determining the Correct Sample Size when AQL points to two Sample Sizes AQL - Acceptable Quality Level 7
L Determining Cpk based on a given Sample Size, Confidence Level, & Reliability Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
M Determining Sample Size needed for Operational Qualification (OQ) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 24
T Determining the appropriate Sample Size using an AQL Table Inspection, Prints (Drawings), Testing, Sampling and Related Topics 29
J Functional Test Sampling - Determining Sample Size to eliminate 100% Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
W Sampling Plan problem - Determining Sample Size for both Attribute and Variables Data ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
S Determining the Total Sample Size using AQL - AQL Lot Quantity question AQL - Acceptable Quality Level 8
1 Determining AQL and Sample Plan - Batch size is from 1-500 AQL - Acceptable Quality Level 24
E Determining Sample Size for Design Verification and Design Validation Other Medical Device and Orthopedic Related Topics 20
N Auditing Proof of Delivery signed by the customer - Determining Sample Size General Auditing Discussions 6
P Determining Sample Size - Filling a bag with three different types of candy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 31
D Determining sample size required based on a 95% confidence level Reliability Analysis - Predictions, Testing and Standards 7
B Determining sample size for drop test and waterproof testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
T Determining Receiving Inspection Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
N Determining Sample Size for Attribute Data Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
C Internal Audit Sampling Plans - Determining Internal Audit Sample Size Internal Auditing 45
Q Determining the Appropriate Sample Size for Process Validation/Qualification Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
T Using Cpk for the purpose of determining sample size/frequencies Capability, Accuracy and Stability - Processes, Machines, etc. 2
G Determining Sample Size and Frequency for Features on Control Plans FMEA and Control Plans 8
S Determining sample sizes for PQ Qualification and Validation (including 21 CFR Part 11) 4
R Determining Sample Sizes for Testing (Chemistry or Microbiology) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
B Selecting a Sample and Determining the Reference Value for a Bias Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
S Determining the requirements for the products and services ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
Q Determining Adverse Effects of Corrective/Preventive Actions ISO 13485:2016 - Medical Device Quality Management Systems 2
M Determining if an Insulin Pen Testing Machine is a Medical Device? EU Medical Device Regulations 4
M Determining a tolerance value for Measuring devices in-house inspection General Measurement Device and Calibration Topics 12
D Determining the the maximum number of reprocessing cycles of attachments CE Marking (Conformité Européene) / CB Scheme 2
R Determining Uncertainty from Gage R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
A IATF 16949 4.3.1 - Determining the scope of the quality management system - supplemental IATF 16949 - Automotive Quality Systems Standard 9
D ISO 9001:2015 4.3 Determining the Scope of the QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
J Determining SPC tolerance Statistical Analysis Tools, Techniques and SPC 21
B Determining SAT Offsets vs TUS Offsets per SAE AMS 2750E AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 0
R Question on determining defective units - I am not recording fixture to part rejected Statistical Analysis Tools, Techniques and SPC 5
S Clause 8.2.2 Determining the requirements for products and services ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Determining Calibration Frequency schedule for items used in production Manufacturing and Related Processes 2
C Determining if Maintenance Contractor is an External Service subject to ISO 9001 Clause 8.4 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 43
S AS9100D PEAR - Examples for organization's method for determining process results? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
I Determining Calibration Tolerance of a Measurement Device General Measurement Device and Calibration Topics 2
J ISO 17025 Documented Procedure for 6.2.5 - Determining competency ISO 17025 related Discussions 4
V Determining FDA 820 (registration) vs ISO 13485 - Supplier gives us the kit ISO 13485:2016 - Medical Device Quality Management Systems 1

Similar threads

Top Bottom