Determining Sample Size in Design V&V activities

Statistical Steven

Statistician
Staff member
Super Moderator
#41
Steven,

I hate it when standards use terms like "when used" or "where appropriate" ambiguously, as they do (IMO) in this 21 CFR 820.250. Maybe they mean what you suggest (non-statistical sampling is okay if you say it is). Maybe they mean if you don't 100% test, then you are sampling, and then you need supporting rationale.

It is much more clear if they use language like this: "Seller shall perform 100% inspection, or, if sampling inspection is used, the sampling shall conform to requirements of blah blah blah..."
The history is even more confusing...it was intended for final inspection and in-process testing, but has been expanded for ALL sampling issues. It needs to be cleaned up, I agree.
 
Elsmar Forum Sponsor
#42
As always, thanks Bev! You are amazingly helpful for those statistically-challenged such as myself! :)

For the benefit of other Covers, and to be sure I've got this right, I'd like to maybe go step-by-step through a concrete, hypothetical, example.

Suppose specification is 20 +/- 2

Step 1: Estimate Variance (standard deviation, SD)
We could just "guesstimate" the variance. We are confident the process produces consistent outputs so the variance should be relatively small (SD<1).
Or, alternatively, we make a rough estimate by taking STDEV from, say, the first 10 units (aside: would this need to be justified?)
STDEV(20,20,21,20,19,18,20,21,20,19) = 0.92
So, for this example, SD = 0.92

Step 2: What is the allowable tolerance (delta, d)
This is taken from the specification (20 +/- 2). So the tolerance is: d = 2

Step 3: What is the sought confidence interval (t = 1 - alpha risk)?
We will choose 95%. Looking up on the t-table, this is a value of: t = 1.96

Step 4: Calculate the sample size
We use the formula for continuous data: n = [t*SD / d]^2
= [(1.96)*(0.92) / 2]^2 = 0.81
Rounding to the nearest integer, gives us n = 1

Hence, according to the calculations above, only 1 sample is required in this example.

Do I have this correct? :cfingers
Thanks for this. Have a few questions. Where does this formula come from (so i can reference it)? Is this specifically for normally distributed data? Also, what's the formula for non-continuous attribute data samples?
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#43
The formula can be found in any decent book on statistics.

it works for any population distributional shape since it is based on the Central Limit theorem that states that the sample averages are roughly normally distributed. it will NOT work (alone) for a non-homogenous process.

see this article: The Secret Foundation of Statistical Inference by Donald Wheeler.
 
#44
Great thanks. The link is broken but I googled it.

I'm trying to define sample size selection in a design verification SOP. Like others in this thread, my company makes few (expensive) pieces of capital equipment. But since we also make some disposables, the SOP should be general. I'm trying to make the case that statistically significant sample size is only necessary to verify some requirements.

Most of the requirements are highly deterministic, such as "System shall have 4 wheels." Its easy to justify that a statistically significant number of samples isn't required in this case.

Other requirements have more variability e.g. "Display shall swivel 90° +/- 5° ." The requirement is important for usability, but not exactly critical. How can i justify not testing a statistically significant number of systems to verify such requirements?
 

Ronen E

Problem Solver
Staff member
Moderator
#45
In my opinion this is not a purely statistical topic. Especially when discussing design verification (where one is required to demonstrate rather than prove), it is more an engineering/D&D methodology topic.

Most of the requirements are highly deterministic, such as "System shall have 4 wheels." Its easy to justify that a statistically significant number of samples isn't required in this case.
Actually in design verification this kind of requirement doesn't require any kind of sampling and even any physical specimens. Remember: the purpose of design verification is to demonstrate that the design output meets the design input. A review of the technical drawings, the specifications and the assembly WI is likely to suffice.

Other requirements have more variability e.g. "Display shall swivel 90° +/- 5° ." The requirement is important for usability, but not exactly critical.
The fact that the requirement has a tolerance doesn't automatically call for testing. This may be verified through engineering analysis, taking into account the variability allowed in specifications (components tolerances etc.). Remember, verification is not about covering all real-life's worst-case scenarios. It is about demonstrating that the design can in essence (or if you will, is likely to) meet the specified requirements; not that it necessarily always (or near always) will. This is only required in design validation, but design validation takes place on a different, higher level: user needs and intended use. We have to prove that the real production line units, under real-life use conditions, will always (or near always, i.e. at a certain high probability) perform as intended and meet user needs. To come back to your example, you'd only need to prove (a somewhat funny word when discussing soft concepts such as usability) that usability is acceptable under all normal use conditions. i.e. that no unacceptable safety use errors occur. And that may be possible even if the specification of 90° +/-5° has not been met...

Criticality of requirements is a separate topic that comes into play once you have the verification results and need to make decisions. Where the requirement is critical you might have to alter the design to meet it, while otherwise you might choose to change the design input instead.
 
Thread starter Similar threads Forum Replies Date
B Determining sample size for device sterility Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
D Determining of sample size for 'Operational Qualification' AQL - Acceptable Quality Level 5
R Determining Sample Size for Medical Device Component Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
S Surveillance Sampling Test - Determining Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
S Determining sample size for inspection to achieve x% confidence re defects Misc. Quality Assurance and Business Systems Related Topics 10
S Determining Sample Size - AQL & LQ AQL - Acceptable Quality Level 10
B Bayes Success-Run on Attribute Data - Determining Sample Size Statistical Analysis Tools, Techniques and SPC 2
G Determining Sample Size for Equipment Qualification Studies Qualification and Validation (including 21 CFR Part 11) 10
D Determining Sample Size for Capability in Six Sigma Project Six Sigma 5
H Determining the Correct Sample Size when AQL points to two Sample Sizes AQL - Acceptable Quality Level 7
L Determining Cpk based on a given Sample Size, Confidence Level, & Reliability Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
M Determining Sample Size needed for Operational Qualification (OQ) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 24
T Determining the appropriate Sample Size using an AQL Table Inspection, Prints (Drawings), Testing, Sampling and Related Topics 29
J Functional Test Sampling - Determining Sample Size to eliminate 100% Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
W Sampling Plan problem - Determining Sample Size for both Attribute and Variables Data ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
S Determining the Total Sample Size using AQL - AQL Lot Quantity question AQL - Acceptable Quality Level 8
1 Determining AQL and Sample Plan - Batch size is from 1-500 AQL - Acceptable Quality Level 24
E Determining Sample Size for Design Verification and Design Validation Other Medical Device and Orthopedic Related Topics 20
N Auditing Proof of Delivery signed by the customer - Determining Sample Size General Auditing Discussions 6
P Determining Sample Size - Filling a bag with three different types of candy Inspection, Prints (Drawings), Testing, Sampling and Related Topics 31
Q Determining Sample Size for FDA Verification and Validation Activities Inspection, Prints (Drawings), Testing, Sampling and Related Topics 41
D Determining sample size required based on a 95% confidence level Reliability Analysis - Predictions, Testing and Standards 7
B Determining sample size for drop test and waterproof testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
T Determining Receiving Inspection Sample Size Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
N Determining Sample Size for Attribute Data Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
C Internal Audit Sampling Plans - Determining Internal Audit Sample Size Internal Auditing 45
Q Determining the Appropriate Sample Size for Process Validation/Qualification Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
T Using Cpk for the purpose of determining sample size/frequencies Capability, Accuracy and Stability - Processes, Machines, etc. 2
G Determining Sample Size and Frequency for Features on Control Plans FMEA and Control Plans 8
S Determining sample sizes for PQ Qualification and Validation (including 21 CFR Part 11) 4
R Determining Sample Sizes for Testing (Chemistry or Microbiology) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
B Selecting a Sample and Determining the Reference Value for a Bias Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
S Determining the requirements for the products and services ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
Q Determining Adverse Effects of Corrective/Preventive Actions ISO 13485:2016 - Medical Device Quality Management Systems 2
M Determining if an Insulin Pen Testing Machine is a Medical Device? EU Medical Device Regulations 4
M Determining a tolerance value for Measuring devices in-house inspection General Measurement Device and Calibration Topics 12
D Determining the the maximum number of reprocessing cycles of attachments CE Marking (Conformité Européene) / CB Scheme 2
R Determining Uncertainty from Gage R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
A IATF 16949 4.3.1 - Determining the scope of the quality management system - supplemental IATF 16949 - Automotive Quality Systems Standard 9
D ISO 9001:2015 4.3 Determining the Scope of the QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
J Determining SPC tolerance Statistical Analysis Tools, Techniques and SPC 21
B Determining SAT Offsets vs TUS Offsets per SAE AMS 2750E AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 0
R Question on determining defective units - I am not recording fixture to part rejected Statistical Analysis Tools, Techniques and SPC 5
S Clause 8.2.2 Determining the requirements for products and services ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Determining Calibration Frequency schedule for items used in production Manufacturing and Related Processes 2
C Determining if Maintenance Contractor is an External Service subject to ISO 9001 Clause 8.4 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 43
S AS9100D PEAR - Examples for organization's method for determining process results? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
I Determining Calibration Tolerance of a Measurement Device General Measurement Device and Calibration Topics 2
J ISO 17025 Documented Procedure for 6.2.5 - Determining competency ISO 17025 related Discussions 4
V Determining FDA 820 (registration) vs ISO 13485 - Supplier gives us the kit ISO 13485:2016 - Medical Device Quality Management Systems 1

Similar threads

Top Bottom