SBS - The Best Value in QMS software

Design Verification Sample Size vs Repeats

DanMann

Starting to get Involved
#1
I've been trying to determine sample sizes vs repeats for a product, but I don't seem to be able to find an answer (and in fact, think I might not know the correct terminology).
An example is that we make a printer and we want to confirm that the printer can print in under 10 seconds. We can test multiple printers (X) and make each printer print multiple times (Y); we obviously want to test as few printers as possible. How do I determine X vs Y and what statistical test should we use?
Sorry if this is a simple question, but I have been searching for about three hours now.
 
Elsmar Forum Sponsor

Miner

Forum Moderator
Staff member
Admin
#2
This is similar to a measurement systems analysis. You are talking about with-printer capability (repeatability) vs. between printer capability (reproducibility). Which is more important from your stakeholders' perspective? Or is it a combination of both?

How do you plan to assess whether either are acceptable (i.e., 1-sample t-test vs. requirement, capability study)?
 

DanMann

Starting to get Involved
#3
It's a combination of both, we want to be confident both that the printer performance will be consistently within the specification and that each printer will be within the specification. If it were a single printer, we would use a 1-sample t-test, but we're not sure what to do when testing across multiple devices. Previously, we have use 1-sample t and spread the total number of prints across the printers available, but in that case, we could just print all of them on 1 printer and this definitely isn't right.
 

Miner

Forum Moderator
Staff member
Admin
#4
A few more questions to better understand your requirements. The 1-sample t-test checks to see whether the mean is different from the requirements, but does not show whether 100% of the results meet specification. This is acceptable for certain types of requirements, such as the average net weight of food products must meet or exceed the advertised weight. However, other requirements expect 100% of the results to meet requirements. Are you just trying to show the average performance of your printers meet or exceed the advertised performance?
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#5
This situation is rarely covered by anything published. I have spent a lot of time putting together the few resources that exist and developing guidelines on how my organization does this. (Someday I'll write a book)

One other question: what constitutes a "print" that must complete in 10 seconds?

the sample size required will depend on:
1) the standard deviation of print times
2) the standard deviation between printers - is there a systemic difference in the average or standard deviation of print times?
 

DanMann

Starting to get Involved
#6
A few more questions to better understand your requirements. The 1-sample t-test checks to see whether the mean is different from the requirements, but does not show whether 100% of the results meet specification. This is acceptable for certain types of requirements, such as the average net weight of food products must meet or exceed the advertised weight. However, other requirements expect 100% of the results to meet requirements. Are you just trying to show the average performance of your printers meet or exceed the advertised performance?
Hi Miner, In this case, it's just the average is within the specification, but I think there will be items where the other scenario is true; do you know what to do in either scenario?
 

DanMann

Starting to get Involved
#7
This situation is rarely covered by anything published. I have spent a lot of time putting together the few resources that exist and developing guidelines on how my organization does this. (Someday I'll write a book)

One other question: what constitutes a "print" that must complete in 10 seconds?

the sample size required will depend on:
1) the standard deviation of print times
2) the standard deviation between printers - is there a systemic difference in the average or standard deviation of print times?
Hi Bev, Tell me when you release your book so I can get a copy! A print is timed from pressing go to the label being released from the printer (we plan on setting up a light gate that the label will break). We can determine a standard deviation for print times by doing some initial testing and we only expect there to be common cause variation between print speeds.
 

John Predmore

Trusted Information Resource
#8
Dr Taguchi advocated the idea of inner array of design factors and outer array of noise factors. He also recommended, when you have identified several noise factors, it may be unnecessary to test under every combination of noise factors. If you have prior knowledge, for example, that high temperature and low humidity and thin paper are worse for your product performance response, then create one scenario with all noise factors at the worse level and run your verification test there. Dorian Shainin employed a similar strategy, where he called the worse level of environmental factors WoW (which represented the Worst of the Worst) and Bob (Best of the Best). Since your design verification is testing against a unilateral specification (must run faster than....) it may suffice to only run your verification experiment under the Worse scenario of noise factors. Mr. Shainin used a variant of Tukey's Endcount Test to test for significance, when the focus is on one end of the distribution rather than the mean. If you can show the worst case hardware combination fulfills the 10 second requirement under the worst case environmental conditions, this approach greatly reduces the number of units you have to test to verify your design.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#9
Correction: The BOB and WOW levels were for all factors not just ‘environmental’ or ‘noise’. The original use was for post hoc testing - pick the worst parts and the best parts as measured by the critical characteristic or the Y. But the concept of running under worst case conditions is a universal fact of physics: if you know which factors and conditions will produce the longest print times when set at their Max allowed limits‘ then you can see how much true margin you have to the specification(s). It is much more powerful and uses many fewer samples or print runs in your case than random sampling.
 

Miner

Forum Moderator
Staff member
Admin
#10
Hi Miner, In this case, it's just the average is within the specification, but I think there will be items where the other scenario is true; do you know what to do in either scenario?
BevD is correct. If you still need to pursue your original plan the number of different printers is more important than the number of repeat measurements for each printer. For example, if you tested 10 printers, your degrees of freedom is 9 (i.e., 10 - 1 = 9). If you perform 1 repeat measurement per printer, your degrees of freedom is 19 (i.e., 10 x 2 = 20, 20 - 1 = 19). Therefore, focus on the number of printers and perform two measurements per printer. You can analyze the results using ANOVA and calculate the within and between printer variation as well as the combined variation. The within printer variation will be in the Residual or error term of the ANOVA table. The combined variation will be the square root of the sum of the two variances. You could then compare the confidence intervals to the requirement.

If you require all units to meet the requirement, this will require a true capability study preceded by an evaluation of the process stability using a control chart. The recommended sample size for a capability study ranges from 50 - 100 printers depending on your information source.
 
Thread starter Similar threads Forum Replies Date
C Stress / Challenge Conditions for Design Verification Testing to Reduce Sample Size 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 11
N Design Verification & Process Validation - Statistical sample sizes Design and Development of Products and Processes 2
R Medical Device Design verification sample prototype Other Medical Device and Orthopedic Related Topics 14
E Sample size for design verification of variable in single use device Design and Development of Products and Processes 20
M Design Verification: "Difference" in Sample Size Calculations? Design and Development of Products and Processes 1
G Sample Size for Design Verification and Validation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
A Sample Size for Design Verification (Prototype Testing) for Medical Devices Qualification and Validation (including 21 CFR Part 11) 12
E Determining Sample Size for Design Verification and Design Validation Other Medical Device and Orthopedic Related Topics 20
M Sample Size in Design Verification & Process Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
T Sample Sizes for Design Verification of Medical Devices Inspection, Prints (Drawings), Testing, Sampling and Related Topics 16
S Which sample to use for PV (Production Validation) or DV (Design Verification) tests Manufacturing and Related Processes 4
Q Differences between Sample Sixes for Design Verification vs. Process Validation ISO 13485:2016 - Medical Device Quality Management Systems 2
M Design Verification Sample Size - What philosophies are out there Design and Development of Products and Processes 5
N Example for design and development planning,input,output,review,verification,validation and transfer Misc. Quality Assurance and Business Systems Related Topics 4
D ISO 13485 - 7.3.6 Design and development verification - Do most folks create a separate SOP? ISO 13485:2016 - Medical Device Quality Management Systems 6
S Documenting Design Verification Test Results (ISO 9001) Design and Development of Products and Processes 1
C Documentation for items used for Design Verification 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
P Design verification driven by new equipment. How is this different than process validation? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
D Design Verification - Is testing required? Design and Development of Products and Processes 5
R Design Verification Documentation ISO 13485:2016 - Medical Device Quality Management Systems 19
R Design verification for interim design outputs - sampling rationale ISO 13485:2016 - Medical Device Quality Management Systems 2
D Testing to failure for design verification Reliability Analysis - Predictions, Testing and Standards 11
D Supplied item design verification Supplier Quality Assurance and other Supplier Issues 5
Ronen E A Rational Basis for Design Verification Design and Development of Products and Processes 5
V Which batches should or could be considered for design validation and design verification? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
S Source of QMS templates including templates for design verification/design validation Document Control Systems, Procedures, Forms and Templates 2
C Deviation in Design verification Other Medical Device Related Standards 6
V Using K-Factor(Tolerance Interval) Analysis for Design Verification Statistical Analysis Tools, Techniques and SPC 3
J Design Verification - Managing Changes Design and Development of Products and Processes 4
S CE Mark Approval Design Verification and Validation Questions CE Marking (Conformité Européene) / CB Scheme 8
V Template / format for Device History File & Design Verification of transdermal patch 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
H Is design verification and validation required for samples accompanying the quotes ? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Pre-Design Verification - Using Continuous / Variables Data Statistical Analysis Tools, Techniques and SPC 2
W Qualification of Design Verification Equipment 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
P ISO9001 Scope Exclusion Verification in a Design Company ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 14
S Verification and Validation of Post Market Design Change Design and Development of Products and Processes 2
H Form that will satisfy Design Review and Design Verification Requirements (7.3) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
J Design Inputs and Verification Activities CE Marking (Conformité Européene) / CB Scheme 5
R Need help on defining scope for Design Verification File for Class III IVD 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 8
smryan Satisfying the ISO 9001:2008 7.3.5 Design & Development Verification requirement ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
P How to separate Product Design Review, Verification, and Validation Activities ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 26
Uriel Alejandro Trouble with a Mechanical Design Technical Verification Check List AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
M Design and Development Outputs Approval, Verification and Validation IATF 16949 - Automotive Quality Systems Standard 4
A ?820.30(d) Design Output Release - Before or After Verification? Design and Development of Products and Processes 6
T DVP&R (Design Verification Plan and Report) for Vehicle Labels? APQP and PPAP 3
W Design History Verification Records Control and Approvals US Food and Drug Administration (FDA) 3
G Legacy Product - Do we need follow QSR 820? Design Verification and Validation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 12
S Design and Development - Review vs. Verification vs. Validation Design and Development of Products and Processes 15
S Acceptance Criteria at Design Verification Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
Chennaiite Is Design Verification Software required to be Calibrated IATF 16949 - Automotive Quality Systems Standard 7

Similar threads

Top Bottom