FDA 21 CFR 820.250 - Does "valid statistical" always mean math?

Mark Meer

Trusted Information Resource
#1
21 CFR 820.250 (Statistical Techniques) uses the term "valid statistical rationale".

I'm wondering if this necessarily means being able to back sampling plans with statistical formulae? ...or can the "statistical rationale" be indirect, say, from published research?

For example, Faulkner (Behavior Research Methods, Instruments, & Computers 2003, 35 (3), 379-383), conducts a study examining the correlation between number of subjects in a usability study, and the number of problems identified.

Can citing a study like this be an acceptable basis (according to FDA) for developing a human-factors study sampling plan? The FDA themselves cite it in their "Draft Guidance for Industry and Food and Drug Administration Staff - Applying Human Factors and Usability Engineering to Optimize Medical Device Design".

However, my concern is that, if scrutinized by the FDA, such a rationale would be difficult to argue, given that it is based on analysis of data gathered from a single study, and the author's conclusions. ...and not on established statistical methods...
 
Elsmar Forum Sponsor

Statistical Steven

Statistician
Staff member
Super Moderator
#2
Not only does valid statistical rationale NOT mean formulas, in many cases its a rationale why statistics were not applied. For example, your human factors study sampling plan might reference HF75 Appendix A. When a standard exists that references a sample size, that is acceptable rationale for the sample size selection. You might also choose to have a sample size of 3 based on very low variability (justification is just a standard deviation of historical data). What they don't want to see is you do not understand WHY you chose the sample size. for example, we used Z1.4 normal inspection level II without any justification why that is the appropriate sampling plan for the problem at hand.



21 CFR 820.250 (Statistical Techniques) uses the term "valid statistical rationale".

I'm wondering if this necessarily means being able to back sampling plans with statistical formulae? ...or can the "statistical rationale" be indirect, say, from published research?

For example, Faulkner (Behavior Research Methods, Instruments, & Computers 2003, 35 (3), 379-383), conducts a study examining the correlation between number of subjects in a usability study, and the number of problems identified.

Can citing a study like this be an acceptable basis (according to FDA) for developing a human-factors study sampling plan? The FDA themselves cite it in their "Draft Guidance for Industry and Food and Drug Administration Staff - Applying Human Factors and Usability Engineering to Optimize Medical Device Design".

However, my concern is that, if scrutinized by the FDA, such a rationale would be difficult to argue, given that it is based on analysis of data gathered from a single study, and the author's conclusions. ...and not on established statistical methods...
 

Mark Meer

Trusted Information Resource
#3
...your human factors study sampling plan might reference HF75 Appendix A
What is "HF75"?

When a standard exists that references a sample size, that is acceptable rationale for the sample size selection.
I'm not sure what suffices as a "standard" in this case? For example, is an FDA publication a "standard"? Or a research paper (such as the one cited)?
 

Statistical Steven

Statistician
Staff member
Super Moderator
#5
What is "HF75"?

Ronen answered this for you.

I'm not sure what suffices as a "standard" in this case? For example, is an FDA publication a "standard"? Or a research paper (such as the one cited)?
Any FDA publication including guidance documents are considered a standard. Other standards would be AAMI, ASTM, ISO, BS, IEC, etc. A research paper is not a standard, but can be referenced as justification for your decision.
 

Mark Meer

Trusted Information Resource
#6
Any FDA publication including guidance documents are considered a standard.
Too bad Applying Human Factors and Usability Engineering to Optimize Medical Device Design is only a draft guidance.

It's a pretty straight forward a prescription:
"For devices intended to be used by more than one group of users that have distinct abilities or use roles, at least 15 participants from each group should participate in validation testing."

What I think warrants discussion is how the FDA justifies their own sampling plans such as this. ...apparently from a single study which made it's conclusions based on data from 60 subjects?

I wonder how likely this would have been accepted as sufficient sample-size justification to the FDA before it was referenced in their draft guidance? ...I suspect it wouldn't have flown...

...but anyone else successfully justifying sampling-plans based on cited research, please weigh-in!
 

Ronen E

Problem Solver
Staff member
Moderator
#7
Based on several past discussions in the Cove, FDA's practice is to treat their draft guidance as final guidance from the date of publication and until it is officially withdrawn or superseded by a final guidance or a more updated draft. Of course relying on this is a bit risky because FDA can always point that it is "just a draft" - at their convenience, but I think they only resort to that in extreme cases. Mind you, some FDA draft guidance has been such for years (no further versions issued nor the guidance withdrawn), which didn't disturb anyone, including the FDA, using it as an authoritative tool (as much as guidance is considered authoritative).

What I think warrants discussion is how the FDA justifies their own sampling plans such as this. ...apparently from a single study which made it's conclusions based on data from 60 subjects?
One important thing to keep in mind regarding sample size decisions is that cost usually plays an important role. In most commercial contexts it is a matter of balancing statistical rigor and cost. AFAIK this approach is at the core of Z1.4. I assume that the FDA closed-in on the figure you quoted based on a combination of pure statistical considerations / research, and economical ones. Even FDA understands that usability trials are expensive, and they have this motto of aiming to avoid over-burdening manufacturers and hindering innovation/progress.

Cheers,
Ronen.
 
Thread starter Similar threads Forum Replies Date
D ISO 13485, FDA 21 CFR 820 and Auditing the Accounting Department ISO 13485:2016 - Medical Device Quality Management Systems 5
Ed Panek 21 CFR Part 820 - FDA Label Requirements 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 8
R How to improve a Validation program and procedures to FDA (21 CFR part 820) & ISO13485 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
Marc Problem with 21 CFR Part 820 - US FDA Quality System Regulations (QSR) sub-forum link - 2 May 2019 Elsmar Cove Forum Suggestions, Complaints, Problems and Bug Reports 1
N Change Control - Compliance with FDA 21 CFR Part 820 Document Control Systems, Procedures, Forms and Templates 3
N Certificate of Compliance from a Sister Company for FDA 21 CFR 820 Exemption 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
S Does FDA expect suppliers to be independently assessed to 21 CFR Part 820 ? US Food and Drug Administration (FDA) 4
Y ISO 13485:2003 &FDA QSR (21 CFR 820) Quality Manual,34 Procedures and Form Book, Video, Blog and Web Site Reviews and Recommendations 8
R Policy Statement for FDA CFR 820 Acceptance Activities of Incoming Products. Other US Medical Device Regulations 1
T Can I approve my own Documentation per FDA QSR 21 CFR Part 820? ISO 13485:2016 - Medical Device Quality Management Systems 1
R What body conducts 21 CFR 820 FDA certification? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 9
A Does any one have FDA 21 CFR Part 820 QSR training material? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 13
A Question about FDA 21 CFR Part 820.90 Nonconforming Product 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
B FDA 21 CFR Part 820 Medical Device Audit Criteria US Food and Drug Administration (FDA) 5
G FDA requirement that 'sister divisions' to be treated as suppliers in 21 CFR 820 ISO 13485:2016 - Medical Device Quality Management Systems 2
Q Books that have a combination of ISO 13485 and FDA Title 21 CFR Part 820 ISO 13485:2016 - Medical Device Quality Management Systems 9
C Chinese Version of FDA Code 21 CFR Part 820 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
Q FDA 21 CFR part 820.22 - Reaudit schedule necessary? Internal Auditing 6
I ISO 13485 and FDA 21 CFR 820 Regulation - Writing my QA Manual ISO 13485:2016 - Medical Device Quality Management Systems 2
D FDA 21 CFR 820 - Manual we might use as a template or where to start 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 14
S FDA 21 CFR 820 QSR Quality Manual example wanted 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
J Contract Manufacturer must adhere to 21 CFR 820 - Can FDA require us to do this? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 35
S Changes or Updates in FDA 21 CFR 820 content? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
N Upgrading ISO 13485:2003 QMS to FDA 21 CFR 820 Compliance ISO 13485:2016 - Medical Device Quality Management Systems 1
H Data Capture under FDA QSR21 CFR 820.70 G ISO 13485:2016 - Medical Device Quality Management Systems 2
D FDA 21 CFR 820.90 - Nonconformance and Device History, and Rework Records 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
S Documents / Following Procedures - FDA 21 CFR Part 820 Quality System Regulation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 12
C Definition Finished Medical Device - Definition of - FDA 21 CFR Part 820.1 Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 13
J ISO 13485 vs. FDA QSR (21 CFR Part 820) - Differences - Includes Spreadsheets ISO 13485:2016 - Medical Device Quality Management Systems 64
A What does the FDA think about 13485? Will the FDA upgrade 21 CFR 820? ISO 13485:2016 - Medical Device Quality Management Systems 3
G FDA QSRs and ISO 9001:2000 - CFR 21 Part 820 and CFR 21 Part 58 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
J Mislabeling - Consider this an FDA notified recall? CFR 806.10 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
L UDI Requirments Gap Analysis - EU MDR vs FDA CFR EU Medical Device Regulations 8
QIE FDA 21 CFR Part 11 "Meaning of Signature" Other US Medical Device Regulations 6
M Suggestions for Electronic Signature Software (FDA 21 CFR Part 11 Compliant) Qualification and Validation (including 21 CFR Part 11) 12
F Is Training Material Considered Labeling - FDA 21 CFR 801 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
B Has FDA amended any standards in the 21 CFR Part 800 series? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
B Revision history for FDA's CFR 21 US Food and Drug Administration (FDA) 1
O Comply with 21 CFR 11, but no other FDA regulations? Qualification and Validation (including 21 CFR Part 11) 4
D Compliance to 21 CFR Part 11 Require Registration to FDA? Qualification and Validation (including 21 CFR Part 11) 9
A FDA Labeling Regulations - 21 CFR part 801.109(c) vs (d): Prescription Labeling Other US Medical Device Regulations 5
S FDA Device Listing Question - "Each Device" Definition - 21 CFR Part 807 Other US Medical Device Regulations 4
E Post Inspection Regulatory Action by US FDA - 21 CFR 803 - MDR Reporting Other US Medical Device Regulations 4
D FDA 21 CFR Part 11 Compliant Digital (electronic) Signatures Document Control Systems, Procedures, Forms and Templates 7
L FDA GMP requirement: Avoid Non contact with floor? 21 CFR Part 110 US Food and Drug Administration (FDA) 1
T Applying for FDA 510(k) for 21 CFR 892.2050 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
P Any one know about FDA's GMP, 21 CFR Part 111 Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 14
D Affordable electronic CAPA system (compliant to FDA 21 CFR Part 11) ISO 13485:2016 - Medical Device Quality Management Systems 1
S FDA CFR Part 56,58,812,809 checklist General Auditing Discussions 0
T Internal Auditing - ISO 9001 auditing vs. FDA CFR auditing - Which to choose? ISO 13485:2016 - Medical Device Quality Management Systems 5

Similar threads

Top Bottom