Number of Participants - Usability Validation

P

pmollem

#1
Hi all,

I have a question about the amount and selection of participants for the usability validation.

The IEC 62366 leaves this topic open it just says: "The appropriate number of subjects depends on the objectives of the research as well as the diversity of the USER population". This puts the responsibility to the manufacturer to make a pragmatic decision.

However if we look to the "Applying Human Factors and Usability Engineering to Optimize Medical Device Design" of the FDA they are much more specific. They speak about a sample size of min 15 participants (also in informal discussion with FDA members they really state this as a reason for rejection). Additionally they state in the report: "To adequately represent users in the United States population, the participants in the validation test should reside in the United States."

I wonder how the industry looks upon these rules the FDA states. For a European based company who is also on the US market this brings some headaches. Because you are forced to organize a usability test with 15 US customers which more or less takes your entire budget leaving out European customers in your test. I also find it hard to understand the rationale why all test participants must be US citizens. Does anyone have practical experience about this topic?

Greetings,
Pieter
 
Last edited by a moderator:
Elsmar Forum Sponsor
#2
Re: Number of participants usability validation

Hi Pieter and welcome to the forums.

From "Handbook of Usability Testing" by Rubin and Chisnell (recommended):
The number of participants you decide to test depends on many factors, including:

The degree of confidence in the results that you require
Available resources
Availability of suitable test participants
Duration of test sessions
Also note in 62366 the bit about the scaling of the usability effort should be determined by risk analysis.

Medical devices are so diverse that there can be no single answer to the number of subjects. Consider the different level of risk from, say, a stethoscope to an infusion pump (frequently cited in the "top ten" most problematic devices). For one you may need just a small group to validate your design, packaging and instructions. For the other you may need considerable testing, at multiple points, all the way through development.

I would have thought that you may be able to justify non-American citizens in your test group if you can show that the differences in first language and culture would be highly unlikely to affect the results. I guess as I am in the UK we might have an advantage there. Failing that - do you have an American school nearby? could you recruit subjects there? or military base?
 

Ronen E

Problem Solver
Staff member
Moderator
#3
However if we look to the "Applying Human Factors and Usability Engineering to Optimize Medical Device Design" of the FDA they are much more specific. They speak about a sample size of min 15 participants (also in informal discussion with FDA members they really state this as a reason for rejection). Additionally they state in the report: "To adequately represent users in the United States population, the participants in the validation test should reside in the United States."
AFAIK this guidance is still in draft status. In late July 2012 I got this response from the FDA HF team:

Thank you for your interest in our draft guidance document. We received a very large number of comments on the document and are working our way through them but it is a slow process. We hope to have the document finalized by the end of 2012.
***

I also find it hard to understand the rationale why all test participants must be US citizens.
Mind you, residence and citizenship are not the same thing ;)

Cheers,
Ronen.
 
P

pmollem

#4
Indeed the document is still in draft status, however I once attended a course where people had their product rejected by the fda because of an insufficient usability validation. They made a claim they have tested for use errors but only included 3 participants in their test. ( was a class III device)

Citizenship is indeed not required, problem is our intended user group consist of people with a specific dental degree so it is not easy to find them outside the US.

Thanks for the reply
 
Thread starter Similar threads Forum Replies Date
D Control Number for Class III and IV devices Canada Medical Device Regulations 0
K How to handle GTINs for different configurations of one device with one part number? Other Medical Device Related Standards 0
K How can Acceptance Number and Reject Number be larger than Sample Size on Z1.4 Table? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
D Determining the the maximum number of reprocessing cycles of attachments CE Marking (Conformité Européene) / CB Scheme 2
K Why 'FD&C act section number' and 'section number' in the title of the act are different? US Food and Drug Administration (FDA) 1
A Risk Number for each software requirement IEC 62304 - Medical Device Software Life Cycle Processes 7
K Adding the Notified Body Number CE Marking (Conformité Européene) / CB Scheme 2
C Quantifying risk in choosing the number of parts, operators and replicates in a GR&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
Jane's Like-for-like critical raw material change qualification - type of testing/ number of lots required ISO 13485:2016 - Medical Device Quality Management Systems 4
Q LOT or Serial Number Symbol not used when the information is contained in the UDI? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
Proud Liberal Power Query to combine multiple tables of unequal number of columns Statistical Analysis Tools, Techniques and SPC 1
Q How is Medical Device Number (MDL) assigned to companies? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
D How to get EUDAMED Single Registration Number (SRN) EU Medical Device Regulations 17
M Informational US FDA – Modifications to the List of Recognized Standards, Recognition List Number: 052 Medical Device and FDA Regulations and Standards News 0
M Nonconformity for missing form number on the job description document Nonconformance and Corrective Action 1
I Number of decimals in equation calculated by MiniTab Using Minitab Software 0
F 2017/745 Article 31 Single Registration Number Medical Device and FDA Regulations and Standards News 5
Z Do we still have to put UDI if we have DI and lot number seperately on our product ? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
Z Two Payment Identification Number (PIN) for the same order in DFUF website 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
Ed Panek GTIN (Global Trade Item Number) for non CE Mark Other Medical Device Related Standards 4
A The latest revision number and changes in automotive core tools IATF 16949 - Automotive Quality Systems Standard 1
G Belgium - How do I get an enterprise CBE Number Other Medical Device Regulations World-Wide 0
M Document Control - Revision Number in Document Names Document Control Systems, Procedures, Forms and Templates 4
M Informational USFDA – FDA Modernization Act of 1997: Modifications to the List of Recognized Standards, Recognition List Number: 051 Medical Device and FDA Regulations and Standards News 0
S How to number and control translated SOPs - Three languages Document Control Systems, Procedures, Forms and Templates 4
qualprod Calculate number of people in processes, considering cycle time/takt time = quantity of people Lean in Manufacturing and Service Industries 9
A Doubt on multiple inspection carrying out for same balloon number during Stage Inspection and Final Inspection? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
K Relabeling or Not Relabeling - Adding our internal Part Number barcode label 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
lilybef Number of lots required for stability testing of a Class II device in the 510(k) Qualification and Validation (including 21 CFR Part 11) 3
J DoE - Number of runs in Plackett-Burman (Minitab) Using Minitab Software 2
G Heat Number on CofC - Stainless steel tube Manufacturing and Related Processes 2
M Determining number of employees within the "Scope" of the QMS ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
qualprod What tier number fits in a company producing paper labels stuck on automotive parts? IATF 16949 - Automotive Quality Systems Standard 7
M Medical Device News TGA Consultation: Changes to a number of definitions and the scope of the medical device regulatory framework in Australia Medical Device and FDA Regulations and Standards News 0
J Question on very low NDC number with tolerable GRR ratio's Capability, Accuracy and Stability - Processes, Machines, etc. 7
S Is Lot Number required in addition to UDI serial number? Other Medical Device Regulations World-Wide 7
T Change control and configuration management - When to create a new model/part number? Other Medical Device and Orthopedic Related Topics 0
D IMDS for machining - Can my customer request a IMDS number for a part? APQP and PPAP 2
shimonv GTIN Number on the Packaging of a Serviced Medical Device EU Medical Device Regulations 0
W Is the RPN (risk priority number) in the PFMEA really a RPN without the detectability ISO 14971 - Medical Device Risk Management 4
M How to count the number of employees? Other Medical Device and Orthopedic Related Topics 4
bobdoering "nds" or Number of Discriminate Samples - the Necessary Tool to Work With "ndc"! Imported Legacy Blogs 0
H FDA product code and regulation number Other US Medical Device Regulations 3
P FDA's DRLM (Device Registration & Listing Module) Listing Number Question 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 9
Sidney Vianna High number of certificate suspensions in the IAQG OASIS database AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 14
J GLP Animal Study - Change number of treatment sites Other US Medical Device Regulations 2
S My department inherited a large number CAPAs ISO 13485:2016 - Medical Device Quality Management Systems 4
R Remote Support - Calculating Number of Employees IATF 16949 - Automotive Quality Systems Standard 2
I Number of HEPA filter in a room US Food and Drug Administration (FDA) 3
Mikey324 GR&R - Little to no part to part variation in single part number Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 14

Similar threads

Top Bottom