Human Factors and Usability Testing in the US - smaller sample size than the required 15 users

[Talia]

Hi,
We are planning a usability testing with a device intended for an orphan designation.
The FDA guidance on human factors testing specifies a minimum of 15 users for each user group, however it will be very difficult to find such a number of adequate users (physicians) in the US and overall...
Does anyone have experience with justifying a smaller sample size of users?

Many thanks!

 

[EmiliaBedelia]

Hi,
We are planning a usability testing with a device intended for an orphan designation.
The FDA guidance on human factors testing specifies a minimum of 15 users for each user group, however it will be very difficult to find such a number of adequate users (physicians) in the US and overall...
Does anyone have experience with justifying a smaller sample size of users?

Many thanks!

To directly answer your question: I don't have any experience with SUCCESSFULLY justifying a smaller sample. We have attempted in the past to justify a different number of participants but were not successful - FDA's feedback was basically, we expect 15 users per user group, no flexibility.

Who is your intended use population? Are you only recruiting experienced users for the study? Is there a possibility of opening up your participant recruitment (eg, can you include both experienced and naive users)? If you truly cannot include more participants and the study cannot be performed with non-experienced users, you may have a better justification for why less than 15 users is sufficient.
If you want to get FDA's buy-in prior to the study, you could submit the protocol as a presub and ask for their feedback. Human factors studies are extremely expensive so this may be prudent if you have doubts.

 

[Talia]

Thank you Emilia for your useful response!
The intended use population are brain surgeons, experienced with such type of procedure (only a few hundreds are performed annually in the US). Therefore we will need to recruit only experienced ones. We were hoping to have this as a justification in the pre-sub...

 

[EmiliaBedelia]

It sounds like you do have a bona fide restriction on the type of user you need to recruit. If you're already planning to do a pre-sub then I would go for it and put in the justification and see what the response is.
The worst case scenario in a pre-sub is that they disagree and tell you you need 15 users. In that scenario you can ask for a conference call to discuss, and perhaps get some more information.

 

[Ed Panek]

Its possible

1) Similar product MAUDE review
2) Risk Review - its a non critical step
3) Complaints on the current device
4) Other mitigation

However, this is your product. You should know if users can utilize it correctly. I would want to know that even without the FDA>

 

[Talia]

It sounds like you do have a bona fide restriction on the type of user you need to recruit. If you're already planning to do a pre-sub then I would go for it and put in the justification and see what the response is.
The worst case scenario in a pre-sub is that they disagree and tell you you need 15 users. In that scenario you can ask for a conference call to discuss, and perhaps get some more information.

Thanks. Will do

 

[Talia]

Its possible

1) Similar product MAUDE review
2) Risk Review - its a non critical step
3) Complaints on the current device
4) Other mitigation

However, this is your product. You should know if users can utilize it correctly. I would want to know that even without the FDA>

Thanks Ed We did some acute performance studies on animals with different users. From a review of PMAs of similar devices (stents) it looks like no human factors studies were conducted at all... it might be OOS for us, we will verify in the pre-sub

 

[Solyneu]

Hello,

I'm bumping this topic because a related question came to me recently.

It might seem naive but, should a 510(k) exempt device be compliant with the guidance?
Don't worry, we applied the usability engineering process and will perform a summative evaluation, as we're going to market our device in EU. However we wonder if the 15 participants/group for the SE is required for our FDA registration based on our 510(k) exemption?

Thanks

 

[Jen Kirley]

Solyneu, that is a very good question. I regret I just don't know. By giving this a bump, I am hoping we can catch the attention of someone who can give you some direction.