IEC 62366 and Medical Device Usability Validation

[zinzibars]

62366 is new to me, so I am studying it to understand how to develop a Usability Engineering File. I'm an industrial designer, and I've been tasked with implementing 62366 for a device that our company is developing.
I've read a lot of threads here and learned a lot, but without a quality or regulatory background I still have a lot to learn! I'm hoping that some of the very knowledgable people here can help me out a little.

Most of the standard seems pretty clear to me, however I am confused about Validation activities. Section 5.9 states that a Usability Validation Plan is to be developed and that it involves the use of representative users. Section 5.9 states that Validation of the usability of the medical device should be done according to the Usability Validation Plan. Okay so far... Now, Annex D @ D.4.7.1 differentiates between VERIFICATION and VALIDATION and states "...VALIDATION insures that the final production model addresses the intended USER needs". So, they are defining that VALIDATION relates to a final production design, but then back in SUBCLAUSE 5.9, they state: "VALIDATION activities should occur throughout the MEDICAL DEVICE'S design, beginning in the early stages of design conceptualization (see D.4.7)". This seems to me to be a contradiction in that validation obviously cannot be done with production units in the early stages of conceptualization.

So, my questions are:
1) What do they really mean by VALIDATION?
2) Are the human factors usability engineering methods and techniques listed in Annex D @ D.5 to be used for verification or for validation during the conceptualization period and up to, but not including production? I understand that once a production product is ready, you can validate it, but how do you do validation before it is ready?
3) Is the difference that during the development process, you are supposed to validate designs, while at the end of the process, you validate the final product? If so, I know that you can verify designs, but how do you validate them?

Well, lots of questions, but hopefully you can see where my confusion is and help this newbie a little!!! Thank you!

 

[Marcelo]

Re: 62366 and Validation

Hello, and welcome to the Cove.

There?s always confusion between verification and validation, please my comment (and other comments) on this thread: Medical Device Design Verification and Validation Differences

and the waterfall model that I attached in this thread: Design Input & Output in Stages - Overkill?

I think the waterfall model is the best to understand the differences.

You can use the methods for verification or validation depending on what you want. You can also use them post production, for example, if there?s a need for a change.

 

[zinzibars]

Re: 62366 and Validation

Marcelo, Thank you for the welcome and for your response! I read through the discussion thread about Design Verification and Validation differences and viewed your waterfall chart. While I thought that I was pretty clear about the differences between Ver. and Val., the discussion was helpful and perhaps I am still not totally clear on it though. I'm not sure if my confusion is because of the latter, or because 62366 is not written clearly.

Let's take an example. Back to my question #2: "I understand that once a production product is ready, you can validate it, but how do you do validation before it is ready?" Let's say a company is developing a hand held medical device. They produce a foam mockup to evaluate the ergonomics of the grip. They have a requirement that the device must fit comfortably within the hands of the 5th to the 95th percentile person within the USA. They would then want to verify that it meets that requirement, so I say that would be VERIFICATION. Does it meet the needs of the end user? Well, you don't have a final product to verify that yet, so I think that you can't do VALIDATION.

While this has become a discussion about VERIFICATION vs VALIDATION, it ties back to the 62366 forum because SUBCLAUSE 5.9 of the standard states: "VALIDATION activities should occur throughout the MEDICAL DEVICE'S design, beginning in the early stages of design conceptualization (see D.4.7)". I still don't see how you can VALIDATE a conceptual design. Do we have to assume that we are VALIDATING a hypothetical device with hypothetical users? Is that what we are supposed to do with all of the human factors usability engineering methods and techniques listed in Annex D @ D.5?

Thanks in advance for any insight into this!!!

 

[Marcelo]

Let's take an example. Back to my question #2: "I understand that once a production product is ready, you can validate it, but how do you do validation before it is ready?" Let's say a company is developing a hand held medical device. They produce a foam mockup to evaluate the ergonomics of the grip. They have a requirement that the device must fit comfortably within the hands of the 5th to the 95th percentile person within the USA. They would then want to verify that it meets that requirement, so I say that would be VERIFICATION. Does it meet the needs of the end user? Well, you don't have a final product to verify that yet, so I think that you can't do VALIDATION.

In this case, verification would be to analyze if the grip was designed as planned, for example, angles, rigidity, whatever characteristic. If they perform usability studies (with the population as required) to analyze if they feel it fits comfortably, it would be validation. In this case, you would be validating a part of the design (the grip), and this could be done long before finishing the whole divide user interface design.

Probably the confusion is due to the common misconception that validation is a phase. It?s not, it?s really mor e a set of several different validation activities (one each for each user need). IEC 62366 tries to make it clear that validation activities can be performed at different phases of the device design and development stages.

Also, the above example I mentioned is not hypothetical, you would use final grip specifications for verification and usability testing with real users for validation.

 

[sagai]

Probably the confusion is due to the common misconception that validation is a phase. It?s not, it?s really mor e a set of several different validation activities (one each for each user need).

Big like from me!

Finding what to do exactly for universal validation is a kind of holly grail, we can do it, we could spend years on it, however our chances to go mad is radically increasing during that time.
My advice is to sit back and relax, have a nice tea, a drink or whatever you like. And think about what would you do to make yourselves confident to blindly let your loved ones be cured by this device, that's all about validation.

It recalls me this book that I would advocate again:
David A. Vogel: Medical Device Software Verification, Validation and Compliance

Cheers!

 

[frivolas]

Ha, yeah... Relax and take a break... in this industry? Dude, challenge accepted!

But you're right, I think that as good engineers and designers we should do our best to avoid analysis paralysis. Also, let's not forget that our Usability Engineering efforts should be focused on mitigating risks and potential hazardous situations. Comfort and ergonomics can be put in a secondary plane unless a specific feature of the device compromises the safety of USE (and please correct me if I'm wrong).

In some devices and applications like combination products, and especially when the patient will be the user, the ergonomics and industrial design of the device play an important role on increasing user satisfaction and therefore, in some specific cases, adhesion to therapy or treatment. Failing here could mean a risk and it would be drug dependant or therapy specific. Something to bounce back with your risk management peeps and start a nice conversation with the drug manufacturer.

I'm also new here (here meaning both the forum and Usability Engineering), so after reading the standards and a bunch of books, websites and this awesome forum, what I'm doing is basing all my verification and validation tests on a pretty thorough risk analysis. It took weeks to be done, but after several analytical evaluations (you know, task analyisis, heuristic evals, cognitive walkthrough) we were able to build a comprehensive UFMEA that tells us every hazardous situation we were able to see or anticipate.

Now, based on that huge excel file, I was able to create a list of critical tasks, or those that have an unacceptable risk. It was pretty straight forward actually. Just pick all the red ones, and discuss with the team about the yellow ones. The green ones should be OK.

With this list of tasks now I'm writing the objectives and goals for my usability tests.

Also, take in account that not all of the Verification you have to do is Usability Verification, but some Design Verification. The tests will be different and will involve different stakeholders (like your QA team). It's kind of "easy" to know who owns which test: If you gotta measure or test a specific parameter in the lab to ensure it will be repeatable after manufacturing and assembly, that belongs to QA. If you have to know how a specific parameter will mitigate a risk or affect the use of the device, then its part of your UE tests.

Testing the spring force for a thumb activated button would be a good example. Let's assume that the button is an emergency stop therefore pressing of the button is critical for safety (all numbers are being pulled from a dark place!):

• Design Verification - The force required to press the spring is less than 10N
• Usability Verification - 5percentile Users are able to exert a force of at least 1.5 times the spring force with their thumbs in the expected hand-grip configuration
• Usability Validation - We confirm that there is the need of an emergency stop button, that it is in the right place with the right size and that the users know when to press it, what it does and especially, that they can press it. You can also test this vs the accompanying documentation.

(don't you just love the word "accompanying"?)

Again, I'm new here, so please correct me if I'm wrong!!! I'll really appreciate any advise on how to improve my process.

Cheers!

 

[Julie_]

Hi ,

I was wondering if anyone has experience with medical device usability activities, in which your none of your identified critical tasks have an unacceptable risk.

In this case, would you still need to perform Summative evaluation? I assume you still need to do it, but on which criteria do you select your scenarios in this case?

Anyone who can provide me with some advise regarding this?
Would be very much appreciated!

Julie

 

[Kuldeep Singh]

Hi everyone.

As i am new to IEC 62366 usability engineering for medical device.I read the standard and try to prepare the usability engineering test plan and test for our Class iib medical device.

Can anyone suggest me that am i going to right way or i have to make changes in this document.

Please share the templates if you have any.Thanks in advance for your feedback

 

[Tobias_HF]

Hi everyone.

As i am new to IEC 62366 usability engineering for medical device.I read the standard and try to prepare the usability engineering test plan and test for our Class iib medical device.

Can anyone suggest me that am i going to right way or i have to make changes in this document.

Please share the templates if you have any.Thanks in advance for your feedback

Hi
you can`t rely on subjective statements from users. What you want to do is, observe what your (way more than 5 please..) users are doing. You need to line out what the correct actions are, and what possible use errors are. Now in your study, you will observe only, and see of users are doing what they are supposed to do. if not, this is either a use errors, or a close call. Or an operational difficulty.
Based on your root cause analysis of each of them, then you can make a "Pass / Fail" assessment.

Reading the standard might not be sufficient, I`d recommend to include an experienced practitioner / HF professional. Especially if its a IIb device...
Bests
Tobias

 

[Kuldeep Singh]

Hi
you can`t rely on subjective statements from users. What you want to do is, observe what your (way more than 5 please..) users are doing. You need to line out what the correct actions are, and what possible use errors are. Now in your study, you will observe only, and see of users are doing what they are supposed to do. if not, this is either a use errors, or a close call. Or an operational difficulty.
Based on your root cause analysis of each of them, then you can make a "Pass / Fail" assessment.

Reading the standard might not be sufficient, I`d recommend to include an experienced practitioner / HF professional. Especially if its a IIb device...
Bests
Tobias

Thanks for your comments.

Have you any template or document for usability engineering test. which can help me to do in a right manner.