Final Validation Unit Requirements - Annex D of IEC 62366

Z

zinzibars

Annex D of 62366 refers to using "production units" for final validation. Since Annex D is an informative guidance section of the standard, I am thinking that it is not a requirement to use production units and would like to hear from any of you that have the relevant regulatory knowledge or practical experience.

Of course the risk is that if non-production units are used for final usability validation and any changes are made to the device after validation but prior to production that may affect the usability of the device or risk assessment, then the final usability validation should redone. However, in my thinking, there is a potentially greater cost result if final production units are used for the validation and the units do not pass the validation. Then one may likely need to enter back into R&D (Design) to resolve the problem by design/engineering, and subsequently re-validate.

Therefore I am thinking that it will be best for our company to use DVT (design verification testing) units for final usability validation. Our DVT units will have components equivalent to final production units including injection molded housings. The things that may change will be the manufacturing instructions because they will likely be scaled to be more effective for larger production quantities, but I don't see that having any effect on device usability.

Thanks for any experience and knowledge!

Ref.:
D.4.7.3 Production unit final VALIDATION
Evaluation of production units employ methods to assure that the MEDICAL DEVICE meets
USER needs and INTENDED USE (i.e. design VALIDATION).
 

Pads38

Moderator
I can understand your concern that any usability problems thrown up using a finished production unit would be expensive and time consuming to rectify. However, I think here we can consider the subtle differences between "verification" and "validation".

To quote from Annex D 4.7.3
Given a thorough VERIFICATION effort, the USER INTERFACE design is likely to be substantiated during final VALIDATION. However, subtleties in operation that were not apparent during VERIFICATION can emerge in final testing. Given a design based on a structured USABILITY ENGINEERING approach, problems uncovered during VALIDATION are usually relatively minor and the required design changes modest.

This would suggest that the best way forward is to conduct most of your testing at an earlier stage, on prototypes, mock-ups etc. This can be to verify parts of your device / parts of the interface. With the confidence built up from that you can go to your finished device with the validation being restricted to how the overall product functions.
 
Z

zinzibars

Pads, I appreciate your referencing the additional text from D.4.7.3, and noting the confidence that should be developed by earlier rigorous verification. I do agree with that.

Due to the substantial time and cost of planning, conducting, and documenting a good usability with representative users, we'd like to only do one usability study, and that would be our final validation. Now, if we use production devices for that and the testing elicits a usability problem that would entail even a relatively minor design change to fix, then a substantial chain of events may result.

Here's a real life example. We have a device that must be operated with a door closed. The user will open the door, then push a button that is revealed once the door is opened. This activates a "system ready" state. The user then closes the door. Closing the door automatically activates a timed device function and illuminates an indicator light to show that the function is active. We think that this automatic activation makes the device operation very simple for the user. We have explained it to potential users and they like the idea. Now, let's say that the device is designed in this manner, and we build production units and use them for a final validation usability study. Perhaps due to the greater number of participants, a few of them find a problem with having the function happen immediately upon closing the device. (I can't anticipate what that problem would be at this time). Based upon the validation result, we decide to implement a switch on the outside of the door in place of automatic door closure activation. Here's the impact to our company:
- Scheduled resumption of production is stopped
- Board of Directors is informed of production delay
- Risk analysis is engaged in to assess risk to design and risk to usability
- Injection molded tooling needs to be modified to place recess for new switch
- Need to spec and procure new switches
- PCB needs to be revised to accommodate switch
- Manufacturing instructions need to be revised
- A new limited build of devices needs to be made for v&v
- Final deign verification testing needs to be conducted again
- Instructions for use need to be revised
- Lots if Engineering Change Orders
- Repeat Usability Validation Testing

That's an abbreviated list! Now, if we had used devices that we built for Final Design Verification instead of production, most of those tasks would certainly still need to occur, but the first two would not happen: resuming production would not have had a firm start date, and the Board of Directors would not have had the expectation of the firm start date. Those two items are huge because jobs are at stake. Why use production devices when final Verification build devices should be substantially the same as production devices, and by using the Verification devices we will avoid the firm production rollout expectation? 62366 does not state the requirement to use production units in the main body of its text; it is only mentioned in the "Informative" annex D. My fear is that a notified body auditor will not agree, but I have no experience to justify that concern.
 

Pads38

Moderator
What you describe is the classic engineering problem that the later in a products life cycle you try and implement a change the more expensive (and damaging to time scales) it becomes. This is true whether the change is forced by usability, EMC, or any other regulatory requirement.

And if the life cycle has gone even further - delivered into the real world - you could end up with a product recall - even more expensive!

But I would suggest that you can do usability testing at an earlier stage without undue expense or delay. Using your example of the switch with access door and indicator lights - that could be prototyped easily. The parts could be duct taped into a cardboard box and connected up to a battery or two. You could verify that people genuinely find the concept intuitive with just a dozen or so of the people around the office. They probably don't need to be 'representative' users to test the mental concept of the switch operation.

How much development testing you do is a classic problem of commercial risk management as well as being part of the regulatory requirements.
 

Marcelo

Inactive Registered Visitor
Anyway, the new edition of IEC 62366 (which will be IEC 62366-1) will require at least two evaluation requirements, formative and summative (see attached draft TOC), so you won?t really be able to do it only once in the future.
 

Attachments

  • IEC 62366-1 CDV TOC.jpg
    IEC 62366-1 CDV TOC.jpg
    136.4 KB · Views: 429
F

frivolas

Anyway, the new edition of IEC 62366 (which will be IEC 62366-1) will require at least two evaluation requirements, formative and summative (see attached draft TOC), so you won?t really be able to do it only once in the future.

Hi Marcelo, do you know when this revised version of the 62366 will come out? I just got an email from the AAMI with a 62366:2007/(R)2013, could you give us some info about this (R) version?

Thanks!
 
Top Bottom