What is the typical industry standard for documenting Usability Risk?

[ryannh]

With the recent release of IEC 62366-1 what is the typical industry standard for documenting usability risk? Are companies starting to turn towards using the FMEA process? Other than the IEC 62366 are there any good resources for ensuring that you are capturing everything?

 

[kreid]

We use 14971 and 62366.

In my opinion 62366 is well written and therefore usable. The -1 draft also looks like a good improvement which we will adopt.

 

[Jen Kirley]

Welcome to the Cove!

The addition of risk management in IEC 62366-1 is also happening in ISO 9001:2015 and similar questions are being asked.

In both standards there is no default format for documenting risk assessment. I often suggest FMEA because there is a great deal of documentation and instruction on that format. If applying the FMEA format I would consider not limiting to three existing factors (severity, occurrence and detection) but would add more factors and change the occurrence and detection to suit your needs. I suggest including a column for regulation and make this a default for Significant status; the Severity factor, which I would keep, could also trigger Significant status.

The most comprehensive and usable examples I have seen added columns to list applicable regulation(s), one for applicable operational control procedure(s), one for monitoring method(s). An additional set of columns could be added to show the effect on risk by actions taken to mitigate through improved operational controls. It is especially important to be able to point to the engineering changes done to lower Severity. I would add a Date column showing when it was done, and/or a reference number/name of a record for the applicable project to lower the risk.

I hope this helps!

 

[ryannh]

KReid, Thanks for the response. I think I should have been more clear in my question. Do you do a usability risk analysis along with product/process risk analysis or is it built into the product/process risk analysis?

 

[Jen Kirley]

KReid, Thanks for the response. I think I should have been more clear in my question. Do you do a usability risk analysis along with product/process risk analysis or is it built into the product/process risk analysis?

If using Excel, an additional column could be used to indicate usability/process and sorted when the user wants to only view usability risk analysis.

 

[Jen Kirley]

I would also add that automotive uses a Design FMEA, in which Severity criteria are specific to usability. Other factors might also be different, as appropriate for usability.

 

[Statistical Steven]

KReid, Thanks for the response. I think I should have been more clear in my question. Do you do a usability risk analysis along with product/process risk analysis or is it built into the product/process risk analysis?

Usability analysis is separate from process and product risk. Both are usually documented in the risk file, but risk assessment based on 14971 alone is not enough. Another document to look at is ANSI/AAMI HE75:2009

 

[Marcelo]

A table such as the one in the attached document is what ISO 14971 (and IEC 62366) requires (I include the sequence or combination of events which is not formally required but makes sense in including)

If you use any hazard analysis technique such as FMEA, FTA, etc, you need to use the result of those to populate the ISO 14971 summary (as they do not have all the information required by ISO 14971).

 

[Marcelo]

Also, it?s important to note some differences between the ISO 14971 process and the risk management required by IEC 62366 (and IEC 62366-1).

The "full" risk management from ISO 14971 requires the analysis, evaluation and control of risks, and risk needs to be analyzed, as the definition, based on severity and probability.

Another important aspect is use error. Use error is a kind of "failure". Failures are not a problem in itself, they lead to a problem. Speaking in ISO 14971 terms, failures are part of the sequence of events that leads to a hazardous situation (but are never the hazard situation itself). Use errors follow the same principles (although, in the case of use errors, some may be a hazardous situation).

Going back to the differences, there?s no known method to estimate the probability of use errors. So it?s not possible to estimate the probability of the hazardous situation (P1), and thus the usability engineering process do not require that the probability of the risk (which is P1xP2) to be estimated, only the severity. Also, there?s no need to evaluate the risk. The rationale is that, as we cannot predict the probability of use error, it?s better to treat (control) all use errors that led to hazardous situation.

So, the requirement is that, for any safety-related use error, the user interface design has to include requirements related to them.

Which means, in practice, that you only perform part of the full ISO 14971 RM process for user error-related risk management.

This can be seen in the comparison table between ISO 14971 and IEC 62366 that in the annex of the standards

 

[kreid]

We have a User Risk Assessment that considers the risks that may come from using the product (we also have process and design risk assessments). We then use our usability testing/study to help validate some of the mitigations we arrived at from our risk assessments.