Assessing Hazard-Related Use Scenarios where control measures exist through standards

ThatSinc

Quite Involved in Discussions
Hi All,

When identifying hazards relating to the interface and documenting hazard-related use scenarios, what is the consensus when dealing with control measures for hazards where the control measures are specified in applicable standards to the device in question.

I feel I'm in a "chicken and egg" situation

As an example, several connections on the back of the device for inputs all have standards defining the dimensions to ensure that they are unique and cannot be misconnected and cause a hazardous situation.
These standards have been identified in the design requirements from review of existing regulatory/standards requirements, not through the usability engineering process.

Is the usability engineering process required to identify the use error of misconnection of inputs through any perception or cognition issues from a complete blank slate to come to the conclusion that these risk control measures are required?
Or should the usability engineering process take into consideration the device design including implementation of applicable standards?



Thanks

M.
 
Device design and risk management are interconnected into a cyclical process, so I understand your chicken-and-egg quandary. The conservative approach would be to include it. Keep in mind that usability is part of your risk management process. Risk management should be started at the very beginning of device development. At that time, the organization should consider potential risks and then use this information, along with other sources, to determine design inputs. It makes sense then to have the misconnection risk documented in your risk analysis with the control being the requirements outlined in the particular standard. You would also have the connections requirements listed as a design input.

When it comes to usability, however, the risk associated with misconnection may be very low and may have low severity. In this case, you might not need to include it in your summative evaluation.

Also keep in mind that risk management documentation should be used as a tool throughout the lifetime of the device. For example, if in the future, someone wanted to change one of the connectors because of a supply chain issue, a review of the risk management documentation should be done as part of the change control process. This review may prevent the connector from being changed.
 

ThatSinc

Quite Involved in Discussions
Thanks for the really detailed reply, that confirmed my feelings and also how I've managed risk management files myself.

I find it unfortunate, in some respects, that this is the case where design standards exist and are usually the first go-to when designing a product that is not novel. It supports the notion that so many business owners have that these things are just paperwork exercises.

It's so often the case that these pieces of documentation are left till well into the design process and/or the design documentation is left in a draft form and so you have no clear documented history showing the iterations of the design inputs and risk documents.

It's been a good few years since I led the risk management process anywhere, and I haven't integrated it with usability before.
However the link with change management is something I am familiar with, more from when companies have tried to "make a process more lean" (read: remove a lot of inspection and test) and it's been referred back to.

With that in mind, and with the significant interlink between usability and risk management, would you consider expansion of the hazard identification relating to usability/interface portion of the risk management file (essentially an expanded annex C style document, noting that Annex C is being moved to the revision of TR24971) a better solution to a usability engineering file than a "stand alone" document for use scenario assessment? Or some form of task based uFMEA that would feed into the hazard analysis?
 

ThatSinc

Quite Involved in Discussions
would you be willing to share an example/template of the uFMEA at all?

I think that might be the best way of making the right linkages.

I'm going around in circles trying to link identified POFs to Use Scenarios, to Hazard Use Scenarios and back to risk control measures and design inputs. I can see what I'm expected to do, but struggling with getting it to make sense on paper.
As previously mentioned, it's a cyclical process, but it shouldn't be driving me around in circles :vfunny:
 
Sorry, I'm not sure giving you a template would help because it would just be one piece of the picture. Also, there are many different ways to do this. My suggestion is to keep it as simple as you can while still maintaining compliance. Don't go overboard if your device is medium/low risk.

Are you familiar with the FMEA process? What are you struggling with specifically?
 

ThatSinc

Quite Involved in Discussions
My problem is keeping it simple; I have a real problem with overthinking things and paralysis by analysis - there being so many ways to do this is part of the issue too. I'm a "learn by observation" person, so reviewing examples allows me to see how the pieces fit together and I can then apply it.

I think it's a separate discussion from this particular thread but, whilst we are here, the below are some of the thoughts I have - not necessarily questions that have right or wrong answers, but trying to understand how others do things to better understand them myself.

  • should each primary operating function have multiple use scenarios, or is the use scenario an more detailed analysis of how the primary operating function is performed
    • If the primary operating function is determined as "setting alarm limits" - would you have multiple use scenarios for this, or a single scenario of how the alarm limits should be set, with the multiple potential failure modes for each step?
    • When performing the use scenario analysis within an uFMEA, should you be using the same criteria as in your 14971 risk plan, or would the identification of hazardous situations in the uFMEA feed up into the hazard analysis to allow you to further assess them there? It seems like double handling, but I'm not sure what is the most efficient approach.
  • Would you organise your user interface specification into each requirement per primary operating function?
  • Is it common to see a separate user interface specification from the design input requirements, if so how would the design input requirements reference forward? (this links back to my other thread on mapping design input requirements, thank you for your input on that)

my main struggle overall in linking usability and risk is how I feel that the definition of hazardous situation isn't consistently applied within the usability standards and risk standard and how to manage that misalignment to ensure consistent analysis - but that's an entirely different topic.

I'm booking myself onto training courses, but with the current situation it's not the best time.

Thanks.

M.
 

Tobias_HF

Involved In Discussions
should each primary operating function have multiple use scenarios, or is the use scenario an more detailed analysis of how the primary operating function is performed
  • If the primary operating function is determined as "setting alarm limits" - would you have multiple use scenarios for this, or a single scenario of how the alarm limits should be set, with the multiple potential failure modes for each step?
That...depends.
What I find helpful is to see the POF really based on the task analysis. Based on observations etc. You will find out what a task is, meaning, in my understanding, a task is something you do...and then you can have a break. Grab a coffee.

Not in the literal sense, but metaphorically speaking: you finished something, and now you can start something else. How you do this, is a different story...

Ideally, the POF and use scenario match - 1 POF has 1 use scenario, which describes the subtasks, preconditions, etc.
In reality, people do all kind of crazy sh... so you probably have multiple ways how to perform "setting alarm limits". And your device might support multiple ways, so you have to describe all of them. And of course with multiple failure modes.

For me it helps to categorize them, lets say "sunshine solution" is the easiest way, followed by whatever is realistic / you have observed (!) during your task analysis.

When performing the use scenario analysis within an uFMEA, should you be using the same criteria as in your 14971 risk plan, or would the identification of hazardous situations in the uFMEA feed up into the hazard analysis to allow you to further assess them there? It seems like double handling, but I'm not sure what is the most efficient approach.

I have done both - and again it depends....
Currently I prefer the second approach: keep the uFMEA as separated as possible, and if you identify a hazardous situation leading to a risk - than only forward it into your 14971 risk plan / risk analysis.
But for me the uFMEA is a design input tool, that helps me to a)document observed / known use errors b)helps me to think about potential use errors / failure modes, together with the team and c)shows that we are compliant. But this is the least important, to be honest...

  • Would you organise your user interface specification into each requirement per primary operating function?
  • Is it common to see a separate user interface specification from the design input requirements, if so how would the design input requirements reference forward? (this links back to my other thread on mapping design input requirements, thank you for your input on that)
Yes! Absolutely! And I really hate it.... ;-)
Traceability is a major concern, I have not yet seen a UI spec that solves this really nice.
In my dream world I have my wireframe / axure prototype, click on a button - this opens a list of UI requirements - connected to the uFMEA, and if necessary to the use errors / risk / mitigation (why this button has to be exactly there) - the formative / summative test protocols which specify the acceptance criteria, criteria for selection into the studies - then the formative / summative test report with the objective evidence - then my summary / acceptance report - aaand finally my account details to where my boss can send the bonus for this awesome work.
I do this with different excel sheets right now, for me it works, but its not a sustainable solution...

Regarding the second point:
As a first step: I would also separate the UI spec in different parts. What I have seen as very helpful is a detailed written description, why the design is how it is. I`ve seen this named as "interaction design spec", and its really helpful for auditors but also new designers, who have to pick up the design. For them its also helpful to see the history of the UI, from first scetches, maybe formative study results, etc up to the final product.
In the end there is a list of requirements, connected to POFs, and then a reference to the UI part. But without the context (the history, so to say) I find this really useless.
 

Tobias_HF

Involved In Discussions
my main struggle overall in linking usability and risk is how I feel that the definition of hazardous situation isn't consistently applied within the usability standards and risk standard and how to manage that misalignment to ensure consistent analysis - but that's an entirely different topic.
Same struggle here.... :-|
 

ThatSinc

Quite Involved in Discussions
Thank you for your reply, I think my mind is on the right path I just have difficulty in putting it down on paper in the right order.

Currently I prefer the second approach: keep the uFMEA as separated as possible, and if you identify a hazardous situation leading to a risk - than only forward it into your 14971 risk plan / risk analysis

So for your uFMEA you use different scoring criteria to your "main" risk file?
Is this criteria documented as a part of your usability engineering process?
As any identified hazard/hazardous situation from a hazard use scenario is mapped forward/upward into the 14971 risk file and assessed there for probability and severity, is there any requirement to document an occurrence rate within the uFMEA?
clause 5.5 and the guidance to that clause seem to advise against it, stating it's only the severity that can be used for assessment in the absence of robust data, but I have routinely seen it in the past with minimal data to support.


As a first step: I would also separate the UI spec in different parts.

I currently have a "User Interface" section within my design input requirements document, with connections, ergonomics, labelling, layout, in.
Some of the requirements in these sections overlap with the "functional requirements" section and "safety" section, but none contradict each other.

Same struggle here.... :-|

I think having an entirely separate uFMEA and mapping that forward will alleviate some of the struggle, with the hazardous situation being the *cause* of the exposure to the hazard in the uFMEA and the hazardous situation being the exposure to the hazard in the Hazard Analysis.

e.g.
Table b.2. from 60601-1 has the use of the glucose meter being difficult to read as the hazardous situation (which results in being administered excessive amounts of insulin as a subsequent event and harm occurring as a result)
Table c.3. from 14971 having the failure to deliver insulin as the hazardous situation, and harm occurring directly as a result of the hazardous situation.
 
Top Bottom