Assessing Hazard-Related Use Scenarios where control measures exist through standards

ThatSinc

Involved In Discussions
#1
Hi All,

When identifying hazards relating to the interface and documenting hazard-related use scenarios, what is the consensus when dealing with control measures for hazards where the control measures are specified in applicable standards to the device in question.

I feel I'm in a "chicken and egg" situation

As an example, several connections on the back of the device for inputs all have standards defining the dimensions to ensure that they are unique and cannot be misconnected and cause a hazardous situation.
These standards have been identified in the design requirements from review of existing regulatory/standards requirements, not through the usability engineering process.

Is the usability engineering process required to identify the use error of misconnection of inputs through any perception or cognition issues from a complete blank slate to come to the conclusion that these risk control measures are required?
Or should the usability engineering process take into consideration the device design including implementation of applicable standards?



Thanks

M.
 
Elsmar Forum Sponsor

indubioush

Quite Involved in Discussions
#2
Device design and risk management are interconnected into a cyclical process, so I understand your chicken-and-egg quandary. The conservative approach would be to include it. Keep in mind that usability is part of your risk management process. Risk management should be started at the very beginning of device development. At that time, the organization should consider potential risks and then use this information, along with other sources, to determine design inputs. It makes sense then to have the misconnection risk documented in your risk analysis with the control being the requirements outlined in the particular standard. You would also have the connections requirements listed as a design input.

When it comes to usability, however, the risk associated with misconnection may be very low and may have low severity. In this case, you might not need to include it in your summative evaluation.

Also keep in mind that risk management documentation should be used as a tool throughout the lifetime of the device. For example, if in the future, someone wanted to change one of the connectors because of a supply chain issue, a review of the risk management documentation should be done as part of the change control process. This review may prevent the connector from being changed.
 

ThatSinc

Involved In Discussions
#3
Thanks for the really detailed reply, that confirmed my feelings and also how I've managed risk management files myself.

I find it unfortunate, in some respects, that this is the case where design standards exist and are usually the first go-to when designing a product that is not novel. It supports the notion that so many business owners have that these things are just paperwork exercises.

It's so often the case that these pieces of documentation are left till well into the design process and/or the design documentation is left in a draft form and so you have no clear documented history showing the iterations of the design inputs and risk documents.

It's been a good few years since I led the risk management process anywhere, and I haven't integrated it with usability before.
However the link with change management is something I am familiar with, more from when companies have tried to "make a process more lean" (read: remove a lot of inspection and test) and it's been referred back to.

With that in mind, and with the significant interlink between usability and risk management, would you consider expansion of the hazard identification relating to usability/interface portion of the risk management file (essentially an expanded annex C style document, noting that Annex C is being moved to the revision of TR24971) a better solution to a usability engineering file than a "stand alone" document for use scenario assessment? Or some form of task based uFMEA that would feed into the hazard analysis?
 

ThatSinc

Involved In Discussions
#5
would you be willing to share an example/template of the uFMEA at all?

I think that might be the best way of making the right linkages.

I'm going around in circles trying to link identified POFs to Use Scenarios, to Hazard Use Scenarios and back to risk control measures and design inputs. I can see what I'm expected to do, but struggling with getting it to make sense on paper.
As previously mentioned, it's a cyclical process, but it shouldn't be driving me around in circles :vfunny:
 

indubioush

Quite Involved in Discussions
#6
Sorry, I'm not sure giving you a template would help because it would just be one piece of the picture. Also, there are many different ways to do this. My suggestion is to keep it as simple as you can while still maintaining compliance. Don't go overboard if your device is medium/low risk.

Are you familiar with the FMEA process? What are you struggling with specifically?
 

ThatSinc

Involved In Discussions
#7
My problem is keeping it simple; I have a real problem with overthinking things and paralysis by analysis - there being so many ways to do this is part of the issue too. I'm a "learn by observation" person, so reviewing examples allows me to see how the pieces fit together and I can then apply it.

I think it's a separate discussion from this particular thread but, whilst we are here, the below are some of the thoughts I have - not necessarily questions that have right or wrong answers, but trying to understand how others do things to better understand them myself.

  • should each primary operating function have multiple use scenarios, or is the use scenario an more detailed analysis of how the primary operating function is performed
    • If the primary operating function is determined as "setting alarm limits" - would you have multiple use scenarios for this, or a single scenario of how the alarm limits should be set, with the multiple potential failure modes for each step?
    • When performing the use scenario analysis within an uFMEA, should you be using the same criteria as in your 14971 risk plan, or would the identification of hazardous situations in the uFMEA feed up into the hazard analysis to allow you to further assess them there? It seems like double handling, but I'm not sure what is the most efficient approach.
  • Would you organise your user interface specification into each requirement per primary operating function?
  • Is it common to see a separate user interface specification from the design input requirements, if so how would the design input requirements reference forward? (this links back to my other thread on mapping design input requirements, thank you for your input on that)

my main struggle overall in linking usability and risk is how I feel that the definition of hazardous situation isn't consistently applied within the usability standards and risk standard and how to manage that misalignment to ensure consistent analysis - but that's an entirely different topic.

I'm booking myself onto training courses, but with the current situation it's not the best time.

Thanks.

M.
 

Tobias_HF

Starting to get Involved
#8
should each primary operating function have multiple use scenarios, or is the use scenario an more detailed analysis of how the primary operating function is performed
  • If the primary operating function is determined as "setting alarm limits" - would you have multiple use scenarios for this, or a single scenario of how the alarm limits should be set, with the multiple potential failure modes for each step?
That...depends.
What I find helpful is to see the POF really based on the task analysis. Based on observations etc. You will find out what a task is, meaning, in my understanding, a task is something you do...and then you can have a break. Grab a coffee.

Not in the literal sense, but metaphorically speaking: you finished something, and now you can start something else. How you do this, is a different story...

Ideally, the POF and use scenario match - 1 POF has 1 use scenario, which describes the subtasks, preconditions, etc.
In reality, people do all kind of crazy sh... so you probably have multiple ways how to perform "setting alarm limits". And your device might support multiple ways, so you have to describe all of them. And of course with multiple failure modes.

For me it helps to categorize them, lets say "sunshine solution" is the easiest way, followed by whatever is realistic / you have observed (!) during your task analysis.

When performing the use scenario analysis within an uFMEA, should you be using the same criteria as in your 14971 risk plan, or would the identification of hazardous situations in the uFMEA feed up into the hazard analysis to allow you to further assess them there? It seems like double handling, but I'm not sure what is the most efficient approach.
I have done both - and again it depends....
Currently I prefer the second approach: keep the uFMEA as separated as possible, and if you identify a hazardous situation leading to a risk - than only forward it into your 14971 risk plan / risk analysis.
But for me the uFMEA is a design input tool, that helps me to a)document observed / known use errors b)helps me to think about potential use errors / failure modes, together with the team and c)shows that we are compliant. But this is the least important, to be honest...

  • Would you organise your user interface specification into each requirement per primary operating function?
  • Is it common to see a separate user interface specification from the design input requirements, if so how would the design input requirements reference forward? (this links back to my other thread on mapping design input requirements, thank you for your input on that)
Yes! Absolutely! And I really hate it.... ;-)
Traceability is a major concern, I have not yet seen a UI spec that solves this really nice.
In my dream world I have my wireframe / axure prototype, click on a button - this opens a list of UI requirements - connected to the uFMEA, and if necessary to the use errors / risk / mitigation (why this button has to be exactly there) - the formative / summative test protocols which specify the acceptance criteria, criteria for selection into the studies - then the formative / summative test report with the objective evidence - then my summary / acceptance report - aaand finally my account details to where my boss can send the bonus for this awesome work.
I do this with different excel sheets right now, for me it works, but its not a sustainable solution...

Regarding the second point:
As a first step: I would also separate the UI spec in different parts. What I have seen as very helpful is a detailed written description, why the design is how it is. I`ve seen this named as "interaction design spec", and its really helpful for auditors but also new designers, who have to pick up the design. For them its also helpful to see the history of the UI, from first scetches, maybe formative study results, etc up to the final product.
In the end there is a list of requirements, connected to POFs, and then a reference to the UI part. But without the context (the history, so to say) I find this really useless.
 

Tobias_HF

Starting to get Involved
#9
my main struggle overall in linking usability and risk is how I feel that the definition of hazardous situation isn't consistently applied within the usability standards and risk standard and how to manage that misalignment to ensure consistent analysis - but that's an entirely different topic.
Same struggle here.... :-|
 

ThatSinc

Involved In Discussions
#10
Thank you for your reply, I think my mind is on the right path I just have difficulty in putting it down on paper in the right order.

Currently I prefer the second approach: keep the uFMEA as separated as possible, and if you identify a hazardous situation leading to a risk - than only forward it into your 14971 risk plan / risk analysis
So for your uFMEA you use different scoring criteria to your "main" risk file?
Is this criteria documented as a part of your usability engineering process?
As any identified hazard/hazardous situation from a hazard use scenario is mapped forward/upward into the 14971 risk file and assessed there for probability and severity, is there any requirement to document an occurrence rate within the uFMEA?
clause 5.5 and the guidance to that clause seem to advise against it, stating it's only the severity that can be used for assessment in the absence of robust data, but I have routinely seen it in the past with minimal data to support.


As a first step: I would also separate the UI spec in different parts.
I currently have a "User Interface" section within my design input requirements document, with connections, ergonomics, labelling, layout, in.
Some of the requirements in these sections overlap with the "functional requirements" section and "safety" section, but none contradict each other.

Same struggle here.... :-|
I think having an entirely separate uFMEA and mapping that forward will alleviate some of the struggle, with the hazardous situation being the *cause* of the exposure to the hazard in the uFMEA and the hazardous situation being the exposure to the hazard in the Hazard Analysis.

e.g.
Table b.2. from 60601-1 has the use of the glucose meter being difficult to read as the hazardous situation (which results in being administered excessive amounts of insulin as a subsequent event and harm occurring as a result)
Table c.3. from 14971 having the failure to deliver insulin as the hazardous situation, and harm occurring directly as a result of the hazardous situation.
 
Thread starter Similar threads Forum Replies Date
R Assessing Pipette Calibration Failures General Measurement Device and Calibration Topics 5
B Interpreting "misuse" when assessing Hazardous Situations ISO 14971 - Medical Device Risk Management 2
A Escalation to CAPA - Assessing if an NC warrants a CAPA Nonconformance and Corrective Action 4
A Assessing Risk for Medical Device Software ISO 14971 - Medical Device Risk Management 7
A Assessing/Mapping Employee Attitude during Competency Mapping (Assessment) IATF 16949 - Automotive Quality Systems Standard 15
J Assessing compliance with ISO 13485 Section 6.1 ISO 13485:2016 - Medical Device Quality Management Systems 10
A Assessing a Preventive Maintenance Strategy - Reliability or Maintenance Statistics Reliability Analysis - Predictions, Testing and Standards 2
D Assessing the Validity of Previous Measuring Results? General Measurement Device and Calibration Topics 8
L Assessing Corrosion Damages on Steel finished externally with Epoxical Paint Various Other Specifications, Standards, and related Requirements 1
Mikey324 Assessing Potential Field Failures (TS 16949 Requirements) Quality Manager and Management Related Issues 5
G Assessing Process Capability on Variation (Hardware Adjustment Mean Shift) Capability, Accuracy and Stability - Processes, Machines, etc. 4
B Assessing a Suppliers Technical Capabilities Supplier Quality Assurance and other Supplier Issues 6
S Objectives and Targets - Assessing a rate of achieving a goal Reliability Analysis - Predictions, Testing and Standards 7
J Assessing the understanding of occupational health and safety requirements Occupational Health & Safety Management Standards 3
T Assessing actuality to apply ISO 14001 ISO 14001:2015 Specific Discussions 12
Douglas E. Purdy Storage & Inventory - Assessing Stock for Deterioration at Planned Intervals 7.5.5.1 IATF 16949 - Automotive Quality Systems Standard 9
B ISO10012:2003 Question - Choosing or assessing the capability of a piece of equipment Other ISO and International Standards and European Regulations 1
A Assessing and managing monopolist suppliers Supplier Quality Assurance and other Supplier Issues 6
J Discrepancies - Determine the Magnitude and Assessing the Risk Nonconformance and Corrective Action 2
L Internal Auditing & Assessing Effectiveness Internal Auditing 8
L Internal Auditing & Assessing Effectiveness Internal Auditing 8
T Assessing Customer SPECIFIED Suppliers Supplier Quality Assurance and other Supplier Issues 9
M Informational Work in progress at the FDA for biological evaluation – Color Hazard and RISk calculator (CHRIS) Medical Device and FDA Regulations and Standards News 0
A Fire hazard classification - Clause in IEC 60601 about gauze and electricity ISO 14971 - Medical Device Risk Management 0
N Should it even be on the hazard analysis (software)? FMEA and Control Plans 2
S ISO 14971 Risk Management - Questions for Hazard identification ISO 14971 - Medical Device Risk Management 2
A Moving and positioning of patient - Mechanical hazard IEC 60601 - Medical Electrical Equipment Safety Standards Series 18
A Risk-benefit Analysis - Hazard Analysis (HA) and FMEAs ISO 14971 - Medical Device Risk Management 17
S The Severity of a Medical Device Hazard - Risk Analysis Clarification ISO 14971 - Medical Device Risk Management 6
D Hazard analysis for our medical device - Hazards seem to overlap 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
F Medical Device HACCP (Hazard Analysis and Critical Control Point) Risk Management ISO 14971 - Medical Device Risk Management 2
L No entanglement hazard of applied parts cables? IEC 60601 - Medical Electrical Equipment Safety Standards Series 4
D Biological and Chemical Risks to the user in the Hazard Analysis ISO 14971 - Medical Device Risk Management 1
E EN/UL/CSA 61010 -1 (ed3) Safety and use of Software to Prevent Electrical Hazard CE Marking (Conformité Européene) / CB Scheme 3
M Hazard Identification for Pest Control Activities Other ISO and International Standards and European Regulations 9
N Hazard Probability of Occurrence Definition ISO 14971 - Medical Device Risk Management 11
A ISO 14971 Figure E.1 that starts with Hazard and ends with Risk ISO 14971 - Medical Device Risk Management 3
R Risk Analysis and Hazard Identification concerning Clinical Decision Support Systems ISO 14971 - Medical Device Risk Management 1
M IVDD ER 8.3 Vs. Hazard Labelling Regulations - Which outranks? EU Medical Device Regulations 2
G Combining Aspect Impact and Hazard Risk Register Miscellaneous Environmental Standards and EMS Related Discussions 8
R Risks which must be Distinctly Identified - Harm, Hazard, Severity ISO 14971 - Medical Device Risk Management 7
S Hazard Identification and Risk Assessment - Can Risk Assessment be "Grandfathered"? Occupational Health & Safety Management Standards 4
2 Medical Device Hazard Risks - Normal Use Risks vs. Faulty Use Risks ISO 14971 - Medical Device Risk Management 4
K Regarding ISO 62304 Clause 7.3.3 - Hazard and Risk Controls IEC 62304 - Medical Device Software Life Cycle Processes 9
S PMA Submission Software Hazard Analysis Document vs. Product Hazard Analysis doc ISO 14971 - Medical Device Risk Management 5
V FMEA vs. Process Hazard Analysis (PHA) FMEA and Control Plans 2
S Aspect vs. Impact and Hazard vs. Risk - Short/clear explanation & example Miscellaneous Environmental Standards and EMS Related Discussions 11
K When, whom, how to conduct hazard analysis of PRP?s, OPRP?s and CCP?s? Food Safety - ISO 22000, HACCP (21 CFR 120) 1
K Seeking guidance on Preliminary Steps to enable Hazard Analysis Food Safety - ISO 22000, HACCP (21 CFR 120) 1
K Differences between "Hazard" and "Risk" in ISO 22000 Food Safety - ISO 22000, HACCP (21 CFR 120) 4
Similar threads


















































Top Bottom