Questions on Hazard Analysis (addition of similar hazardous situations, assessment of overall residual risk)

stm55

Involved In Discussions
Hi All!

Hoping for some input from some Risk Management experts! We're trying to make updates to our RM documents to align with 14971 expectations. There's a lot to update (basically transitioning away from using FMEAs as the focal point of our RM), but I think we are moving in the right direction.

A couple items I wanted some further guidance on:

1) What is the most appropriate way to aggregate similar hazardous situations (if at all)? To give a specific example:-

Let's say we have a hazard line item related to Bacteria exposure for a catheter. This could be broken down into a few different sequence of events/hazardous situations (i.e. Device used after expiration of sterile barrier when bacteria has been introduced onto device; device not aseptically handled; device reinserted into patient; sterile barrier not intact and device inserted into patient; device not properly sterilized by manufactuer; etc). There are a few degrees of associated "Infection" Harms with different severities (i.e. Fever, Serious infection requiring surgery, death).

What is the best (or at least a recommended) way of documenting this in a Hazard Analysis? I was first thinking of doing this as a specific Hazard line item (i.e. HAZ-001: Bacteria), with sub-lines of the different Hazardous situations/sequence of events. There'd be the different Harms associated as well with this overall line item (same harms for each HazSit/Seq). Then it gets tricky for me... I could blanketly say P1 is a single number of those situations happening (i.e. 0.001) or I could give each HazSit its own P1. For P2, let's say 0.10 for Fever, 0.01 for Serious infection, 0.001 for Death. Then I figure we would multiply P1*P2 for each iteration (i.e. .01*P_Fever... .01*P_Death) and multiply each by the corresponding severity. This may lead to different composite scores (i.e. a High probability of fever may be a better or worse overall score compared to a very low probability of death) which can be used to assess if Controls are needed.

Is this an appropriate way to look at this? or is there an easier option? We could alternatively calculate the individual P1's for each HazSit, but then that seems to complicate the overall table.

2) My next question is then around how to look at the different Hazardous Situations related to the same Hazard-- separately or combined? For example, if you are saying that there are 50 distinct ways of exposing the user to the Bacteria Hazard, should you add them all up to get an overall P1 (i.e. Exposure to bacteria from any means)? Like if you're saying there's a 1 in 1000 chance of device being reinserted, a 1 in a 1000 chance of device packaging being compromised, etc, there's an overall 50 in 1000 chance of bacteria exposure... the 1 in 1000's may individually be ok, but the 50 in 1000 may require further action. Or could they just be considered individually and the overall residual risk could assess that?

Is this more appropriately handled on a case by case basis? For example, maybe the bacteria exposure may have a bunch of relatively unrelated hazardous situations (and therefore could be better assessed separately), but perhaps there's a different Hazard that has relatively similar hazardous situations

3) As for the overall residual risk assessment -- does anyone have practical examples of what this could look like? I reviewed 24971 and I understand the theory, but would be good to see an actual example put into practice.

Sorry for the long post, but appreciate any help!!!!!!!
 

Tidge

Trusted Information Resource
Some of the questions are currently being addressed elsewhere, I'll make a comment on

3) As for the overall residual risk assessment -- does anyone have practical examples of what this could look like? I reviewed 24971 and I understand the theory, but would be good to see an actual example put into practice.

I've seen this addressed in a variety of ways, often at the same company. Mileage varies.

A (defective) approach I witnessed at one company was a simply counting of lines-of-analysis and applying some sort of arbitrary counting measure as the assessment. Don't do this.

One approach was to make a stand-alone "overall residual risk assessment" document. This document was (my opinion) a sort of fluff piece (done to close a gap in an audit finding) that said little more than "per policy _____ and the risk analysis document(s) such-and-such, we the undersigned find the risks to be acceptable / acceptable given these benefits (circle one)."

That evolved into a slightly more coherent policy of including the overall residual risk assessment in the risk management report.

In each of those cases, I saw a tendency to not want to take much of a look at the documents that are nominally supporting the conclusion of an overall risk assessment (or Benefit-Risk Analysis). A single Risk Control Options Analysis (tied to the Hazard Analysis) can be used to support an overall residual risk assessment and BRA, but this works best if the RCOA is focused more on the risks and how they are actually controlled, and less about making some statement about each of the possible controls tied to any one risk's line of analysis. Folks have different attitudes about how to construct and use RCOA.
 

d_addams

Involved In Discussions
Hi All!

Hoping for some input from some Risk Management experts! We're trying to make updates to our RM documents to align with 14971 expectations. There's a lot to update (basically transitioning away from using FMEAs as the focal point of our RM), but I think we are moving in the right direction.

A couple items I wanted some further guidance on:

1) What is the most appropriate way to aggregate similar hazardous situations (if at all)? To give a specific example:-


2) My next question is then around how to look at the different Hazardous Situations related to the same Hazard-- separately or combined? For example, if you are saying that there are 50 distinct ways of exposing the user to the Bacteria Hazard, should you add them all up to get an overall P1 (i.e. Exposure to bacteria from any means)? Like if you're saying there's a 1 in 1000 chance of device being reinserted, a 1 in a 1000 chance of device packaging being compromised, etc, there's an overall 50 in 1000 chance of bacteria exposure... the 1 in 1000's may individually be ok, but the 50 in 1000 may require further action. Or could they just be considered individually and the overall residual risk could assess that?

Is this more appropriately handled on a case by case basis? For example, maybe the bacteria exposure may have a bunch of relatively unrelated hazardous situations (and therefore could be better assessed separately), but perhaps there's a different Hazard that has relatively similar hazardous situations

3) As for the overall residual risk assessment -- does anyone have practical examples of what this could look like? I reviewed 24971 and I understand the theory, but would be good to see an actual example put into practice.

Sorry for the long post, but appreciate any help!!!!!!!
for 1 - there are no 'similar' hazardous situations, they are the same or they aren't. One of the issues, imo, is you've mis-identified the hazard as 'bacteria' instead of the product anti-function. The product has a function of 'be sterile' or 'maintain sterility', so the hazard is 'loss of sterility'. There can be 50 different failure modes to get to 'loss of sterility', but they all get you to loss of sterility. When you use the anti-functions as your hazard, the answer to #1 becomes clear, all losses of the same function result in the same hazard(s).

for 2 - combined, sort of. Risk Management has the illusion of a quantitative practice, but it is mostly qualitative assessments occasionally informed by quantitative data. Case in point here is the false assumption of precision. Do you really believe you have such precise estimates of P1 that you can 'simply sum' all the P1s to get an accurate estimate of the overall occurrence? How we handle this is a) make it clear in the HA/Risk Assessment that those P1s are informed by the individual estimates, prior performance, and engineering judgement, not quantitative sums. So you'll need to use judgement to adjust those 'summed' P1s so that the overall harm estimates make sense. A bit of top down (known harm rates) and bottoms up (known failure mode rates) depending on the data quality will eventually get you a usable overall assessment.

for 3 - to start you'll need to demonstrate meeting your risk acceptability criteria. End points have been achieved and if you did state explicit criteria (ie. a quantitative profile) you'll need to have met those. In all cases it still comes down to making a subjective benefit/risk assessment. Essentially the 'benefit/risk conclusion' magically is just stated after some long winded descriptions of benefits and risks.

While people talk about wanting to do quantitative benefit assessment and quantitative risk assessment to 'make benefit/risk assessment easier' there won't be any inherently obvious answers at the end of that rainbow. Mostly because benefit and risk have different units and thus their balance cannot be evaluated numerically. One can build a construct of defined limits, etc. However, the declaration of that construct as the benefit/risk assessment criteria is still completely subjective. I wouldn't dissuade anyone from advancing their quantification of benefits and risks, but I would stop them from believing doing so enables a quantitative assessment benefit/risk without a subjective foundation.
 

ThatSinc

Quite Involved in Discussions
One of the issues, imo, is you've mis-identified the hazard as 'bacteria' instead of the product anti-function. The product has a function of 'be sterile' or 'maintain sterility', so the hazard is 'loss of sterility'.

I'd say that bacteria is most definitely a hazard and that loss of sterility is part of a sequence of events that would lead to a hazardous situation, that could lead to harm.
A hazard is a source of harm, something that when exposed to can cause harm.

To limit consideration of a hazard to a product anti-function would somewhat negate the normal use risks that 14971 requires you to assess.
e.g. sharp edges on a scalpel blade, whilst being a functional requirement of the device, are a hazard; assessing the risks of these in normal use is still required.



To the question of overall residual risk; where there are multiple different sequences of events that could lead to any identified hazardous situation, I tend to look at those sequences of events, how they have been controlled, and why the risk exists in the first place (i.e. is the reason that the risk exists in the first place a part of the device function).

I find the activity of justifying why risks exist in the device and why you can't reduce them further, or why reduction would adversely affect the risk/benefit ratio, goes a long way to justification of overall residual risk acceptability.

Look at how you have defined your risk acceptability criteria, do you just have a "risk acceptability matrix" or do you have other criteria stated in a more bullet point/prescriptive fashion?

TS.
 

d_addams

Involved In Discussions
I'd say that bacteria is most definitely a hazard and that loss of sterility is part of a sequence of events that would lead to a hazardous situation, that could lead to harm.
A hazard is a source of harm, something that when exposed to can cause harm.

To limit consideration of a hazard to a product anti-function would somewhat negate the normal use risks that 14971 requires you to assess.
e.g. sharp edges on a scalpel blade, whilst being a functional requirement of the device, are a hazard; assessing the risks of these in normal use is still required.



To the question of overall residual risk; where there are multiple different sequences of events that could lead to any identified hazardous situation, I tend to look at those sequences of events, how they have been controlled, and why the risk exists in the first place (i.e. is the reason that the risk exists in the first place a part of the device function).

I find the activity of justifying why risks exist in the device and why you can't reduce them further, or why reduction would adversely affect the risk/benefit ratio, goes a long way to justification of overall residual risk acceptability.

Look at how you have defined your risk acceptability criteria, do you just have a "risk acceptability matrix" or do you have other criteria stated in a more bullet point/prescriptive fashion?

TS.
Bacteria is one type of infectious microbe. Are you going to list virus and mold spores and every toxic substance and on and on as your 'hazards'? The value in using the anti-functions is it drives a direct line of sight between the requirements and risk (all requirements are risk mitigations to prevent the anti-functions the anti-functions are the sources of harm).

Your concern with 'normal use risks' assumes that a FMEA (or failure conditions) is the only design analysis you are conducting. Use a 'Therapy Risk Assessment' to capture those risks inherent in the process. The p1=1 and the P2s are the reported rate. While the harms are identical if not indistinguishable from those associated with a failure of the product, for complex products/therapies it really clarifies the sources of your risk and how to manage them. For the example of infection, your observed rates will be orders of magnitude greater than your instances of sterile barrier failure or user error. Having to point back to a FMEA with only product failure and user error is unsatisfying to me when reconciling performance with your risk file.
 

Bryan Ye

Starting to get Involved
For the assessment of the overall residual risk, we can give an example, as shown in the figure below, if the red and blue parts are both acceptable for a single risk.
If all risks fall in the red part and all risks fall in the blue part, it is obvious that the overall residual risk is different. Red is obviously the best result.
You need to decide how all individual risks are distributed to be acceptable, which is what the standard says, "taking into account the contributions of all residual risks". One way to do this is to set a weight factor for each small square. Combine the weight factor and the number of risks to calculate the overall risk of each small square. Then you can get the overall residual risk.
The difficulty lies in setting the weight factor and deciding the acceptable value of the overall residual risk.

I just give my idea based on the explanation of the standard below. I don’t know if there is a better solution. It seems too complicated to set a weight factor for each risk.

The evaluation of overall residual risk is a challenging task that cannot be achieved by adding all
individual risks numerically. The difficulty arises for the following reasons:
— Each probability of occurrence of harm is related to a different harm with different severity and can
be related to different hazardous situations.
— Probabilities are often known with different degree of uncertainty. Some probabilities could
be known precisely from either historical data or testing. Other probabilities might be known
imprecisely such as estimates by expert judgment, or cannot be estimated such as the probability of
a software failure.
— It is not possible to combine the severities of individual harms within the broad categories usually
employed in risk analysis.



Questions on Hazard Analysis (addition of similar hazardous situations, assessment of overall residual risk)
 
Top Bottom