Estimation of overall residual risk. How to?

[IOANNIS.G]

Dear community,

I am struggling to understand how to estimate overall residual risk (as it is also described in ISO 14971:2019). In fact, I am trying to make it simple in order to understand it. I would very much appreciate if you can provide some help by considering an example:

I have (for example) a medical device or an in vitro diagnostic medical device. Lets assume that for this device I have identified 3 hazards/hazardous situations and after analysing the associated risks (by using for example a 3x3 risk matrix--> probability of occurence of the harm and severity of the harm), I get the following risk levels (by adding the probability and severity values):

Hazard #1 : Risk level=2 (for example this is considered low risk)
Hazard #2: Risk level=4 (for example this is considered medium risk)
Hazard #3: Risk level=6 (for example this is considered high risk)

(Individual) Risk acceptance criterion: Risk level =<3.

For the sake of simplicity, lets assume that the above risk levels are also the final residual risk levels per hazard.

How can I estimate the overall residual risk for this device?

 

[Steve Prevette]

A quick search on residual risk led to this reasonable looking article What is Residual Risk? How is it Different from Inherent Risk? (techtarget.com)

Sounds like the methodology is you complete your initial risk analysis, PRA, FMEA and then look at the decisions you made on removing the risk, mitigating the risk, and those risks that you chose not to deal with due to cost vs benefit. So if you decided Hazard #1 was an "Oh never mind" issue, that is still some level of residual risk that you accepted. For Hazard #2 you decided that a warning label for the user was sufficient, you still have the possibility of it not being foolproof as fools are so ingenious (Samuel Clemens) and there is a residual risk.

 

[Ed Panek]

The Inherent risk of an Xray Device is the damage X-rays have on DNA.

The residual risk is after shielding, focused energy, and power levels have been mitigated or addressed.

A risk analysis might make the X Ray levels still problematic in which case a risk benefit analysis is undertaken. Slight increase in cancer vs detection of cancers or broken bones.

 

[Tidge]

WARNING: What follows should be considered bits of my own attitude to 14971-compliant risk management. Your mileage may vary.

Overall Residual Risk is best understood not as a stand-alone data point; it can only be understood in the context of what the Original Risk Profile looked like prior to the implementation of Risk Controls. Medical Device Manufacturers are required to reduce the risk as much as possible, and so if you don't know where you started, it is impossible to know to what extent you have reduced the risk (if at all).

Risk Control Option Analysis is a mechanism by which it may be possible to determine that some risks cannot be reduced further.

The Overall Risk-Benefit Analysis (or Benefit-Risk Analysis, if you prefer) takes the Overall Residual Risk and makes an explicit determination that the device provides a benefit that outweighs the risks.

The final risk profile of a device (typically summarized in something like a Risk Management Report) may have to do some balancing of a final RCOA and a ORBA, but this is a potential minefield IMO. The current motivations for 14971 have IMO stripped out several of the contexts that have historically been used in RCOA and ORBA. For example, an "expense" context has been removed... presumably because folks imagine manufacturers cutting corners to save money, but some medical devices can't be priced out of reach of users. My advice is to not be so blunt as to explicitly include "financial cost" in a RM file.

 

[IOANNIS.G]

All,

Thank you for your prompt and very much appreciated feedback.

The question is how can I estimate or calculate the overall residual risk for a medical device after I have estimated the individual residual risks (see my simplified example).

At the end of the day I have to decide not only whether the residual risk #1, #2 etc. are acceptable, but (furthermore) I must assess the overal residual risk for this device. The question is how to do it.

 

[Ed Panek]

Is the device new in the market? If not you can use the Maude FDA database to see how often and what type of issues are reported and use that as a launch-off point

You should understand the Maude database as we are required to update risk numbers as data comes in. You may estimate a new type of risk based on analysis.

 

[Tidge]

The question is how can I estimate or calculate the overall residual risk for a medical device after I have estimated the individual residual risks (see my simplified example).

You should use the same scales for both.

 

[yodon]

Let me start out by saying that I don't believe there's a way to calculate an overall residual risk score.

24971 section 8.3 describes a variety of approaches for evaluating overall residual risk. @Ed Panek mentions one, comparing your device to similar ones on the market.

We normally take 2 approaches. The first (as described in 24971 8.3(b)) is basically a heat map, plotting all the final risk scores into the acceptability matrix. Never considered that particularly robust but if you see clustering towards "generally acceptable" (low end of the scores) then you can make a reasonable case for safety. If you see clustering towards unacceptable, you may need to take additional actions to justify the overall residual risk.

The second approach we take is along the lines of 24971 8.3(a) but we fold in aspects of the FDA Guidance Factors to Consider Regarding Benefit-Risk in Medical Device Product Availability, Compliance, and Enforcement Decisions. This is also generally described somewhere in one of the MEDDEVs so I think it's a pretty commonly-understood approach. And it's worked for us, so far!

 

[Tidge]

Risk management is more about information than it is about data. Data points are discrete; information is the relationship between pieces of data.

I generally prefer the "heat map" approach, so that the process of getting-to-the-final-answer looks like:

1. After the implementation (and evaluation) of risk controls, see how the "heat" is reduced (or how risks migrate from high to low)
2. Focus on the high heat areas, explaining why they are remain "high heat" and why they (a) cannot be reduced further and (b) why this final profile is acceptable

The information in point 1 is about the informed transition from one state of knowledge to a (better understood) state of knowledge.
The information in point 2 is about the relationship between that (final) state of knowledge and the medical context of the device.

Keep in mind: It is always possible that something that was once considered low risk becomes high risk because of some new, fundamental data/information (e.g. bacteria growth in cooler-heaters used during CABG), this is part of the reason why periodic risk reviews are done.

 

[IOANNIS.G]

You should use the same scales for both.

Hi Tidge. Thanks for your answer. It helps me a step further. My question would be completely answered if someone could estimate what is the overall residual risk based on the example I have mentioned. Then, I think, I would completely understand how to estimate the overall residual risk from the individual residual risks (see example).

Until now, I have (for the example medical device):

Residual risk #1=2
Residual risk #2=4
Residual risk#3= 6

Some of those residual risks are acceptable and some not (based on the established criteria for acceptability).

Now I want to estimate the overall residual risk for this example medical device.....and I am stuck. Whether this will be acceptable or not is another question.