Estimation of overall residual risk. How to?

#1
Dear community,

I am struggling to understand how to estimate overall residual risk (as it is also described in ISO 14971:2019). In fact, I am trying to make it simple in order to understand it. I would very much appreciate if you can provide some help by considering an example:

I have (for example) a medical device or an in vitro diagnostic medical device. Lets assume that for this device I have identified 3 hazards/hazardous situations and after analysing the associated risks (by using for example a 3x3 risk matrix--> probability of occurence of the harm and severity of the harm), I get the following risk levels (by adding the probability and severity values):

Hazard #1 : Risk level=2 (for example this is considered low risk)
Hazard #2: Risk level=4 (for example this is considered medium risk)
Hazard #3: Risk level=6 (for example this is considered high risk)

(Individual) Risk acceptance criterion: Risk level =<3.

For the sake of simplicity, lets assume that the above risk levels are also the final residual risk levels per hazard.

How can I estimate the overall residual risk for this device?
 
Elsmar Forum Sponsor

Steve Prevette

Deming Disciple
Leader
Super Moderator
#2
A quick search on residual risk led to this reasonable looking article What is Residual Risk? How is it Different from Inherent Risk? (techtarget.com)

Sounds like the methodology is you complete your initial risk analysis, PRA, FMEA and then look at the decisions you made on removing the risk, mitigating the risk, and those risks that you chose not to deal with due to cost vs benefit. So if you decided Hazard #1 was an "Oh never mind" issue, that is still some level of residual risk that you accepted. For Hazard #2 you decided that a warning label for the user was sufficient, you still have the possibility of it not being foolproof as fools are so ingenious (Samuel Clemens) and there is a residual risk.
 

Ed Panek

QA RA Small Med Dev Company
Leader
Super Moderator
#3
The Inherent risk of an Xray Device is the damage X-rays have on DNA.

The residual risk is after shielding, focused energy, and power levels have been mitigated or addressed.

A risk analysis might make the X Ray levels still problematic in which case a risk benefit analysis is undertaken. Slight increase in cancer vs detection of cancers or broken bones.
 

Tidge

Trusted Information Resource
#4
WARNING: What follows should be considered bits of my own attitude to 14971-compliant risk management. Your mileage may vary.

Overall Residual Risk is best understood not as a stand-alone data point; it can only be understood in the context of what the Original Risk Profile looked like prior to the implementation of Risk Controls. Medical Device Manufacturers are required to reduce the risk as much as possible, and so if you don't know where you started, it is impossible to know to what extent you have reduced the risk (if at all).

Risk Control Option Analysis is a mechanism by which it may be possible to determine that some risks cannot be reduced further.

The Overall Risk-Benefit Analysis (or Benefit-Risk Analysis, if you prefer) takes the Overall Residual Risk and makes an explicit determination that the device provides a benefit that outweighs the risks.

The final risk profile of a device (typically summarized in something like a Risk Management Report) may have to do some balancing of a final RCOA and a ORBA, but this is a potential minefield IMO. The current motivations for 14971 have IMO stripped out several of the contexts that have historically been used in RCOA and ORBA. For example, an "expense" context has been removed... presumably because folks imagine manufacturers cutting corners to save money, but some medical devices can't be priced out of reach of users. My advice is to not be so blunt as to explicitly include "financial cost" in a RM file.
 
#5
All,

Thank you for your prompt and very much appreciated feedback.

The question is how can I estimate or calculate the overall residual risk for a medical device after I have estimated the individual residual risks (see my simplified example).

At the end of the day I have to decide not only whether the residual risk #1, #2 etc. are acceptable, but (furthermore) I must assess the overal residual risk for this device. The question is how to do it.
 

Ed Panek

QA RA Small Med Dev Company
Leader
Super Moderator
#6
Is the device new in the market? If not you can use the Maude FDA database to see how often and what type of issues are reported and use that as a launch-off point

You should understand the Maude database as we are required to update risk numbers as data comes in. You may estimate a new type of risk based on analysis.
 

yodon

Leader
Super Moderator
#8
Let me start out by saying that I don't believe there's a way to calculate an overall residual risk score.

24971 section 8.3 describes a variety of approaches for evaluating overall residual risk. @Ed Panek mentions one, comparing your device to similar ones on the market.

We normally take 2 approaches. The first (as described in 24971 8.3(b)) is basically a heat map, plotting all the final risk scores into the acceptability matrix. Never considered that particularly robust but if you see clustering towards "generally acceptable" (low end of the scores) then you can make a reasonable case for safety. If you see clustering towards unacceptable, you may need to take additional actions to justify the overall residual risk.

The second approach we take is along the lines of 24971 8.3(a) but we fold in aspects of the FDA Guidance Factors to Consider Regarding Benefit-Risk in Medical Device Product Availability, Compliance, and Enforcement Decisions. This is also generally described somewhere in one of the MEDDEVs so I think it's a pretty commonly-understood approach. And it's worked for us, so far!
 

Tidge

Trusted Information Resource
#9
Risk management is more about information than it is about data. Data points are discrete; information is the relationship between pieces of data.

I generally prefer the "heat map" approach, so that the process of getting-to-the-final-answer looks like:
  1. After the implementation (and evaluation) of risk controls, see how the "heat" is reduced (or how risks migrate from high to low)
  2. Focus on the high heat areas, explaining why they are remain "high heat" and why they (a) cannot be reduced further and (b) why this final profile is acceptable
The information in point 1 is about the informed transition from one state of knowledge to a (better understood) state of knowledge.
The information in point 2 is about the relationship between that (final) state of knowledge and the medical context of the device.

Keep in mind: It is always possible that something that was once considered low risk becomes high risk because of some new, fundamental data/information (e.g. bacteria growth in cooler-heaters used during CABG), this is part of the reason why periodic risk reviews are done.
 
#10
You should use the same scales for both.
Hi Tidge. Thanks for your answer. It helps me a step further. My question would be completely answered if someone could estimate what is the overall residual risk based on the example I have mentioned. Then, I think, I would completely understand how to estimate the overall residual risk from the individual residual risks (see example).

Until now, I have (for the example medical device):

Residual risk #1=2
Residual risk #2=4
Residual risk#3= 6

Some of those residual risks are acceptable and some not (based on the established criteria for acceptability).

Now I want to estimate the overall residual risk for this example medical device.....and I am stuck. Whether this will be acceptable or not is another question.
 
Thread starter Similar threads Forum Replies Date
A Calculating Risk Estimation ISO 14971 - Medical Device Risk Management 28
T Process Potential estimation for binary data Capability, Accuracy and Stability - Processes, Machines, etc. 3
F Issues in Uncertainty Estimation Measurement Uncertainty (MU) 5
C Uncertainty Estimation for Spark Optical Emission Device Measurement Uncertainty (MU) 1
K Estimation Sigma (Within) when subgroup n=1 (MiniTab) Capability, Accuracy and Stability - Processes, Machines, etc. 6
N Risk Severity Estimation for Medical Devices as per ISO 14971 ISO 14971 - Medical Device Risk Management 12
M Standard Deviation Estimation Formula - Student Question Statistical Analysis Tools, Techniques and SPC 4
T Sample Size Estimation using Mintab 16 Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
C Tolerance Estimation of a Load Cell I want to Calibrate General Measurement Device and Calibration Topics 9
R Procedure for Measurement Uncertainty Estimation for Electrical Measurements Measurement Uncertainty (MU) 2
S Uncertainty Estimation in Temperature Measurement by K type Thermocouples Measurement Uncertainty (MU) 7
S Software Risk Estimation: Probability of Medical Device Software Anomaly Occuring ISO 14971 - Medical Device Risk Management 9
S Type B Uncertainty Estimation - Accuracy and Calibration Error Measurement Uncertainty (MU) 4
F Reliability of a System - Estimation of R(t) having constant failure rate "Lambda" Reliability Analysis - Predictions, Testing and Standards 2
R Calibration of Jensen JTM-30 clamp meters (estimation) General Measurement Device and Calibration Topics 4
BradM Uncertainty calculation - Estimation for a Mettler Toledo XP 56 balance at 20 grams Measurement Uncertainty (MU) 11
M Time frequency estimation for SPC data collection Statistical Analysis Tools, Techniques and SPC 9
M Quality estimation of one and an entire set of elements Statistical Analysis Tools, Techniques and SPC 2
S Design and Development Estimation and Offer - Sales vs. Engineers Design and Development of Products and Processes 2
B Rational Subgroup and Estimation of Variability Due to Common Causes Statistical Analysis Tools, Techniques and SPC 7
K Cost Estimation of Sand Casting Process including the Pig Loss in estimation Manufacturing and Related Processes 7
Q Manufacturing Cost Estimation and Costing Procedures Manufacturing and Related Processes 4
J Point Estimation - I need to use the Quadratic Regression Model - Minitab 14 Using Minitab Software 6
M PFMEA Severity - What is Process FMEA Severity estimation based on? FMEA and Control Plans 77
S Estimation of uncertainity for Internal Laboratory General Measurement Device and Calibration Topics 2
T Estimation of MU for working thermometers General Measurement Device and Calibration Topics 5
W MU - Type B estimation - Which distribution to use? Triangle? Rectangular? General Measurement Device and Calibration Topics 3
M Effort estimation guidelines - Software industry (Project Management) Software Quality Assurance 5
I Realization processes input into overall risk ISO 14971 - Medical Device Risk Management 2
silentmonkey Overall Benefit/Risk Analysis - Risk Management VS Clinical Evaluation ISO 14971 - Medical Device Risk Management 3
M ISO 14971:2019: Criteria for overall residual risk ISO 14971 - Medical Device Risk Management 11
K Overall residual risk according to ISO 14971:2019 ISO 14971 - Medical Device Risk Management 5
C What is the difference between "Overall Risk" and "Risk"? (ISO 14971) ISO 14971 - Medical Device Risk Management 10
P Total Variation in Data - Overall Importance of the Principal Components Reliability Analysis - Predictions, Testing and Standards 1
Marc US wind capacity surpasses hydro, overall generation to follow World News 0
M Can OEE (overall equipment effectiveness) exceed 100%? Manufacturing and Related Processes 2
M Where does OEE (Overall Equipment Effectiveness) belong? Quality Tools, Improvement and Analysis 11
L OEE (Overall Equipment Effectiveness) and Cost Savings Manufacturing and Related Processes 2
S Measuring OEE (Overall Equipment Effectiveness) for each Job, not for Machine Lean in Manufacturing and Service Industries 2
8 OEE (Overall Equipment Effectiveness/Efficiency) Help Lean in Manufacturing and Service Industries 17
K Is there an overall standard for calibration of items used in TS 16949 Certified Co. IATF 16949 - Automotive Quality Systems Standard 3
Sam Lazzara ISO 14971 Clause 7 - Evaluation of Overall Residual Risk Acceptability ISO 14971 - Medical Device Risk Management 3
D Overall Equipment Effectiveness (OEE) Calculations Manufacturing and Related Processes 4
P Overall Cpk for Different Parts to Same Customer Statistical Analysis Tools, Techniques and SPC 6
Q Availability Calculation for monitoring Overall Equipment Efficiency Manufacturing and Related Processes 2
AnaMariaVR2 OEE (Overall Equipment Efficiency) on Pharma Packaging Lines Lean in Manufacturing and Service Industries 8
S How to Calculate Overall FPY (First Pass Yield) Quality Tools, Improvement and Analysis 2
Y OEE (Overall Equipment Efficiency) help Lean in Manufacturing and Service Industries 6
D QA's Role in the Overall Picture of the Organization and in Day to Day Activities Misc. Quality Assurance and Business Systems Related Topics 10
O Using SPC to determine overall Thread Quality Manufacturing and Related Processes 16

Similar threads

Top Bottom