What is the difference between "Overall Risk" and "Risk"? (ISO 14971)

C

cytokinesis

#1
I'm seriously at a loss here.

Before I start, don't chime in with something like "Overall residual risk evaluation is the point where residual risk is viewed from a broad perspective."
Please just don't :frust:

Scenario A
The first analysis lends itself to modularity, and so I've broken it up into about a dozen sections. Each section has its own set of hazard identifications, estimations, controls, and residual risk assessments. All of the hazards in each section have been mitigated, and the residual risks are acceptable.

So far so good.

What exactly does an examination of overall residual risk look like? Its not like I have any new hazards to add.

My thought was to take the most severe risk from each section, put the 12 of them onto one risk table, and demonstrate that all of them fall within the acceptable zone.

That seems a little pointless, since its just a repetition of information from the lower-level analyses, but I'm honestly not sure what further steps are expected.

Scenario B
Assuming my plan for scenario A makes sense, what does "overall risk" mean in a smaller scenario when all of my hazards are considered in one risk study?

Any clarity you guys can give me on this subject is greatly appreciated.

Thanks.
 
Elsmar Forum Sponsor

yodon

Staff member
Super Moderator
#3
I was just involved in a discussion where we talked about this. Indeed, it's not easy to nail down.

The example that came up in the discussion was good for the purpose of understanding the concept. It was a cartoon with many strips of duct tape covering small cracks all over the fuselage of an airplane. Individually, each risk was (arguably) sufficiently controlled but the overall risk would be quite bad.

This followed with a good discussion of AN approach to making it a bit more tangible through the use of a heat map. If you're using the standard red/yellow/green matrix, you can plot all the residual risks in the cells. If you're clustered down in the green, you may have an argument for a pretty safe product. If there's a lot of cluster near the red, maybe a concern.

I tried this out on one simple project and it worked pretty well! I did have some clustering in the yellow but a review of those risks showed that it was endemic to the device type and not to the device itself (i.e., all such devices have the same risk ... and the only mitigation was through labeling alone so no reduction).
 

Marcelo

Inactive Registered Visitor
#4
f you're using the standard red/yellow/green matrix, you can plot all the residual risks in the cells. If you're clustered down in the green, you may have an argument for a pretty safe product. If there's a lot of cluster near the red, maybe a concern.
Unfortunately, this make no sense for the same reason you mentioned.

Even if a risk matrix could be used to verify risk acceptability (it cannot, it's only a risk ranking tool), having a plot of risks in a "zone"only means that the risk are "on the same level" (even this is not true) when compared to each other. It says nothing about the overall risk.
 

Marcelo

Inactive Registered Visitor
#5
As I mentioned before, you can use some techniques, such as risk summing, if you want to perform overall risk evaluation focused on quantitatively risk management, but there's no risk summing method that is considered consensus in the literature.

Another questions is if a risk summing method is necessary.
 

yodon

Staff member
Super Moderator
#6
Unfortunately, this make no sense for the same reason you mentioned.

Even if a risk matrix could be used to verify risk acceptability (it cannot, it's only a risk ranking tool), having a plot of risks in a "zone"only means that the risk are "on the same level" (even this is not true) when compared to each other. It says nothing about the overall risk.
Hmm.. I think I'll disagree. It does show where the residual risks have accumulated. I'm not saying it's necessarily the best or only method. And clearly, it can be misused and possibly be misleading. As I noted, when I plotted it out for one of my projects, the results were quite clear and telling. If nothing, I think it's a good tool for the toolbox.

I haven't fully digested the risk summing paper you provided in the other post (thank you, by the way) but in a quick scan, it looks like they're using just this technique in at least part of the method.

Certainly more to consider and I'm glad the original poster initiated the discussion. I would hope others would weigh in with their approaches to assessing overall residual risk.
 
C

cytokinesis

#7
Thank you both for your help. :thanx:

I've followed along the "Understanding and Applying Total Risk Summing" slides you'd linked in your other post, Marcelo.

The thing that still escapes me is, once I've calculated ELR and CLR, what am I ultimately trying to determine in doing so?

Is the point to ensure that the resulting "iso-risk" line does not fall on the unacceptable risk part of the graph?
 

Ronen E

Problem Solver
Staff member
Moderator
#8
I would hope others would weigh in with their approaches to assessing overall residual risk.
Honestly, that stage in ISO 14971 is something I never really felt comfortable with. That and the risk/benefit ratio.

Perhaps risk summing is the way to go (haven't read the paper yet), but I feel what industry needs is less process-burden in risk management, not more. Some (most?) people just won't follow if the process is too complicated / lengthy / burdensome.
 

Marcelo

Inactive Registered Visitor
#9
Hmm.. I think I'll disagree. It does show where the residual risks have accumulated. I'm not saying it's necessarily the best or only method. And clearly, it can be misused and possibly be misleading. As I noted, when I plotted it out for one of my projects, the results were quite clear and telling. If nothing, I think it's a good tool for the toolbox.
The problem is that use the matrix alone (as always). A risk matrix does not show "accumulated" risk as you mentioned. It's not a tool for that (although you might think it shows this). You would be better using density strips plot or something like that as a heat map.

But even this does not really take into account the overall risk, as it simply is a visual way to show individual risks.
 
Last edited:

Marcelo

Inactive Registered Visitor
#10
I did mention sometimes the risk summing example, but I don't like it either.

One of the problems with the overall risk is that it will need different acceptance criteria than the individual risks. This is superficially mentioned in ISO 14971 but due to the spread use of the risk matrix as the only "acceptance" tool, it usually is just not taken into account.

Another problem, which goes back to the "automatic" use of the risk matrix, is that risk evaluation (and acceptability) should have a strong rationale, based in technical arguments. This is easier done (and stronger), than a mathematical tool such as a risk summing or any other (and goes that way Ronen mentioned as a less-burdensome process, although it will be more burdensome in a technical way).

Something that I've been discussing the in revision of ISO 14971 is the need for better arguments for risk acceptability, see for example document An Introduction to System Safety Management in the MOD, item 6.4 Making Risks ALARP where it mentions the HSE principles for claiming that a risk is ALARP.

This is also the same type of discussing regarding the risk/benefit analysis, which is right now a "get out of the jail" free card in ISO 14971, but should in fact also be part of risk evaluation (and the risk acceptability criteria).

A good way to show how this can be done is detailed in the FDA document Factors to Consider Regarding Benefit-Risk in Medical Device Product Availability, Compliance, and Enforcement Decisions. Again here, the analysis has nothing to do with estimating probability and severity (although it uses the ones estimated as factors), but focus on technical arguments.

In fact, I've been using these arguments from the FDA doc as a basis for overall residual risk evaluation for my clients (it's not exactly that, but it does make it more clear to the clients which insists on using risk matrices only that it's not just that).
 
Last edited:
Thread starter Similar threads Forum Replies Date
A What is the difference between Design Process, Process Design and Design Control? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
R What's the major difference between Green Belt and Black Belt in term of training and project Six Sigma 3
T Difference between a subcontractor and a supplier ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 21
H Difference between Stainless Steel 316 ASTM F899 and ASTM A276 Other Medical Device Related Standards 3
A Exact terms for a plating failure and difference between rejection rate and failure rate Manufacturing and Related Processes 9
M Difference between MSA and MSE? General Measurement Device and Calibration Topics 1
gramps What is the difference between discrete and continuous variables? Problem Solving, Root Cause Fault and Failure Analysis 5
JoCam Difference between Approval and Registration - ISO 13485:2016 ISO 13485:2016 - Medical Device Quality Management Systems 2
S Difference between EU-MDR Annex IX and the Annex-combo X&XI EU Medical Device Regulations 2
T ISO 17025:2017 Clause 4.2.2 - The difference between "be notified" and "be informed" ISO 17025 related Discussions 4
Jimmy123 What is the difference between Error Proofing and Controls? ISO/IATF 16949 - Control Plans FMEA and Control Plans 16
C IEC 60601-1-8, difference between table 4 and annex D IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
A What is the difference between Basic UDI-DI and UDI-DI? EU Medical Device Regulations 5
Q What is the difference between AS9100D 9.3.2.f and 9.3.3.a AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 7
M Aluminum - What is the difference between 6061-T6 and 6061-T651 (both per ASTM B211)? Manufacturing and Related Processes 4
P What is the exact difference between Risk and Opportunity in context of ISO 27001? IEC 27001 - Information Security Management Systems (ISMS) 7
S Difference between Surface Finish (Ra) and Flatness (GD&T) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
O Air Flow - Which is the operational difference between LAF (vertical and horizontal) and RLAF? Manufacturing and Related Processes 2
S What the difference is between Stub Acme & Acme thread? Oil and Gas Industry Standards and Regulations 1
S DO 178B - What is the difference between review and verification? Federal Aviation Administration (FAA) Standards and Requirements 1
T The difference between SOP and Kaizen Standardization Lean in Manufacturing and Service Industries 2
K Difference between intended purpose and intended use of the device EU Medical Device Regulations 9
Q What is the difference between normal and licensed internal auditor? VDA Standards - Germany's Automotive Standards 9
J What is the difference between Process Variation and Tolerance? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
O Difference Between PFMEA & Control Plan FMEA and Control Plans 3
S Difference between an Advisory Notice (ISO 13485) and a Field Safety Notice? ISO 13485:2016 - Medical Device Quality Management Systems 3
T Difference between "data analysis" and "management review" ISO 13485:2016 - Medical Device Quality Management Systems 4
qualprod What is the difference between 7.4.1 (2008) and 8.4.1 (2015)? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
D Difference between uncertainty and expanded uncertainty of measurement General Measurement Device and Calibration Topics 1
S EASA Part 145 - The difference between non-certifying staff and Certifying staff Federal Aviation Administration (FAA) Standards and Requirements 2
S The difference between a Medical Device Accessory and Component Canada Medical Device Regulations 2
S What is the difference between a service request and a complaint? ISO 13485:2016 - Medical Device Quality Management Systems 2
I What is the difference between OOS/OOT Imported Legacy Blogs 3
A Definition Difference between Quality System Procedure and Standard Operating Procedure (SOP) Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 4
Q Difference between Monitor & Measurement ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
T QA vs. RA - The difference between QA and RA Coffee Break and Water Cooler Discussions 6
A Difference between Discrimination and Least Count IATF 16949 - Automotive Quality Systems Standard 2
K Difference between Medical Electrical Equipment and Medical Equipment System IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
M Usability Standard - The difference between IEC 60601-1-6 to IEC 62336 Human Factors and Ergonomics in Engineering 1
S What is the difference between Verification and Validation of Test Reports? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
A Difference between ISO 9001:2015 Clauses 8.1 and 8.5.1 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
M IATF 16949 Cl. 8.4.2.2 vs 8.6.5 - Is there any difference between these clauses? IATF 16949 - Automotive Quality Systems Standard 7
C The difference between a Critical Dimension vs. Critical Characteristic Misc. Quality Assurance and Business Systems Related Topics 2
A ISO 9001:2015 - The difference between 6.1.1b and 6.1.1d - IMHO, d) implies b) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
A Difference between requirement for 4.4.1a (inputs) and 4.4.1d (resources needed)? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
R The difference between Feasibility Assessment and Contract Review Contract Review Process 2
V Difference between Non-Conformance, Variance and Deviation ISO 13485:2016 - Medical Device Quality Management Systems 7
A Difference between Environmental Impact and Risk ISO 14001:2015 Specific Discussions 7
A Difference Between Complaints and Feedbacks in terms of Follow-Up Actions Needed Customer Complaints 3
M What is the difference between ISO 9000 and ISO 9001? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2

Similar threads

Top Bottom