What is the difference between "Overall Risk" and "Risk"? (ISO 14971)

C

cytokinesis

I'm seriously at a loss here.

Before I start, don't chime in with something like "Overall residual risk evaluation is the point where residual risk is viewed from a broad perspective."
Please just don't :frust:

Scenario A
The first analysis lends itself to modularity, and so I've broken it up into about a dozen sections. Each section has its own set of hazard identifications, estimations, controls, and residual risk assessments. All of the hazards in each section have been mitigated, and the residual risks are acceptable.

So far so good.

What exactly does an examination of overall residual risk look like? Its not like I have any new hazards to add.

My thought was to take the most severe risk from each section, put the 12 of them onto one risk table, and demonstrate that all of them fall within the acceptable zone.

That seems a little pointless, since its just a repetition of information from the lower-level analyses, but I'm honestly not sure what further steps are expected.

Scenario B
Assuming my plan for scenario A makes sense, what does "overall risk" mean in a smaller scenario when all of my hazards are considered in one risk study?

Any clarity you guys can give me on this subject is greatly appreciated.

Thanks.
 

yodon

Leader
Super Moderator
I was just involved in a discussion where we talked about this. Indeed, it's not easy to nail down.

The example that came up in the discussion was good for the purpose of understanding the concept. It was a cartoon with many strips of duct tape covering small cracks all over the fuselage of an airplane. Individually, each risk was (arguably) sufficiently controlled but the overall risk would be quite bad.

This followed with a good discussion of AN approach to making it a bit more tangible through the use of a heat map. If you're using the standard red/yellow/green matrix, you can plot all the residual risks in the cells. If you're clustered down in the green, you may have an argument for a pretty safe product. If there's a lot of cluster near the red, maybe a concern.

I tried this out on one simple project and it worked pretty well! I did have some clustering in the yellow but a review of those risks showed that it was endemic to the device type and not to the device itself (i.e., all such devices have the same risk ... and the only mitigation was through labeling alone so no reduction).
 

Marcelo

Inactive Registered Visitor
f you're using the standard red/yellow/green matrix, you can plot all the residual risks in the cells. If you're clustered down in the green, you may have an argument for a pretty safe product. If there's a lot of cluster near the red, maybe a concern.

Unfortunately, this make no sense for the same reason you mentioned.

Even if a risk matrix could be used to verify risk acceptability (it cannot, it's only a risk ranking tool), having a plot of risks in a "zone"only means that the risk are "on the same level" (even this is not true) when compared to each other. It says nothing about the overall risk.
 

Marcelo

Inactive Registered Visitor
As I mentioned before, you can use some techniques, such as risk summing, if you want to perform overall risk evaluation focused on quantitatively risk management, but there's no risk summing method that is considered consensus in the literature.

Another questions is if a risk summing method is necessary.
 

yodon

Leader
Super Moderator
Unfortunately, this make no sense for the same reason you mentioned.

Even if a risk matrix could be used to verify risk acceptability (it cannot, it's only a risk ranking tool), having a plot of risks in a "zone"only means that the risk are "on the same level" (even this is not true) when compared to each other. It says nothing about the overall risk.

Hmm.. I think I'll disagree. It does show where the residual risks have accumulated. I'm not saying it's necessarily the best or only method. And clearly, it can be misused and possibly be misleading. As I noted, when I plotted it out for one of my projects, the results were quite clear and telling. If nothing, I think it's a good tool for the toolbox.

I haven't fully digested the risk summing paper you provided in the other post (thank you, by the way) but in a quick scan, it looks like they're using just this technique in at least part of the method.

Certainly more to consider and I'm glad the original poster initiated the discussion. I would hope others would weigh in with their approaches to assessing overall residual risk.
 
C

cytokinesis

Thank you both for your help. :thanx:

I've followed along the "Understanding and Applying Total Risk Summing" slides you'd linked in your other post, Marcelo.

The thing that still escapes me is, once I've calculated ELR and CLR, what am I ultimately trying to determine in doing so?

Is the point to ensure that the resulting "iso-risk" line does not fall on the unacceptable risk part of the graph?
 

Ronen E

Problem Solver
Moderator
I would hope others would weigh in with their approaches to assessing overall residual risk.

Honestly, that stage in ISO 14971 is something I never really felt comfortable with. That and the risk/benefit ratio.

Perhaps risk summing is the way to go (haven't read the paper yet), but I feel what industry needs is less process-burden in risk management, not more. Some (most?) people just won't follow if the process is too complicated / lengthy / burdensome.
 

Marcelo

Inactive Registered Visitor
Hmm.. I think I'll disagree. It does show where the residual risks have accumulated. I'm not saying it's necessarily the best or only method. And clearly, it can be misused and possibly be misleading. As I noted, when I plotted it out for one of my projects, the results were quite clear and telling. If nothing, I think it's a good tool for the toolbox.

The problem is that use the matrix alone (as always). A risk matrix does not show "accumulated" risk as you mentioned. It's not a tool for that (although you might think it shows this). You would be better using density strips plot or something like that as a heat map.

But even this does not really take into account the overall risk, as it simply is a visual way to show individual risks.
 
Last edited:

Marcelo

Inactive Registered Visitor
I did mention sometimes the risk summing example, but I don't like it either.

One of the problems with the overall risk is that it will need different acceptance criteria than the individual risks. This is superficially mentioned in ISO 14971 but due to the spread use of the risk matrix as the only "acceptance" tool, it usually is just not taken into account.

Another problem, which goes back to the "automatic" use of the risk matrix, is that risk evaluation (and acceptability) should have a strong rationale, based in technical arguments. This is easier done (and stronger), than a mathematical tool such as a risk summing or any other (and goes that way Ronen mentioned as a less-burdensome process, although it will be more burdensome in a technical way).

Something that I've been discussing the in revision of ISO 14971 is the need for better arguments for risk acceptability, see for example document An Introduction to System Safety Management in the MOD, item 6.4 Making Risks ALARP where it mentions the HSE principles for claiming that a risk is ALARP.

This is also the same type of discussing regarding the risk/benefit analysis, which is right now a "get out of the jail" free card in ISO 14971, but should in fact also be part of risk evaluation (and the risk acceptability criteria).

A good way to show how this can be done is detailed in the FDA document Factors to Consider Regarding Benefit-Risk in Medical Device Product Availability, Compliance, and Enforcement Decisions. Again here, the analysis has nothing to do with estimating probability and severity (although it uses the ones estimated as factors), but focus on technical arguments.

In fact, I've been using these arguments from the FDA doc as a basis for overall residual risk evaluation for my clients (it's not exactly that, but it does make it more clear to the clients which insists on using risk matrices only that it's not just that).
 
Last edited:
Top Bottom