Risks which must be Distinctly Identified - Harm, Hazard, Severity

[rothlis]

Sorry for the length of this post. I would like to enlist the expertise on this forum to help define the proper extent to which risks should be distinctly identified. First, let me lay the ground work by defining the elements of risk:

1. Harm: Physical injury or damage to the health of people, or damage to the property or the environment.
2. Hazard: Potential source of harm
3. Severity: Measure of the possible consequences of a hazard
4. Risk: Combination of the probability of occurrence of harm and the severity of that harm

These definitions are straight from 14971. I think that the definition of severity can lead to confusion because “consequences of a hazard” implies that severity is a measure of multiple consequences. However, risk is defined as applying to a single harm. This is further muddied by the fact that harm is often dictated not only by the hazardous situation, but by the duration of the hazardous situation and the vulnerability of the affected person, both of which could be described as a widely variable continuum. For example, let's take the common hazard of electric shock. Clearly the harm can be time dependent. A microsecond of exposure may not cause injury, but several seconds of exposure likely will and can be deadly. Furthermore, the harm is dependent on the person. A patient with a weak heart is more likely to go into cardiac arrest than one with a strong heart.

This ambiguity leads evaluators to have to choose between a few different techniques:

1. Define harm in a generic sense (e.g., as “electric shock” instead of cardiac arrest, severe burn, minor burn, etc…) and then assign probability as the likelihood of the hazardous situation.
   • The risk is difficult to assess due to ambiguity in the harm. This also potentially omits a mitigation that would have been necessary if each harm had been considered individually.
2. Define harm to be the most likely consequence of the hazard and assign probability accordingly.
   • This ignores the worst case outcome. Maybe this is OK if that outcome is highly improbable, but you won’t know unless you consider each outcome.
3. Define harm to be the worst case outcome and assign probability only for that harm.
   • The most probable harm may not be properly addressed.
4. Define harm to be the worst case outcome and assign probability as the likelihood of the hazardous situation.
   • The resulting risk may not represent an actual scenario and may force risk control measures that aren’t actually necessary.
5. Exhaustively cover all possible harms that could result from the hazard.
   • All possible scenarios are properly addressed but the resulting analysis may challenge War and Peace for length which, in turn, puts everybody in a foul mood.

In my view, #5 is the most complete and proper interpretation of 14971. This comes from the fact that risk is defined as applying to a single harm. 14971 also emphasizes that the probability of a hazard may not equal the probability of harm (D.2.2.1), and that probability needs to consider the level or extent of exposure (D.3.2.2). As a whole, I think this points toward uniquely identifying every possible harm with unique consideration for the conditions of the exposure (i.e., duration) and the people involved (i.e., vulnerability). Even so, I suspect that the majority of analyses are not doing this.

So my questions are:

1. Which technique do you think is most correct? If not one of these five, then what?
2. Which technique most closely resembles your current practice and how has that been viewed by auditors?

I'll also note that there is a similar question previously posted (topic 51897) but it did not go into as much detail and it does not appear that an obvious conclusion was reached. I'm hopeful that the collective expertise here can bring clarity and establish a consensus on this topic. Thanks.

 

[kgott]

Rothlis; I agree that there is some variability in the way these things are defined, the context and application. You obvoisuly have a detailed understanding of the subject but sometimes these things are not an exact science and they are there to give us some rules and approaches to assess and manage risk in a reasonably consistent way.

Like many things in this imperfect world; apply it to one scenario or set of cirumstances and it works, apply it to another and it dosen't.

There are many issues with managing safety and risk in the work place, this is just one of them.

 

[Lucasmf]

In my experience at several medical device companies we follow #5 but with the qualifier that I usually "call it good" if we have a line item for the most probable harm and the a second line for the 'worst case' harm. Each of these lines then has their own occurrence estimate. Your risk assessment then being the product of severity x occurrence will lead you to take the appropriate risk control measures either on the more common harm or the worst case harm as appropriate.

But as the old saying goes "you can make any hazard lead to death" so a well calibrated team doing the evaluation is very helpful.

 

[rothlis]

have a line item for the most probable harm and the a second line for the 'worst case' harm

I like this approach. This seems like a reasonable to way put some bounds on the analysis.

But as the old saying goes "you can make any hazard lead to death"

How do you avoid this? Is it reasonable to have a rule that a risk isn't included in the analysis if you determine that the initial probability (before risk control) is already at the lowest level?

 

[Lucasmf]

Your risk analysis process per 14971 should always include an assessment prior to risk control and a re-assessment after risk control. "Acceptable" risks pre-risk control should not require further action.

To avoid everything leading to death you can have a rule that any event with a probability of occurrence less than "x" can be ignored. The x should be appropriate to your product and industry. For example in an airplane a one a million chance of death occurring may be significant but in a surgical procedure a one in a million chance of death is in the "noise" and can therefore be ignored.

Hope that helps.

 

[Peter Selvey]

The problem is that there is a infinite number of hazardous situations and consequences that could be analyzed. One of the most important advances in risk management will be establishment of normative methods which allow the analysis to be focused where it can be effective.

Unfortunately, there is no such flexibility in the current 14971. Of course most auditors and regulators are happy to apply some flexibility, but there is no useful guidance how to focus the analysis and hence opinions vary widely.

It has been suggested that you can eliminate events with a probability of X to simplify the analysis, but this still grossly underestimates the sheer number of "risk controls" taking place in everyday design and production, literally 100,000's of individual actions taken to reduce risk. The probabilities leading to these risk controls would be well above any reasonable value of X.

If there are so many risk controls, and most of them are undocumented, how come medical devices don't harm people left, right and center?

The answer is evolution. Most of the underlying technology used in modern medical devices has evolved to a point where it is extremely reliable and well controlled. It is often the same technology in our TVs, iPhones, PCs etc, which need this high reliability to be economic. Plus a lot of medical devices have been around for a long time, to the point where a lot of risk controls are taken as read.

That gives us a first hint at one area where risk management should be focused: new areas of the device. It's not the only area that warrants attention, but a good start.

If fact, understanding why things go right most of the time is the best way to identify areas that might go wrong.

The need to simplify applies equally to the consequences as it does the selection of hazardous situations. That simplification will depend greatly on the risk control. A production test, inherent design or strong protection system that all but eliminates the problem means that there is little benefit in analyzing the consequences. On the other hand, a slow responding or inaccurate protection system might control some consequences and not others; the analysis would then have to be separated depending on the type and nature of harm.

The idea that you need to start the analysis without risk controls in place is misguided, and legally unsound (a hypothetical object that has no risk controls is not a "medical device" because it would never be placed on the market, and therefore does not need to be analyzed ... it's a dead end). Rather, the only legal requirement is the analyze the final device that is placed on the market, with all risk controls in place, and show that the risk is acceptable.

 

[Roland chung]

The idea that you need to start the analysis without risk controls in place is misguided, and legally unsound (a hypothetical object that has no risk controls is not a "medical device" because it would never be placed on the market, and therefore does not need to be analyzed ... it's a dead end). Rather, the only legal requirement is the analyze the final device that is placed on the market, with all risk controls in place, and show that the risk is acceptable.

If starting the analysis with all risk controls in place, we would not find a risk needs to be reduced. That means the common analysis matrix table is wrong. What is the correct way?

 

[Peter Selvey]

This is just predicting the future, not now of course.

But I would expect that in the future risk management would require the manufacturer to select a much smaller number of "significant" situations which then would require much more detailed analysis than is provided now. There may or may not be risk controls; that's not the point.

One of the key triggers whether a situation is "significant" might be whether or not it's obvious by inspection. Consider the following two cases:

Case 1: Production test of a patient monitor temp function, using dummy resistors representing 25.00, 35.00 and 45.00 degrees (from YSI 400 series), and the monitor must read these values exactly (i.e. 25.0, 35.0, 45.0).

Case 2: rinsing of a endoscope to remove production greases, performed at 22-28degC, 10 minutes, with Type X solvent, flow rate Y etc; solvent changed once a month.

Both are important production risk controls.

For Case 1, a qualified engineer and auditor should know it's an effective and expected risk control. I don't think it's worth putting this kind of thing in the risk management file; again keeping in mind there are 1000's of similar risk controls in design and production and we can't document them all.

For Case 2: here, it's not obvious by inspection the risk control is effective. Research is required to know what a reasonable limit for residuals grease is; there would need to be tests done to show at the worst case conditions the greases are removed. I've been an auditor and asked questions like - why did you choose 10 minutes, and some old guy pulls out some scrappy document from 10 years ago showing tests for residuals at different flow rates / temp etc. There's a lot of characteristics here that are important to document not only to prove it's OK in the first place, but also to have on file in case of design changes in the future. Perfect for storing in the risk management file.

There are other reasons a situation might be significant, for example, borderline cases; cases where conflicting requirements means the risk can't be eliminated and so on.

The important point is that these must be covered in something more than a single line in a traceability matrix. And if so, the number of items analyzed must be vastly reduced.