Risk Assessment according to ISO 14971 - Medical Device Software

2

20110517dpr

Hello Everybody :bigwave:

I have some questions related to risk assessment for software used in medical devices and would be glad to get advices from experts.

Let's assume that I have a physiological monitoring device, driven by firmware and software components. A possible hazard for that the device is not showing correct values, due e.g. to a software bug. This bug may occur in some rare pathological cases, assume 1 people out of 10,000.

The likelihood that the hazard "incorrect values shown" becomes a hazard situation is hence 1/10,000.

Now the probability that this hazard situation becomes a harm might be quite different. For instance, the software can fail to display correct result as soon as this pathological case occurs. Should the medical decision relies exclusively on the results shown, it would lead to a probability of harm of 1.

My understanding of ISO 14971 is that we should consider the likelihood that a hazard situation becomes a harm; and not the whole chain. In the first case, the probability that the harm occurs is 1 (if it happens that the patient has the pathological case); in the second case it is 1/10,000. Which is a hell of difference!

Am I right, or am I missing something?

Furthermore, the probability that the hazard situation becomes a harm might depend on the country where the device is operated. For instance, in some countries, where the users are highly trained, they shall not trust the results, hence lowering the likelihood. In some other countries, the likelihood shall be much higher.

I understand that I have to take the entire clientèle into account; that is if I have different likelihood depending on the device location; I should take the highest one for the risk assessment (that would make sense). Is my understanding correct? Or is it really meant that I should take care of the ratio of device this country has (e.g. if only 10% my devices are in that country, then I should multiply the likelihood of harm by 0.1) ? The latter seems suspicious to me.

Any advices would be greatly appreciated!

Thanks in advance,
/lew
 

somashekar

Leader
Admin
Re: Risk assement accordingly to ISO 14971

<<< I understand that I have to take the entire clientèle into account; that is if I have different likelihood depending on the device location; I should take the highest one for the risk assessment (that would make sense). Is my understanding correct? Or is it really meant that I should take care of the ratio of device this country has (e.g. if only 10% my devices are in that country, then I should multiply the likelihood of harm by 0.1) ? The latter seems suspicious to me. >>>

Too lenghty, but I guess you prefer not to share the exact cause and effects. However keeping in mind the human behaviours irrespective of the country and going about the Risk assessment is ideal. Please also note that the Risk assessment document is dynamic and can always be improved as a response to situations. Going further the control measure in your case may be a user communication appropriately, thus ensuring a closed loop.
Good luck ....
 
D

danpa

Lewis,
While I am not an expert on risk mgmt, my opinion is that alot depends on how you structure your risk management study. I prefer to look at software as part of the overall system, and it is the system that we do risk management on (Software alone generally can not cause the harm, it must be part of a system with physical interfaces to cause harm).
As such, I would look at the probability of the system causing the harm and not assume "1" as the probability.
As a side note, I am always suspect of hard quantitative numbers like 1/10,000 when conducting risk management for software. I prefer qualitative terms such as Frequent, Probable, Remote, Improbable.
Small differences in the hard numbers can have huge differences in final risk assessment and the hard numbers are very difficult to quantify for software system failures. We know that the software will always behave the same way with the same set of conditions, but determining how often a certain set of conditions will exist is often impossible.
I have the same problem with various countries using the products differently and have not come up with good guidance for how to account for this in the risk mgmt assessment.
 

Marcelo

Inactive Registered Visitor
My understanding of ISO 14971 is that we should consider the likelihood that a hazard situation becomes a harm;


Did you see the second edition of ISo 14971? It explains some problematic issues regarding hazards and hazards situation (for example Figure E.1 - Pictorial representation of the relationship of hazard, sequence of events, hazardous situation and harm). There it can be seen that you do not only have to take into account the likelyhood that a hazard situation turn into harm, but also the likelihood that the hazard, after a sequence of events, turn into a hazardous situation. So there´s in fact two probablities of ocurrence of harm.

Also, you´re corret when you say that you have to use the highest likelihood depending on the country. In fact i would say more, you have to take into consideration the accepted culltural values of the country/population/etc. into your risk acceptability policy and risk analysis, so this is in fact much broader than what you said.
 
2

20110517dpr

Gidday,

Thanks mmantunes, danpa and somashekar for your answers!

As a side note, I am always suspect of hard quantitative numbers like 1/10,000 when conducting risk management for software. I prefer qualitative terms such as Frequent, Probable, Remote, Improbable.
Absolutely right. Actually, I am using hard figures to make the example somewhat more palpable for the reasoning.

There it can be seen that you do not only have to take into account the likelyhood that a hazard situation turn into harm, but also the likelihood that the hazard, after a sequence of events, turn into a hazardous situation. So there´s in fact two probablities of ocurrence of harm.
Yes, and in fact I think I believe that I know where the misunderstanding is coming from.

If I assess the risk that, for a given patient, the hazard turns into harm, it is indeed 1/10,000. But if I assess the risk that, during the life of my product, the hazard turns into harm for at least one patient, then the likelihood is very high (given that we have something like 1,000 operations per year and device ).

Of course, the risk management policy in place defines what means terms like "likely" etc. So If I refer to that policy, there is no ambiguity at all.

However, I have a last question. When speaking about risk's likelihood does the standard mean: "the likelihood that the hazard turns into harm for a given patient" or "the likelihood that the hazard turns into harm for at least one patient during the foreseen product's life". Or is it left to the risk management policy to define what is meant exactly?

TIA,
/lew.
 
Last edited by a moderator:
G

gholland

"However, I have a last question. When speaking about risk's likelihood does the standard mean: "the likelihood that the hazard turns into harm for a given patient" or "the likelihood that the hazard turns into harm for at least one patient during the foreseen product's life". Or is it left to the risk management policy to define what is meant exactly?"


Reading ISO14971, Annex E it is stated that the guidance is to consider both when determining 'Probability of occurence of harm (Figure E.1)'. The probability you're hunting for is basically the probability of the Hazard occurring (P1) multiplied by the probability of the Hazardous situation (P2).

In the 'real world' you'd have to come up with some sort of 'Frequent, Probable,....' ranking and be able to defend it. At that point you can go to your risk chart (Chart D.7 in Annex D for example) and see where you stand. If you have a frequent 'P1' but a remote 'P2' then your probability of the risk occuring may be Remote and you may be able to defend it to yourself and to your auditors. I would definitely get management buy-in as to your Probability of Occurence' ranking and heavily document the rationale if your failure mode is lethal.

:2cents:
 

Marcelo

Inactive Registered Visitor
However, I have a last question. When speaking about risk's likelihood does the standard mean: "the likelihood that the hazard turns into harm for a given patient" or "the likelihood that the hazard turns into harm for at least one patient during the foreseen product's life". Or is it left to the risk management policy to define what is meant exactly?

It´s up for the policy, meaning, the manufacturer. This directly impacts the risk acceptability criteria (in fact it´s one of it´s components) so the standard does not have a saying on this.
 
Top Bottom