ISO 14971 and IEC 62304 - Medical Device Software House

[KoD_RP]

Hi,

I am working at a software house, ISO:13485 certified, that develops software for medical devices.
The typical business flow is the following: the customer(device manufacturer) provides the product specifications. We develop the software according to ISO:13485, conduct verification testing and then deliver the software to the customer in order to perform system testing and validation (in most of the cases, the customer just provides a simulation interface to the medical device for intercommunication).

We have to alter our QS in order to comply with ISO 14971 and IEC 62304.

I would appreciate your comments on the following issues:

Risk Analysis/Severity: When we identify software hazards, we narrow them just on the software expected behavior e.g. our software calculates the required volume that should be transferred by a pipettor from position A to position B. In case our calculation algorithm is wrong, we will pass wrong data to the underlying instrument interface.
The severity of this risk depends on the point of view: if we focus just on the software, this is a major risk since the final outcome is hypothetically endangered (wrong transfer step). I say hypothetically, because the underlying instrument interface might have already mechanisms to identify such errors (therefore the same issue, doesn't have the same level of severity).
Question 1: How shall we calculate severity in this case? Shall we focus just on the software behaviour and ingore all sequencial components?
Question 2: Typically, the overall medical device solution we develop is not life threating (it doesn't produces results) nor causes injures. Wherever we have looked the severity is mainly related to human factor. Is there an alternative way, acceptable by 14971, to define severity scales closer to our needs?

Risk Analysis/Probability: According to IEC 62304, the probability of all software failures should be considered 100%. Based on this, the overall risk remains the same, regardless of the control measures taken (the severity doesn't change). Therefore, at the end of the risk management process, we will not be able to reduce the risks to acceptable levels (please keep in mind that I am referring just on the software part).
Question 3: Is this something acceptable according to 14971 and 62304? If yes, what do we have to provide to the customer as documentation?

Risk Management:
Question 4: Do we have to conduct a risk/benefit analysis or this is a task of the medical device manufacturer since he has the overall information? Are there any other risk management steps that we might be skipping due to our particular role on the overall project?

Thanks a lot

 

[Ronen E]

A1: Your risk management scope is the software, not the finished device, so your severity should be ranked in terms of software outcome. Your client might take that output and utilise it in their risk management process where severity will be evaluated based on the finished device outcome.

A2: ISO 14971 does not dictate any specific severity ranking (harm to patient or otherwise). It is your responsibility to establish the severity scale and follow it consistently.

A4: The risk/benefit analysis makes sense at the finished device level, where the clinical benefit is realised. If what drives your ISO 14971 compliance is a client requirement (directly or via another standard) I would have tried to argue that there's no point in a risk/benefit analysis (as ISO 14971 intends it) at the component level. Generally ISO 14971 was intended for finished devices so when attempts are made to utilise it at the components level funny things might happen.

 

[KoD_RP]

Thanks for your feedback Ronen. It's more clear now!

Could anyone help me on question 3?

 

[yodon]

A1: Your risk management scope is the software, not the finished device, so your severity should be ranked in terms of software outcome.

I think I may want to take exception to this statement. To me, risk has to be analyzed within the scope of the system; otherwise, the effects can be interpreted quite differently. For (an extreme) example, say the software is controlling a defibrillator. Without considering system scope, one could argue that if the software crashes, you just need to restart it (an "inconvenience"). In the scope of the system, though, if the software crashes and delays delivery of therapy, it could be catastrophic.

I would think that the software risk analysis would be conducted under the auspices of the client's overall risk management approach, using their ratings systems. And, thus, regarding question 3, the software analysis folds in nicely and (presuming the client's RM process is compliant to 14971) would meet the client's needs for compliance.

To be fair, Ronen did say:

Your client might take that output and utilise it in their risk management process where severity will be evaluated based on the finished device outcome.

But I think partnering with the client to do the work in the context of the system and aligned with their procedures would be most efficient and effective.

 

[Marcelo]

According to IEC 62304, the probability of all software failures should be considered 100%. Based on this, the overall risk remains the same,

The probability of the software failure is 100 %, which is an event in the sequence of events that leads to the hazardous situation (part of P1). This does not mean that the probability P is 100 %.

 

[Ronen E]

I think I may want to take exception to this statement. To me, risk has to be analyzed within the scope of the system; otherwise, the effects can be interpreted quite differently. For (an extreme) example, say the software is controlling a defibrillator. Without considering system scope, one could argue that if the software crashes, you just need to restart it (an "inconvenience"). In the scope of the system, though, if the software crashes and delays delivery of therapy, it could be catastrophic.

I would think that the software risk analysis would be conducted under the auspices of the client's overall risk management approach, using their ratings systems. And, thus, regarding question 3, the software analysis folds in nicely and (presuming the client's RM process is compliant to 14971) would meet the client's needs for compliance.

To be fair, Ronen did say:

Your client might take that output and utilise it in their risk management process where severity will be evaluated based on the finished device outcome.

But I think partnering with the client to do the work in the context of the system and aligned with their procedures would be most efficient and effective.

I agree that collaboration is the best approach. In that sense I think that there's little point in conducting a full-blown ISO-14971-style RM process at the component supplier level. At that level a bare bones FMEA or the like would be appropriate, and that could feed into the finished device RM process (conducted at the finished device manufacturer), through the participation of the component supplier representative(s), playing the expert role. I think that the pacemaker example drives that exact argument home. That example is an extreme and obvious one, but in less obvious situations there's no way a component supplier could correctly judge the severity at the system level - most of the time they wouldn't even have acces to the relevant information.

The problem is that collaboration of that kind is not always offered or wanted by the client, and as a supplier you can't force it.

In general, I think that this discussion demonstartes the futility of taking standards that were created for finished devices and strictly applying them at the ingredient level. Regardless, some clients require it so sometimes there's no choice other than trying to come up with a reasonable response.

 

[kreid]

I agree with Ronen's first response.
For question 3 the software failures can be mitigated to some extent by using good processes. For functional failures, these should be mitigated at the system/device level. This could be managed by you delivering your risk management file to your customer. The customer might also include some risk criteria in their production spec to you so that they might influence the design you implement.

 

[KoD_RP]

@Ronen:

The problem is that collaboration of that kind is not always offered or wanted by the client, and as a supplier you can't force it.

In general, I think that this discussion demonstrates the futility of taking standards that were created for finished devices and strictly applying them at the ingredient level. Regardless, some clients require it so sometimes there's no choice other than trying to come up with a reasonable response.

I couldn't agree more. In most of the cases you get a specification's document without really getting involved with the actual medical process. Therefore, you try to do your best to comply with a standard and explain to the customer why some standard requirements are not required in this case (not always easy to convince).

@Marcelo:

The probability of the software failure is 100 %, which is an event in the sequence of events that leads to the hazardous situation (part of P1). This does not mean that the probability P is 100 %.

Typically, we have to deliver to the customer a risk management file containing hazardous situations related to the software part e.g. imagine a software calculating pipetting sequences for a pipettor. A hazardous situation is that the pipetting sequence created by the software is wrong. This could eventually lead to wrong test preparation. The probability is P1 + P2 + .. + Pn (P1: our software fails), (P2: customers device software fails to handle the error) (P3: customers device hardware fails to handle the error).
On our case, we have no knowledge of P2 - Pn (not even how many sequential components are involved in this sequence). Thus, unfortunately our risk analysis is only based on P1.

@kreid:

For question 3 the software failures can be mitigated to some extent by using good processes. For functional failures, these should be mitigated at the system/device level. This could be managed by you delivering your risk management file to your customer. The customer might also include some risk criteria in their production spec to you so that they might influence the design you implement.

Thanks for the feedback. As you can understand, it's not always easy to get such information from the customer as Ronen argued.

Point of view
As a component partner, I would expect that the following flow should cover the needs of a customer, concerning 14971, given our role on the overall project:

• We identify hazardous situations that our software could trigger.
• We estimate the severity (having always in mind the big picture - what could happen to the patient if the sequential components fail to handle the error)
• We set the probability always to 1 (since we don't know when the software will fail).
• As control measures, we aim to increase the detectability.
• As verification procedure we perform extensive testing (just on the software) to verify the detectability (the more tests, the merrier) - there is no other way to verify the effectiveness on software level -
• We define a number of successful tests per case as an indicator to reduce the probability "as much as possible".
• We calculate the residual risk and skip the risk/benefit analysis (it makes sense only to complete products)
• We create the risk management report and deliver it to the customer.
• Post-production info: Customer performs the system testing and validation and verifies the effectiveness of our control measures as well.

Wishful thinking?:

 

[Marcelo]

Typically, we have to deliver to the customer a risk management file containing hazardous situations related to the software part e.g. imagine a software calculating pipetting sequences for a pipettor. A hazardous situation is that the pipetting sequence created by the software is wrong.

This is not a hazardous situation because there's no exposure of the patient/user/etc. to harm. What is it, as I mentioned before, is part of the sequence of events that leads to a hazardous situation to the patient/user/etc. So,what you can do is to define part of the sequence of events, and then the device manufacturer can provide the final part of the sequence, the hazardous situation, harm, etc.

 

[Ronen E]

This is not a hazardous situation because there's no exposure of the patient/user/etc. to harm. What is it, as I mentioned before, is part of the sequence of events that leads to a hazardous situation to the patient/user/etc. So,what you can do is to define part of the sequence of events, and then the device manufacturer can provide the final part of the sequence, the hazardous situation, harm, etc.

...which is exactly what I meant when I wrote that there is little sense in an expectation from ingredient suppliers to meet ISO 14971 in full, on their own.