IVD Device Software - Risk Classification

ThatSinc

Quite Involved in Discussions
Hi All,

currently looking at IVD Device software for an image processing system
The processing equipment creates a digital image of the sample at ultra high resolution for review by clinician to aid in the diagnostic process.

The equipment itself does not provide any results for +ve or -ve, and the image is not the sole piece of information used in the diagnostic process.

Software safety classification is somewhat out of my wheelhouse, so the logic of whether something has the potential to cause serious injury or non-serious injury vs. whether the risk is acceptable is slightly confusing to me in risk management context.

Is the logic that if the control measures exist in (non-independent) software alone and the risk reduction is considered acceptable (in accordance with 14971 criteria) but if you were to ignore the software control measures it would be unacceptable then it's considered class B/C (in accordance with 62304)?


If there was a misdiagnosis as a result of software failure causing image corruption, it could cause serious injury, but in reviewing the risks involved the probability of harm occurring is negligible based on the standard clinical process for verification of diagnosis through multiple clinical markers prior to treatment or non-treatment.

As such, based on these control measures it could be considered Class A?

Thanks for any guidance,

TS.
 

Tidge

Trusted Information Resource
This reply is going to be full of personal opinions, motivated by long experience and a deep awareness that folks have been pushing for computerized image processing to help in diagnosis for more that 30 years. I'm not even going to pretend to derive an answer from "first principles." While I am comfortable speaking of risk in the context of 62304 development, the SSSC also serves as "how seriously" to take the development effort.

From the risk PoV: Barring legislative action (or a firmly established common basis of care)(*1) in a jurisdiction, I don't think SSSC Class C is warranted as long as the device marketer is making it clear that the diagnostic technique is NOT directing treatment. As soon as a knife (or chemical, or biologic, or radiation, or any therapy) is directed at a patient because of the diagnostic tool, *I* would want to know that I've taken the development of the diagnostic tool as seriously as possible.

(*1) I mean to imply that a jurisdiction would require (via laws or regulations) that the device (effectively) be treated as (akin to) Class C. See "Blood Management Software" in the USA. Also there is a possibility that a diagnostic tool is so commonly used (in medical practice) that it becomes the de facto determining factor in treatment.

From the effort PoV: Class A development is (by design) practically equivalent to "it looks like it works" without requiring as much evidence that the development team knows how it works. This is overly simplistic, but keep in mind a "diagnostic" software package that is Class A could be nothing more than a relational database (or spreadsheet!) with some indices for filtering based on symptoms. the past multiple decades have tried for more than this (neural networks for mammography image analysis were all the rage in the 1990s, seems like it comes back every six years). As long as CNTRL+F works, the software will return something.

Personally: If SSSC class C was not warranted (or otherwise unpalatable to MWER, see *1) and I didn't otherwise have a clear category based on precedent, I lean towards class B for a few reasons:
  • If you develop per Class A, and an authority feels it is Class B or C (at the time of submission, or at a later time) it is VERY difficult to generate the artifacts for class B or C. Going from B to C is much less effort.
  • Class A can be interpreted (right or wrong) as "rationalizing away" risk considerations... this has a little bit of the look of a chicken/egg situation... if we don't do the analysis (if not required by 62304 because of Class A) how do we know we did it correctly?
  • Class A doesn't worry about SOUP contributing to hazardous situations... which often leads to development teams not always taking SOUP seriously, even if it is managed per the configuration management process. (for example, Class A doesn't require a formal relationship between architecture and requirements!)
Finally... and I know this sounds like chicken little... it's only a matter of time before a developer/vendor of medical device software runs into someone (it could be an elected chairman of a legislative committee) who bluntly, without nuance, equates cybersecurity with medical device safety. When this happens, appeals to consensus standards are unlikely to sway opinion. A Class B development approach will allow for a firmer basis for refuting criticisms, including those that can come from the direction of cybersecurity.
 

Hi_Its_Matt

Involved In Discussions
@Tidge raises good points, and from a project and regulatory delay risk perspective, following processes for Class B is certainly the safest thing to do.
But I do think, based on your description of the system and its intended use, that Class A may be the right determination, depending on how your organization has defined acceptable/unacceptable risk. (Of course, acceptable/unacceptable should be based on state of the art and clinical benefit, not just "whatever management feels is tolerable/desired.")

The standard says to evaluate risk control measures external to the software system, and Note 1 further clarifies that this can include health care procedures. So if your intended use, labeling, marketing claims, etc. make it clear that the image(s) should be used within the broader context of a patient's symptoms and clinical history, and the images shouldn't be relied on as the sole piece of information for making significant clinical decisions, AND you can support that this is how the system is used in real practice, that then that would go pretty far in claiming that the risk of any software failures is low/acceptable.


IVD Device Software - Risk Classification

NOTE 1 External RISK CONTROL measures can be hardware, an independent SOFTWARE SYSTEM, health care procedures, or other means to minimize that software can contribute to a HAZARDOUS SITUATION.

Disclaimer: I have never been in the position where I had to defend to a regulatory agency our software safety classification determination. So someone with that experience may disagree with my assessment.

Side question: @Tidge I have seen you use the acronym SSSC before. I'm assuming that's Software Safety Classification? What is the extra S for?
 

Tidge

Trusted Information Resource


Side question: [USER=319708]@Tidge
I have seen you use the acronym SSSC before. I'm assuming that's Software Safety Classification? What is the extra S for?

I never work with SaMD (software divorced from any hardware) so throw in an extra "S": Software System Safety Classification. It's a habit I got into a long time ago because for the medical devices I work with, it is neither practical nor appropriate to separate software from the rest of the system when doing risk analysis. There are all sorts of dimensions of design/risk analysis that feed into my preference; generally I find it more straightforward to treat the software system as having a safety classification then it is to waste too much effort trying to subdivide design elements/risk controls between "code" and everything else (for example: memory).
 

Hi_Its_Matt

Involved In Discussions
Well considering that "software system" is mentioned in the standard about 200 times, I feel a little silly now. Thanks for clarifying.
 

ThatSinc

Quite Involved in Discussions
(Of course, acceptable/unacceptable should be based on state of the art and clinical benefit, not just "whatever management feels is tolerable/desired.")

If only management would listen when I advise that the risk acceptability matrix *can* and *should probably* change depending on the type of device being assessed taking into consideration the preliminary CER data that shows what state of the art is with regards to that type of device and intended use.
Nope, the matrix is the matrix and it's red in one corner, green in the other and yellow across the middle, if it's red then it's bad and we've gotta do something about it, if it's yellow we kinda say some waffly words to justify why we're not doing more and if it's green we get frustrated that we even had to assess it in the first place - stays the same whether they're making a sticking plaster or an automated surgical robot.


you can support that this is how the system is used in real practice,

In this case, yes, it can be supported.
For so many devices I've worked with the intended use and limitations are so far removed it's scary.

---

You've brought up something else that's a slight difference between 62304 and the FDA's LOC for software - that's the post-mitigations factor.

The FDA recommend (as per the current guidance document from 11th May 2005, and the draft document set to replace this) that all assessment of whether a software failure can cause serious injury is taken before control measures are implemented.

Here I see a chicken and egg situation with regards to defining whether a design output is a risk control measure for a potential software related failure hazard, or whether the design output in order to satisfy a design input requirement results in a software failure not being able to cause a hazard.

e.g. device with a heating element that's intended to provide low levels of heating, with temperature controlled by software, requirement is to provide heat up to 30C.
If the heater element selected is 2000W, and the software controlling the temperature fails - this could cause serious injury.
If the heater element selected is 200W, and the software controlling the temperature fails - this could not cause serious injury.

To perform as required, the device only requires a 200W heater element.

In the 14971 risk process, the device has an output of heat - so you would assess the risk of burns through skin contact with the heat source, and you would document that you've controlled this through selection of a 200W heater element.

However, for documenting the software LOC, how can you make an assessment pre-mitigation?
 

Tidge

Trusted Information Resource
You've brought up something else that's a slight difference between 62304 and the FDA's LOC for software - that's the post-mitigations factor.

(snip)

Here I see a chicken and egg situation with regards to defining whether a design output is a risk control measure for a potential software related failure hazard, or whether the design output in order to satisfy a design input requirement results in a software failure not being able to cause a hazard.

(snip)

However, for documenting the software LOC, how can you make an assessment pre-mitigation?

My attitude is that the FDA wants evidence that the designers have thought about the big picture, and doesn't necessarily need to see that the designer has over-thought the big picture. What follows is based on the minor/moderate/major way of thinking.

In the case of a medical device that happens to have both hazards (e.g. extreme heat... that is, something that could be moderate or major LoC pre-mitigation) and software, the diligent approach as requested by the FDA would be to minimally provide an architecture document that makes it obvious what role the software plays in, or is allocated to, the (possible) presentation of the hazard. The submission can of course try to side-step blatant presentation of this sort of allocation (e.g. "Any idiot can see that this software has nothing to do with any hazard!"), but why make it harder on the reviewers? With an architecture that clearly allocates roles and responsibilities of the elements of the device, it ought to be possible to correctly assert (in appropriate cases) that the software is neither controlling, nor mitigating, nor contributing to the risks that would otherwise drive a moderate or major LoC. This circumstance ought to be obvious from the system level Hazard Analysis.

As I see it, the FDA LoC is essentially just another gating point that (ultimately) serves as a "red-face" cross-check against a 62304-inspired SSSC of A/B/C. In the case of the imaginary heater: suppose the software was doing nothing but running a clock display, and it neither controlled nor monitored the heating element. It might be tempting (and even justified!) to lean towards a LoC of Minor, but if it turned out that the medical device was used to warm premature infants, and the clock on the heater was the mechanism used by attendants to adjust the treatment regimen for neonates... it might be important enough (from the risk profile) to initially consider a higher level-of-concern for the device. If all of the software units involved with the clock later get categorized as 62304 class A, all I would suggest as "extra work" would be a software hazard analysis that makes it obvious where the risk controls actually are, if not in software.
 

ThatSinc

Quite Involved in Discussions
In the case of a medical device that happens to have both hazards (e.g. extreme heat... that is, something that could be moderate or major LoC pre-mitigation) and software, the diligent approach as requested by the FDA would be to minimally provide an architecture document that makes it obvious what role the software plays in, or is allocated to, the (possible) presentation of the hazard.

So the hazard of heat being controlled prior to any consideration of the software function of the device, to limit the amount of heat able to be physically generated, so that any failure of software in controlling the output of the heater couldn't result in a hazard would be logical to accept as "pre-controls".
The limiting of the heat source hasn't been determined as an additional control measure in case of software failure, but as a control measure "in general".

In the case of the imaginary heater: suppose the software was doing nothing but running a clock display, and it neither controlled nor monitored the heating element. It might be tempting (and even justified!) to lean towards a LoC of Minor, but if it turned out that the medical device was used to warm premature infants, and the clock on the heater was the mechanism used by attendants to adjust the treatment regimen for neonates... it might be important enough (from the risk profile) to initially consider a higher level-of-concern for the device.

In this scenario, either if the clock was intended to be used for that purpose or it was reasonably foreseeable that it would be used in this manner, I would entirely agree.
 

Hi_Its_Matt

Involved In Discussions
Nope, the matrix is the matrix and... it stays the same whether they're making a sticking plaster or an automated surgical robot.
Ooof. My deepest sympathies.
That is borderline non-compliant with the idea in 14971 that the criteria for risk acceptability take into account the generally acknowledged state of the art. Nobody in their right mind would accept the same level of risks from a "nice to have" medical device than they would from a life-saving implant. But anyways...

This is where the idea of "pre-risk control" or "pre-mitigation" analysis is so difficult for me (and I'm sure many others).
You have to consider, to a certain extent, some aspects of the design when evaluating risks "pre-mitigation."
For example, there are certain types of light bulbs that get hot enough to burn someone, if they were to touch it. But you could run a single 100mW LED for 5 years and it wouldn't ever get hot enough to burn someone. If a design has low-output LEDs, I simply wouldn't include "thermal burn" as a potential harm. And I certainly wouldn't list the use of LEDs as a risk control.

Back to your heater example, if the heating element of a 200W heater can never physically get hot enough to cause serious injury, then failure of software controlling the temperature of that heading element simply won't result in a hazardous situation in which a serious injury could occur! From a software perspective, the worst case scenario of failing to control the temperature is maybe "patient discomfort," instead of a more serious burn.

If there was never a reason to consider using a 2000W heater, then I wouldn't consider using a 200W heater to be a risk control either.
 

Tidge

Trusted Information Resource
I can see the general point being made, yet "Inherent By Design" is a recognized category of risk control.
 
Top Bottom