New to ISO 14971. Comparing to MIL-STD-882

bladpart

Registered
Wondering if anyone has had some experience with both medical device safety/risk management and also aerospace system safety. I spent some time as a system safety and reliability engineer in the aerospace industry. I'm pretty familiar with the process as far as MIL-STD-882, 1472, HDBK-217, 454 and a few other DoD standards. I've moved back into med device where I'd previously been more involved in quality but not risk management specifically.

In any case, I'm trying to draw parallels between the aero system safety experience I have and medical device risk management (ISO 14971). I see a lot of similar concepts and slightly different definitions, but I also feel like there are some disconnects. Curious to know if anyone else has had to move between the two fields and has any insight on what they learned or had to adapt to.

I think the big thing for me surrounds hazard analysis. I'm seeing that we have a "Hazard Analysis" that seems to be a combo of several specific types of hazard analyses. In particular I think I'm seeing a System Hazard Analysis and Operating & Support Hazard Analysis combined into one artifact. Does that sound correct? How about a Functional Hazard Analysis? Subsystem Hazard Analysis?

Next, there seems to be a huge focus/reliance on FMEAs which feels odd. FMEA is a great tool, but so is FTA, sneak circuit analysis, common cause failure analysis, interlock analysis, and more (usefulness depending on the project, or course). I'm also not seeing any FMECAs or reliability predictions. Are those not commonly performed or leveraged for hazard identification and assessment in med device?

Last, this might be unique to the company I'm with but I'm running into conceptual issues with defining a system. I'm seeing situations where we offer a "system" but not every piece of the system is required, or some of the pieces of the system could be replaced by third party products. Just a generic example off the top of my head. A full up system might have a user interface, some kind of power supply for the medical device, an ecg monitor, saline pump, a consumable catheter. In practice though some customers will have a third-party ecg monitor, or a third-party saline pump, or they may use a third-party catheter with the system. Doing a system level hazard analysis on that kind of modular system isn't something I've had experience with. In Aero, if you imagine an inertial navigation box with GPS, someone might order the same thing without GPS, but we'd be contracted to "design" that modified box, create/run a separate product development process for that which includes a separate hazard analysis on that configuration. Very straight forward. For a highly modular system though, creating HAs for each possible configuration doesn't seem like the right answer.
 

Tidge

Trusted Information Resource
Welcome to the industry! Do keep in mind that the two industries are very different, in many ways, even if you see echoes of one in the other. Start with 14971 and build from that as opposed to trying to fit another industry's standards into medical devices.

I'll speak to a few points, please keep in mind I am not attempting an explanation from "first principles", nor from a historic perspective... my words are meant to reassure you that there are answers just be aware that I may be simplifying certain things for brevity.

Last, this might be unique to the company I'm with but I'm running into conceptual issues with defining a system. I'm seeing situations where we offer a "system" but not every piece of the system is required, or some of the pieces of the system could be replaced by third party products. (snip) Doing a system level hazard analysis on that kind of modular system isn't something I've had experience with.

In medical devices, there is an obligation to make sure that the use of the device (as designed) is safe (i.e. free of unacceptable risks) to patients and users. This includes considering different combinations/configurations, and should also include consideration of reasonably foreseeable misuses. How much detail this analysis entails depends in some part of the nature of the device and the jurisdiction where it will be marketed.

Next, there seems to be a huge focus/reliance on FMEAs which feels odd. FMEA is a great tool, but so is FTA, sneak circuit analysis, common cause failure analysis, interlock analysis, and more (usefulness depending on the project, or course). I'm also not seeing any FMECAs or reliability predictions. Are those not commonly performed or leveraged for hazard identification and assessment in med device?

FMEA are commonly used as subordinate documents in a risk analysis; they don't make good top level tools for analyzing risks. There exists at least two decades worth of analysis of that topic. Simply put: analyses of failure modes (and faults) don't really analyze risks. No single tool (FMEA, FMECA, FTA, etc.) is encouraged over others... and attempts to make any of those speak to risks is usually very clumsy and only ends up satisfying one element of a mature risk analysis.

I think the FMEA favoritism is primarily driven by history and legacy. Stick around and you are very likely to see references to "Design FMEA", "Use FMEA", "Process FMEA", and even "Software FMEA". A (generally) unspoken mechanism that can be used to judge the maturity of an organization's risk files (which include FMEA) is to see if they use the term "RPN" and if so what they call it. If the word "Risk" is included in it... then they likely aren't as mature as they could be (in medical device risk analysis).

What I am writing in this paragraph may not be precisely true, but it feels true enough for me: Recently (maybe the past five years?) some critical mass of otherwise well-informed (in the subject of risk controls in medical devices) folks came to the conclusion that they weren't telling a logically consistent story with Design FMEAs, so there has been an effort to eliminate (forgive some technical jargon) "(D)etectablity ratings from the left side of the FMEA"... these analyses are still called "DFMEA" but perhaps those folks would have been better off using a different analysis tool... who is to say?
 

bladpart

Registered
Awesome! Thanks for the response on this.

I think it's fair to call out that I shouldn't be looking to fit another industry's standards into medical device. As I've had some more time to think about what I'm really trying to ask here while writing up a response, I think my bigger concern is with the current state of the process that I'm witnessing. I'm aware that the process has shortcomings, and one of our team's current objectives is to execute an improvement process for HA and RM in general. I've seen a few things that I'm concerned about and it tends to be my past experience that prompts that concern. Sorting out what is a legitimate concern vs what is simply unique to this industry is the challenge. Interestingly enough, your response coupled with an FDA session I participated in today somehow managed to tie up a few of those concerns today, whereas I feel like I've been researching/spinning wheels since August on some things.

Good to hear that it's appropriate to consider the different possible configurations. My past experience was already leading me to believe that would require analysis of each possible configuration (System with black Box A and B, system with only Box A, system with only Box B). I guess I said it didn't seem like the right answer, but that contradicted my gut feeling about it. This is a shortcoming in our current process.

Your comments on the FMEAs align exactly with what I've seen. It seems like there are quite a few minds that our group will need to change regarding them. There is definitely a lot of favoritism with them (DFMEA, UFMEA, PFMEA), and it's clear that they are/were relied on as risk documents. That was the first thing that nearly knocked me over. I mentioned that FMEAs are a great tool--as you've mentioned though, not for top level risk analysis--but for identifying hazards that might have a single fault or several faults that lead to the hazard (hazardous situation, I need to make sure I'm not using 882 definitions). I'm glad to hear someone say that no single tool is encouraged (or rather, I don't think it's appropriate to rely on just one tool). In any case, I think this helps further solidify my observations/feelings that the process isn't as mature as it could be. I've been working with individuals to try and capture the current state and what gaps are existing in the RM and HA process. Between those interviews and reviewing past artifacts, that's the picture that's starting to coalesce. Don't get me started on the issues I'm seeing with systems engineering and how requirements are written (or when they're written).

Just to give an example of what I observed with the FMEAs. The FMEA documents I've reviewed call our failure effects as hazards which baffles me. They do include risk assessment columns but not RPN which is reassuring. But failure effects aren't necessarily hazards! Some of them might combine to be a sequence of events for A hazardous situation. Some might directly be the initiating mechanism for a hazardous situation. But otherwise they're not strictly hazards, nor does the FMEA capture hazards during normal operation (it isn't meant to of course). I was pretty confused but I also didn't want to speak up about it until I had a chance to mesh into the position.

At the end of this, agreed that the industries are very different, but also not that different from a high level. So far as I've been able to tell from studying someone like Clifton Ericson for aero/defense and now studying someone like Bijan Elahi, the basic principles and analysis are all there but it's just a different language. Kind of like how if I can write code in C++, it's not a HUGE leap to get into Java. The syntax is different, but coding isn't that foreign once you know the syntax. Think that's just what I'm trying to get confident with here, the syntax for this industry (and using 14971).
 

Tidge

Trusted Information Resource
Just to give an example of what I observed with the FMEAs. The FMEA documents I've reviewed call our failure effects as hazards which baffles me. They do include risk assessment columns but not RPN which is reassuring. But failure effects aren't necessarily hazards! Some of them might combine to be a sequence of events for A hazardous situation. Some might directly be the initiating mechanism for a hazardous situation. But otherwise they're not strictly hazards, nor does the FMEA capture hazards during normal operation (it isn't meant to of course). I was pretty confused but I also didn't want to speak up about it until I had a chance to mesh into the position.

Your instincts are correct, w.r.t that which I have bolded above. 14971 defines HAZARD as a potential source of HARM.

In the risk management file, I find it most convenient to organize the top level document by HAZARDS, which are most evidently recognized as elemental (or elemental-adjacent) things like: Electrical Energy, Thermal Energy, Kinetic Energy, Ionizing Energy, Infectious Agents, etc. A failure mode can expose a human to a hazard, but sometimes a medical device is designed to expose a human to a hazard... and there will be risks associated with those circumstances as well.

As an aside, I have a slight professional worry that medical device risk management is at risk for slipping into a mode of thinking where non-clinical harms such as (for lack of a technical term) "hurt feelings" are tried to be squeezed into a 14971 context, specifically to try to rationalize cybersecurity controls. I completely believe that cybersecurity concerns around Protected Health Information are well-founded... its just that I don't think that treating those types of concerns as HARMS in a 14971 context is well-motivated.
 

yodon

Leader
Super Moderator
The FMEA documents I've reviewed call our failure effects as hazards which baffles me.

This is, unfortunately, EXTREMELY common in the industry. I do some RM training from time to time and I spend a good deal of time on "getting in the right mindset." We always establish a standard set of harms (with associated fixed severity) for a product and forcing the choices from that list (and if something isn't on it or if the severity is not accurate, back to the HA we go).
 

colinkmorgan

Managing Director
AAIM TIR57 provides information related to assessing cybersecurity risks in parallel to the safety impact (e.g., ISO 14971) and provides context on when and where cybersecurity risks may cross over into patient safety risks. Unfortunately, we cannot only look at cybersecurity through the lens of impact to patient information for medical devices, as a cybersecurity risk could lead to a potential impact to patient safety. Now, cybersecurity risks should not typically create a new harm, rather when a cybersecurity risk is identified as potentially impacting patient safety, it would be in relation to the already defined harms (e.g., found in FMEA or Health Hazard Analysis).

The other key note is that cybersecurity needs to be evaluated from an exploitability perspective, not probability/likelihood. When something like a vulnerability is identified it’s critical to understand how easy or hard it is for a malicious person to take advantage of it, not how likely it is for them to take advantage of it. This is something that is different from the traditional risk assessment/management for medical devices and is described in the 2016 FDA Post Market Management of Cybersecurity Guidance.

As a hypothetical example, imagine a WiFi enabled infusion pump has a critical vulnerability that can be exploited across the hospital network, enabling the malicious individual to easily modify the drug library on the pump, changing the dosing a patient will receive. This may be considered an uncontrolled risk (per FDA 2016 Cybersecurity guidance) and require evaluation for impact to patient safety and could lead to a field action or recall. This is the difference with medical devices, in that patient safety impact needs to be considered for certain cybersecurity risks.

Thanks,
Colin Morgan
Managing Director
Apraciti, LLC | Medical Device Cybersecurity
[email protected]
 

Tidge

Trusted Information Resource
AAIM TIR57 provides information related to assessing cybersecurity risks in parallel to the safety impact (e.g., ISO 14971) and provides context on when and where cybersecurity risks may cross over into patient safety risks.

I'm a big fan of the TIR57 guidance, and I've leveraged the CIA (Confidentiality, Integrity, Availability) concerns quite well (IMO) in my medical device risk files. My professional concern is that since 14971 is the consensus standard for safety risks, that there are "lowest common denominator" modes of thinking that folks will try (and try, and try again) to use to shoehorn all of the cybersecurity concerns into 14971. This wouldn't be the first time that some (well-meaning, but) backwards thinking worked its way into the realm of medical devices, and those that use software.
 

VinceTech

Involved In Discussions
The fundamental thing shall remains unchanged. Hazard analysis is analysis which doesn't matter what type of hazard analysis we call them. The risk is Severity and Probability. this is the rule. In medical industry, different company, different department, or different management offer many names for what they are doing but the principle is often forgot: risk is severity and probability of harm. This rule applied to engineering, social science, medication, and any industry.
FMEA is a tool, it can be departed from risk assessment. However, many engineers learn risk analysis from starting using FMEA. This is the reason FMEA is often considered to be risk assessment. This is costly mistake. Just relying on FMEA is often insufficient for hazard analysis.
Yes, it's very confusing while defining a system. The simple principle you can apply is to consider what at the parts you will need to meet EP and BS. The minimum parts required for achieving EP and BS can be your system.

Just sharing idea, not sure 100% right. Any comments from experts?
 
Top Bottom