Risk trace matrix vs design trace matrix

[stm55]

I've been trying to wrap my head around the traceability aspect of hazards to harms to risk controls, residual risk assessments, etc as well as the design related tracing of user needs, inputs, outputs, verification evidence etc. While these things seem to go somewhat hand in hand, I am envisioning it to be pretty clunky to combine these into a single document. I've also digested quite a bit of information related to these two topics over the past couple weeks, so my brain is a little fried, and I'm hoping someone could provide a comment that will click for me!

We currently rely on our FMEAs for essentially all risk documentation. I understand this is insufficient in the typical form of an FMEA. In trying to remedy this, I am thinking there should be (minimally) a master trace matrix or something similar that would list all potential hazards, hazardous situations, harms, etc, and then continue on to further columns that show any controls were verified as implemented/effective, a residual risk assessment, etc. In my mind, the FMEA can relate to this, or essentially provide a partial subset of the line items in this overall trace matrix. I'm sure there are other ways to skin the cat as well, but something like this seems like a reasonable way to satisfy 14971 without having to create a laundry list of new risk docs. Does anyone have a good example of this, or can anyone provide an alternative option for someone who is looking to expand from an FMEA to cover the remaining requirements of 14971?

My question on DI/DO traceability I assume is a separate one that would typically be covered in a separate document, but wanted to throw this in as well since this is a separate item I am looking into, and wanted to see if anyone had other relevant comments that bridges these two things together.

Thanks!!!!

 

[Ed Panek]

We use a hazard trace based on risks.

We also have a design trace that includes all other requirements. Regulatory. Quality Purchasing. Human factors. production. Shipping. Essential performance. User request. Configuration spec. Power spec. Etc.

Aside from risks we need to buy parts. Inspect them. Store them. Pull them. Build them. Ship them. Support them. Cradle to grave.

 

[stm55]

We use a hazard trace based on risks.

We also have a design trace that includes all other requirements. Regulatory. Quality Purchasing. Human factors. production. Shipping. Essential performance. User request. Configuration spec. Power spec. Etc.

Aside from risks we need to buy parts. Inspect them. Store them. Pull them. Build them. Ship them. Support them. Cradle to grave.

So would it be fair to say that you have a Risk Management Plan and Report, but between those things the main standalone document is your hazard trace? I assume this is just a large spreadsheet that would in turn reference other risk documents-- for instance in your verification of effectiveness of controls you might call out some separate report that confirmed this?

Then your design trace matrix is an entirely separate document that lists all the design inputs, user requirements, outputs, methods used to verify/validate them, etc? No other real explicit linkage to the hazard trace you mentioned?

 

[Ed Panek]

We have an annual risk review. We utilize post market surveillance data from our complaint database and MAUDE data from FDA to list and address risks. This is an input to management review.

Risks constantly evolve. Treatment methods change, where they use the device (for example home use is rising).

 

[stm55]

Understood.. I wasn't intending to imply that this should be a one-and-done process. That said, I do still have the same questions as in my last post

 

[yodon]

As you note, FMEAs alone are insufficient to make a complete risk file.

We use table C.1 to create the master list of hazards, harms, and severity. It's not a "trace matrix" but it's the one place from which FMEAs must get that information (updated if the work in the FMEAs identify gaps).

There are 2 aspects to residual risk: on individual risks and overall. The EU wants to see residual risk analysis / justification for each risk (there has been considerable discussion on this and there seems to be consensus that it's ok to limit the scope of this to the ones with higher risk scores). The overall residual risk is considered along with the overall benefit-risk analysis.

We have the following documents in our risk files:

• Plan
• Characterization (questions from annex A in 24971) - we couple this with the UE characteristics from 62366
• Hazard analysis (table C.1 from 14971 as described above)
• FMEAs (design, usability, software, process)
• Benefit-risk analysis
• Risk Report

We generally trace controls either directly to V&V testing (bear in mind both verification for implementation and verification of effectiveness!) or to system / software requirements (which then get traced to tests).

It's what we do and is not the only way, of course. Hope that helps.

 

[stm55]

As you note, FMEAs alone are insufficient to make a complete risk file.

We use table C.1 to create the master list of hazards, harms, and severity. It's not a "trace matrix" but it's the one place from which FMEAs must get that information (updated if the work in the FMEAs identify gaps).

There are 2 aspects to residual risk: on individual risks and overall. The EU wants to see residual risk analysis / justification for each risk (there has been considerable discussion on this and there seems to be consensus that it's ok to limit the scope of this to the ones with higher risk scores). The overall residual risk is considered along with the overall benefit-risk analysis.

We have the following documents in our risk files:

• Plan
• Characterization (questions from annex A in 24971) - we couple this with the UE characteristics from 62366
• Hazard analysis (table C.1 from 14971 as described above)
• FMEAs (design, usability, software, process)
• Benefit-risk analysis
• Risk Report

We generally trace controls either directly to V&V testing (bear in mind both verification for implementation and verification of effectiveness!) or to system / software requirements (which then get traced to tests).

It's what we do and is not the only way, of course. Hope that helps.

Thanks! That is helpful! I think I understand your general system, but can you explain a little more the "trace" portion? I'm envisioning your characterization and hazard analysis generally give you your listing of hazards/harms/etc. Are you saying that it is essentially a summary document where there is a column that gives a risk score, and then if needed a column with the controls added, and finally columns calling out your V&V and residual risk decisions? (this is the trace matrix I am envisioning)

Or are you basically saying that your Benefit Risk Analysis/Risk Report doc would essentially list any higher level risks and then summarize the verification activities/assessments associated with them?

I understand the requirements of the standard, but I am just struggling to figure out the most effective means of documenting. I think an FMEA-type document intuitively makes sense since it will have all the columns and can trace everything to the different individual documents if needed. However the FMEA itself wont work as the main "trace" document for a number of reasons (one of the most notable being that it wouldnt include non-fault conditions)

 

[Tidge]

I understand the requirements of the standard, but I am just struggling to figure out the most effective means of documenting. I think an FMEA-type document intuitively makes sense since it will have all the columns and can trace everything to the different individual documents if needed. However the FMEA itself wont work as the main "trace" document for a number of reasons (one of the most notable being that it wouldnt include non-fault conditions)

FMEA are inherently deficient for a holistic analysis of risks because:

1. FMEA literally only analyze failure modes, and not all risks come from failure modes.
2. It is possible to have an FMEA dominated by no-risk failure modes, which confound the issue of risk analysis
3. Failure modes in one (type of) FMEA may have to be addressed in a different FMEA E.g.: Impossible to design a sterile device? Add a sterilization step in manufacturing.

 

[yodon]

In our approach, the hazard analysis establishes the definitive set of harms and their associated severity. It does not get into controls or risk scoring.

Controls and risk scoring is in individual FMEAs.

I presume this statement, from the standard, is what you're asking about regarding traceability:

the risk management file shall provide traceability for each identified hazard to:
— the risk analysis;
— the risk evaluation;
— the implementation and verification of the risk control measures; and
— the results of the evaluation of the residual risks.

Other than the 3rd bullet, all that is covered in the FMEA (for each risk). Risk controls have unique identifiers which enable them to be traced to the tests where they are verified (implementation and effectiveness).

Does that clarify? Bear in mind this is the approach we take and is not prescriptive by any means. Other paths can be taken but we like our approach.

 

[stm55]

In our approach, the hazard analysis establishes the definitive set of harms and their associated severity. It does not get into controls or risk scoring.

Controls and risk scoring is in individual FMEAs.

I presume this statement, from the standard, is what you're asking about regarding traceability:

the risk management file shall provide traceability for each identified hazard to:
— the risk analysis;
— the risk evaluation;
— the implementation and verification of the risk control measures; and
— the results of the evaluation of the residual risks.

Other than the 3rd bullet, all that is covered in the FMEA (for each risk). Risk controls have unique identifiers which enable them to be traced to the tests where they are verified (implementation and effectiveness).

Does that clarify? Bear in mind this is the approach we take and is not prescriptive by any means. Other paths can be taken but we like our approach.

That seems to make sense, but I guess my struggle then is what about risks/hazards that would not be on an FMEA? For example, non-fault conditions... This is not a "failure" but still could be a hazard. For examples, "user uses device after expiration", "user does not properly dispose of device", "user uses device and doesn't realize they are allergic to its materials", etc. All these can potentially lead to harm which should be addressed. Also, FMEA will generally not look at multi-fault conditions, reasonably foreseeable misuse, etc

Also, from my understanding, the resulting actions brought about by FMEA high RPN's are not necessarily the same as what ISO14971 gets at. For example, a PFMEA might lead to an action such as equipment PM to mitigate the risk of tooling becoming dull, etc. Whereas 14971 seems to want you to look at the risk associated with patient harm. An unacceptable risk should be dealt with by inherently safe design, user alarms, instructions for safety, etc. So the same general issue may lead to you changing the design of the product-- which an FMEA may not steer you toward.

Please let me know if I am looking at this incorrectly, but I am seeing in many places that the FMEA does not seem to be the most appropriate place to be the sole spot where you do your risk mitigation.