Rationale for Risk Acceptability Matrix - ISO 14971

david316

Involved In Discussions
Hello,

I have a question about defining Risk Acceptability Criteria (RAC) as required by 14971:2007. Specifically about the use of a matrix as per Annex D. Conceptually, when defining a RAC matrix should the acceptable area cover known unavoidable risks associated with the therapy. For example, when using an x-ray there is some harm associated with the therapy. Clearly the benefits outweigh the risk but should I construct my RAC table so severities and probabilities that align with this use (i.e. being exposed to x-rays as an unavoidable consequence) fall into the acceptable region?

The problem I see with this approach is that just because there are some inherit harms (with certain probabilities and severities occurring) you shouldn’t make these acceptable in your matrix as the matrix doesn’t explicitly cover the source of the harm i.e. you should not be able to justify poor engineering just because there is some inherit risk associated with the therapy.

So if I don’t use risk related to therapy to define my matrix, what process should I use to define the acceptable region. Should I limit the RAC criteria to be based on acceptable harm due to design/engineering faults given similar state of the art devices and then perform risk vs benefit analysis to justify risks that are inherit with the therapy or is there some other approach that is more suitable?

Thanks for any input.
 

Marcelo

Inactive Registered Visitor
Hello David, and welcome to the Cove!

First, a risk matrix is not an acceptability tool, it's a ranking tool and should not be used as the sole driver to accept risk (please see this post for a discussion on this: https://elsmar.com/Forums/599407-post5.html)


Hello,

I have a question about defining Risk Acceptability Criteria (RAC) as required by 14971:2007. Specifically about the use of a matrix as per Annex D. Conceptually, when defining a RAC matrix should the acceptable area cover known unavoidable risks associated with the therapy. For example, when using an x-ray there is some harm associated with the therapy. Clearly the benefits outweigh the risk but should I construct my RAC table so severities and probabilities that align with this use (i.e. being exposed to x-rays as an unavoidable consequence) fall into the acceptable region?


This kind of risk should in principle be non-acceptable, so then you need to use a risk/benefit analysis to justify it. I know some people do the other way around, but it does not make much sense.




The problem I see with this approach is that just because there are some inherit harms (with certain probabilities and severities occurring) you shouldn’t make these acceptable in your matrix as the matrix doesn’t explicitly cover the source of the harm i.e. you should not be able to justify poor engineering just because there is some inherit risk associated with the therapy.


So if I don’t use risk related to therapy to define my matrix, what process should I use to define the acceptable region. Should I limit the RAC criteria to be based on acceptable harm due to design/engineering faults given similar state of the art devices and then perform risk vs benefit analysis to justify risks that are inherit with the therapy or is there some other approach that is more suitable?

Thanks for any input.

See comment above about using matrix.
 

david316

Involved In Discussions
Thank you for the reply Marcelo. I have read the link posted but I am still a little confused. In both the 2007 and 2012 versions of the 14971 there are examples around the use of a matrix to help quantify/rank risks. The specific part I am confused about, is when constructing the matrix what do I use to justify the appropriate sections of the matrix. e.g. assuming I have quantified my severity and probability definitions, how do I justify a risk with minor severity and improbably is "insignificant risk" as per 2012 addition or "acceptable risk" as per 2007 version. It seems to me this is very difficult to do and often there is no rationale and the matrix just seem to appear with areas indicating acceptable/insignificant risk.

Thanks
 

Marcelo

Inactive Registered Visitor
Thank you for the reply Marcelo. I have read the link posted but I am still a little confused. In both the 2007 and 2012 versions of the 14971 there are examples around the use of a matrix to help quantify/rank risks. The specific part I am confused about, is when constructing the matrix what do I use to justify the appropriate sections of the matrix. e.g. assuming I have quantified my severity and probability definitions, how do I justify a risk with minor severity and improbably is "insignificant risk" as per 2012 addition or "acceptable risk" as per 2007 version. It seems to me this is very difficult to do and often there is no rationale and the matrix just seem to appear with areas indicating acceptable/insignificant risk.

Thanks

First, please note that there's no 2012 versions of ISO 14971, the 2012 version is an European version (EN) with the weird content deviations (most of which are technically incorrect).

Second, I tried to easily answer your question with an example (in the other post) on how to deal with risk acceptability, but without a risk matrix (which you do not need and should really avoid).

Third, if you really want to know the answer to your question the way it is, you really need to know how to correctly create a risk matrix (which ISO 14971 unfortunately does not offer as guidance).

See, for example, the risk matrix chapter on this document NASA System Engineering “Toolbox” for Design-Oriented Engineers and first comment (from Mark Powell) on this discussion : https://www.linkedin.com/pulse/anyone-still-using-risk-heatmaps-ant%C3%B3nio-castro-caldas
 

david316

Involved In Discussions
Third, if you really want to know the answer to your question the way it is, you really need to know how to correctly create a risk matrix (which ISO 14971 unfortunately does not offer as guidance).

Thank you the informative link. Unforuntanely I would like to know the answer to the question the way it is :). I have read section 3 in the NASA document. In the link it says "Note that not the analyst but management establishes and approves the risk tolerance boundaries". So as I understand this, management sets these risk boundaries. I would assume they set it based on their appetite for risk as well as what they feel is acceptable to society. If we are using a matrix as part of conformance to 14791 (which I agree is not ideal) is management under any obligation to justify were they define these risk boundaries?

Thanks
 

david316

Involved In Discussions
Rephrasing my question:

If part of your risk acceptability criteria makes use of a risk matrix that defines acceptable and unacceptable level of risk, does ISO 14791 require you to justify why certain combinations of severity and probability are acceptable?

Followup question... what approach do companies normally take to define these levels?
 

Marcelo

Inactive Registered Visitor
If we are using a matrix as part of conformance to 14791 (which I agree is not ideal) is management under any obligation to justify were they define these risk boundaries?

The risk management model of ISO 14971 is a little different because it incorporated the "top. management" concept of ISO 9001 (and I'm not saying that this is good :-_).

Anyway, top management is required to define the policy to determine the criteria for risk acceptability. Please note that it does not prevent top management to also define/help define the criteria.

I created a framework that I adapted from the NASA RM handbook, see annexed file.
 

Attachments

  • Risk informed decision framework.jpg
    Risk informed decision framework.jpg
    92.7 KB · Views: 609

Marcelo

Inactive Registered Visitor
If part of your risk acceptability criteria makes use of a risk matrix that defines acceptable and unacceptable level of risk, does ISO 14791 require you to justify why certain combinations of severity and probability are acceptable?

ISO 214971 does have a requirement that says that "Any system used for qualitative or quantitative categorization of probability of occurrence of harm or severity of harm shall be recorded in the risk management file." However, there's no formal requirement that the system is explained.

Hoever, I always suggest that, if you use a risk matrix as a starting point to rank risks, you clearly define the matrix. There's a bunch of literature and other sources that can be used for that (for an example, see Appendix 4 on the Reducing risks, protecting people - HSE’s decision-making process handbook.


Followup question... what approach do companies normally take to define these levels?

Unfortunately, most of the companies I know simply copy a risk matrix, either from the examples of ISO 14971 or from some other source, without understanding how it works, basically because they want to show something to "pass the audit".
 

david316

Involved In Discussions
Thank you for your answer. Is the following summary correct?

-Ideally define risk acceptability criteria similar to the suggestion in your first reply.
-If you are using a risk matrix as part of you risk acceptability criteria you should be using it as a risk ranking tool.
-Avoid using a risk matrix with regions labelled acceptable and unacceptable as part of your risk acceptability criteria. But if you do, there is no requirement to explain what rationale was used to set your areas of acceptable risk or unacceptable risk. Note, I am aware that you need to justify the residual risk is acceptable so that kind of ensures you can't have unreasonable criteria.


As a followup, I can't help but wonder if you use a risk acceptable criteria as you suggested in your first post, how do you apply this to other standards that require you to ensure the risk is acceptable. For example I'm sure 60601-1 has lots of clauses with wording to effect. Another example is as part of 63204 when determining software classification you need to consider whether failure of your software will lead to unacceptable risk. Its not clear to me how you would apply your risk acceptability criteria to this example.

Thanks again.
 

Marcelo

Inactive Registered Visitor
Thank you for your answer. Is the following summary correct?

-Ideally define risk acceptability criteria similar to the suggestion in your first reply.
-If you are using a risk matrix as part of you risk acceptability criteria you should be using it as a risk ranking tool.
-Avoid using a risk matrix with regions labelled acceptable and unacceptable as part of your risk acceptability criteria. But if you do, there is no requirement to explain what rationale was used to set your areas of acceptable risk or unacceptable risk. Note, I am aware that you need to justify the residual risk is acceptable so that kind of ensures you can't have unreasonable criteria.

For the first one, I would add that the main thing is - risk acceptability has to take into consideration more than just the components of the definition of risk (combination of probability and severity) and thus the risk acceptability criteria should not be only about probability and severity.

For the second one I would add that a risk matrix is always a risk ranking tool (it's the nature of the tool), so it should be used accordingly

For the third one, I would add that you can use a risk matrix with regions, but if you do, you should again use it as a guide to ranking only, not to really accept risk. For example, the original use of risk matrices was to define who (which level of authority) would justify why the risk is acceptable.

A final one, which stems from the others and is not clear at all in ISO 14971 - you should justify why risk are acceptable. For example, the ALARP principle (from HSE) recognizes three approaches to making a claim that risk is ALARP:

- Good practice arguments which demonstrate that risk control measures comply with relevant good practice as defined in ACoPs, HSE guidance, standards etc

- Qualitative first principles arguments based on common sense or professional judgement to weigh possible risk reduction against the necessary “sacrifice”

- Quantitative first principles arguments based on numerical techniques such as Cost Benefit Analysis (CBA) to weigh possible risk reduction against the necessary “sacrifice”.

All of them requires justification. This is more or less what I used as a basis for part of the criteria in my example from the other thread.

As a followup, I can't help but wonder if you use a risk acceptable criteria as you suggested in your first post, how do you apply this to other standards that require you to ensure the risk is acceptable. For example I'm sure 60601-1 has lots of clauses with wording to effect. Another example is as part of 63204 when determining software classification you need to consider whether failure of your software will lead to unacceptable risk. Its not clear to me how you would apply your risk acceptability criteria to this example.

See my last statement above - you justify why the risk is acceptable.

Also, using 14971 together with other standards can be tricky, but needs to be incorporated in tour criteria. For example, to use with 60601-1, you should use the flowchart we created in ISO 24971 (basically, if everything in the flowchart is complied with, and testing shows that the requirement for the standard is fulfilled, you can deem the risk is acceptable - this is a way to justify acceptability by the "good practice argument" as above).

The approach for 62304 is a little different, but is in practice the same (and we made it a little more clear on the amendment by introducing the "external to the software" part of the risk classification).
 
Top Bottom