SBS - The best value in QMS software

Rationale for Risk Acceptability Matrix - ISO 14971

david316

Involved In Discussions
#1
Hello,

I have a question about defining Risk Acceptability Criteria (RAC) as required by 14971:2007. Specifically about the use of a matrix as per Annex D. Conceptually, when defining a RAC matrix should the acceptable area cover known unavoidable risks associated with the therapy. For example, when using an x-ray there is some harm associated with the therapy. Clearly the benefits outweigh the risk but should I construct my RAC table so severities and probabilities that align with this use (i.e. being exposed to x-rays as an unavoidable consequence) fall into the acceptable region?

The problem I see with this approach is that just because there are some inherit harms (with certain probabilities and severities occurring) you shouldn’t make these acceptable in your matrix as the matrix doesn’t explicitly cover the source of the harm i.e. you should not be able to justify poor engineering just because there is some inherit risk associated with the therapy.

So if I don’t use risk related to therapy to define my matrix, what process should I use to define the acceptable region. Should I limit the RAC criteria to be based on acceptable harm due to design/engineering faults given similar state of the art devices and then perform risk vs benefit analysis to justify risks that are inherit with the therapy or is there some other approach that is more suitable?

Thanks for any input.
 
Elsmar Forum Sponsor

Marcelo

Inactive Registered Visitor
#2
Hello David, and welcome to the Cove!

First, a risk matrix is not an acceptability tool, it's a ranking tool and should not be used as the sole driver to accept risk (please see this post for a discussion on this: https://elsmar.com/Forums/599407-post5.html)


Hello,

I have a question about defining Risk Acceptability Criteria (RAC) as required by 14971:2007. Specifically about the use of a matrix as per Annex D. Conceptually, when defining a RAC matrix should the acceptable area cover known unavoidable risks associated with the therapy. For example, when using an x-ray there is some harm associated with the therapy. Clearly the benefits outweigh the risk but should I construct my RAC table so severities and probabilities that align with this use (i.e. being exposed to x-rays as an unavoidable consequence) fall into the acceptable region?


This kind of risk should in principle be non-acceptable, so then you need to use a risk/benefit analysis to justify it. I know some people do the other way around, but it does not make much sense.




The problem I see with this approach is that just because there are some inherit harms (with certain probabilities and severities occurring) you shouldn’t make these acceptable in your matrix as the matrix doesn’t explicitly cover the source of the harm i.e. you should not be able to justify poor engineering just because there is some inherit risk associated with the therapy.


So if I don’t use risk related to therapy to define my matrix, what process should I use to define the acceptable region. Should I limit the RAC criteria to be based on acceptable harm due to design/engineering faults given similar state of the art devices and then perform risk vs benefit analysis to justify risks that are inherit with the therapy or is there some other approach that is more suitable?

Thanks for any input.
See comment above about using matrix.
 

david316

Involved In Discussions
#3
Thank you for the reply Marcelo. I have read the link posted but I am still a little confused. In both the 2007 and 2012 versions of the 14971 there are examples around the use of a matrix to help quantify/rank risks. The specific part I am confused about, is when constructing the matrix what do I use to justify the appropriate sections of the matrix. e.g. assuming I have quantified my severity and probability definitions, how do I justify a risk with minor severity and improbably is "insignificant risk" as per 2012 addition or "acceptable risk" as per 2007 version. It seems to me this is very difficult to do and often there is no rationale and the matrix just seem to appear with areas indicating acceptable/insignificant risk.

Thanks
 

Marcelo

Inactive Registered Visitor
#4
Thank you for the reply Marcelo. I have read the link posted but I am still a little confused. In both the 2007 and 2012 versions of the 14971 there are examples around the use of a matrix to help quantify/rank risks. The specific part I am confused about, is when constructing the matrix what do I use to justify the appropriate sections of the matrix. e.g. assuming I have quantified my severity and probability definitions, how do I justify a risk with minor severity and improbably is "insignificant risk" as per 2012 addition or "acceptable risk" as per 2007 version. It seems to me this is very difficult to do and often there is no rationale and the matrix just seem to appear with areas indicating acceptable/insignificant risk.

Thanks
First, please note that there's no 2012 versions of ISO 14971, the 2012 version is an European version (EN) with the weird content deviations (most of which are technically incorrect).

Second, I tried to easily answer your question with an example (in the other post) on how to deal with risk acceptability, but without a risk matrix (which you do not need and should really avoid).

Third, if you really want to know the answer to your question the way it is, you really need to know how to correctly create a risk matrix (which ISO 14971 unfortunately does not offer as guidance).

See, for example, the risk matrix chapter on this document NASA System Engineering “Toolbox” for Design-Oriented Engineers and first comment (from Mark Powell) on this discussion : https://www.linkedin.com/pulse/anyone-still-using-risk-heatmaps-ant%C3%B3nio-castro-caldas
 

david316

Involved In Discussions
#5
Third, if you really want to know the answer to your question the way it is, you really need to know how to correctly create a risk matrix (which ISO 14971 unfortunately does not offer as guidance).
Thank you the informative link. Unforuntanely I would like to know the answer to the question the way it is :). I have read section 3 in the NASA document. In the link it says "Note that not the analyst but management establishes and approves the risk tolerance boundaries". So as I understand this, management sets these risk boundaries. I would assume they set it based on their appetite for risk as well as what they feel is acceptable to society. If we are using a matrix as part of conformance to 14791 (which I agree is not ideal) is management under any obligation to justify were they define these risk boundaries?

Thanks
 

david316

Involved In Discussions
#6
Rephrasing my question:

If part of your risk acceptability criteria makes use of a risk matrix that defines acceptable and unacceptable level of risk, does ISO 14791 require you to justify why certain combinations of severity and probability are acceptable?

Followup question... what approach do companies normally take to define these levels?
 

Marcelo

Inactive Registered Visitor
#7
If we are using a matrix as part of conformance to 14791 (which I agree is not ideal) is management under any obligation to justify were they define these risk boundaries?
The risk management model of ISO 14971 is a little different because it incorporated the "top. management" concept of ISO 9001 (and I'm not saying that this is good :-_).

Anyway, top management is required to define the policy to determine the criteria for risk acceptability. Please note that it does not prevent top management to also define/help define the criteria.

I created a framework that I adapted from the NASA RM handbook, see annexed file.
 

Attachments

Marcelo

Inactive Registered Visitor
#8
If part of your risk acceptability criteria makes use of a risk matrix that defines acceptable and unacceptable level of risk, does ISO 14791 require you to justify why certain combinations of severity and probability are acceptable?
ISO 214971 does have a requirement that says that "Any system used for qualitative or quantitative categorization of probability of occurrence of harm or severity of harm shall be recorded in the risk management file." However, there's no formal requirement that the system is explained.

Hoever, I always suggest that, if you use a risk matrix as a starting point to rank risks, you clearly define the matrix. There's a bunch of literature and other sources that can be used for that (for an example, see Appendix 4 on the Reducing risks, protecting people - HSE’s decision-making process handbook.


Followup question... what approach do companies normally take to define these levels?
Unfortunately, most of the companies I know simply copy a risk matrix, either from the examples of ISO 14971 or from some other source, without understanding how it works, basically because they want to show something to "pass the audit".
 

david316

Involved In Discussions
#9
Thank you for your answer. Is the following summary correct?

-Ideally define risk acceptability criteria similar to the suggestion in your first reply.
-If you are using a risk matrix as part of you risk acceptability criteria you should be using it as a risk ranking tool.
-Avoid using a risk matrix with regions labelled acceptable and unacceptable as part of your risk acceptability criteria. But if you do, there is no requirement to explain what rationale was used to set your areas of acceptable risk or unacceptable risk. Note, I am aware that you need to justify the residual risk is acceptable so that kind of ensures you can't have unreasonable criteria.


As a followup, I can't help but wonder if you use a risk acceptable criteria as you suggested in your first post, how do you apply this to other standards that require you to ensure the risk is acceptable. For example I'm sure 60601-1 has lots of clauses with wording to effect. Another example is as part of 63204 when determining software classification you need to consider whether failure of your software will lead to unacceptable risk. Its not clear to me how you would apply your risk acceptability criteria to this example.

Thanks again.
 

Marcelo

Inactive Registered Visitor
#10
Thank you for your answer. Is the following summary correct?

-Ideally define risk acceptability criteria similar to the suggestion in your first reply.
-If you are using a risk matrix as part of you risk acceptability criteria you should be using it as a risk ranking tool.
-Avoid using a risk matrix with regions labelled acceptable and unacceptable as part of your risk acceptability criteria. But if you do, there is no requirement to explain what rationale was used to set your areas of acceptable risk or unacceptable risk. Note, I am aware that you need to justify the residual risk is acceptable so that kind of ensures you can't have unreasonable criteria.
For the first one, I would add that the main thing is - risk acceptability has to take into consideration more than just the components of the definition of risk (combination of probability and severity) and thus the risk acceptability criteria should not be only about probability and severity.

For the second one I would add that a risk matrix is always a risk ranking tool (it's the nature of the tool), so it should be used accordingly

For the third one, I would add that you can use a risk matrix with regions, but if you do, you should again use it as a guide to ranking only, not to really accept risk. For example, the original use of risk matrices was to define who (which level of authority) would justify why the risk is acceptable.

A final one, which stems from the others and is not clear at all in ISO 14971 - you should justify why risk are acceptable. For example, the ALARP principle (from HSE) recognizes three approaches to making a claim that risk is ALARP:

- Good practice arguments which demonstrate that risk control measures comply with relevant good practice as defined in ACoPs, HSE guidance, standards etc

- Qualitative first principles arguments based on common sense or professional judgement to weigh possible risk reduction against the necessary “sacrifice”

- Quantitative first principles arguments based on numerical techniques such as Cost Benefit Analysis (CBA) to weigh possible risk reduction against the necessary “sacrifice”.

All of them requires justification. This is more or less what I used as a basis for part of the criteria in my example from the other thread.

As a followup, I can't help but wonder if you use a risk acceptable criteria as you suggested in your first post, how do you apply this to other standards that require you to ensure the risk is acceptable. For example I'm sure 60601-1 has lots of clauses with wording to effect. Another example is as part of 63204 when determining software classification you need to consider whether failure of your software will lead to unacceptable risk. Its not clear to me how you would apply your risk acceptability criteria to this example.
See my last statement above - you justify why the risk is acceptable.

Also, using 14971 together with other standards can be tricky, but needs to be incorporated in tour criteria. For example, to use with 60601-1, you should use the flowchart we created in ISO 24971 (basically, if everything in the flowchart is complied with, and testing shows that the requirement for the standard is fulfilled, you can deem the risk is acceptable - this is a way to justify acceptability by the "good practice argument" as above).

The approach for 62304 is a little different, but is in practice the same (and we made it a little more clear on the amendment by introducing the "external to the software" part of the risk classification).
 
Thread starter Similar threads Forum Replies Date
V Criteria and Rationale for Selection of Risk Management Tool ISO 14971 - Medical Device Risk Management 2
M Minimum sample size - Guidance and statistical rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
R Design verification for interim design outputs - sampling rationale ISO 13485:2016 - Medical Device Quality Management Systems 2
I Device modifications - Clinical sample size rationale EU Medical Device Regulations 5
R Rationale for changes to ISO Standards Other Medical Device Related Standards 4
S PMCF (Post Market Clinical Followup): Rationale for Exemption - Software Medical Devices EU Medical Device Regulations 6
S Literature Search - Notified Body Had Questions About our Rationale CE Marking (Conformité Européene) / CB Scheme 9
M Rationale for 60601-1-2 Accompanying Documents Requirements IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
S Please explain Medical Device Product Classification Rationale 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
M Revising Declaration of Conformity and Classification Rationale EU Medical Device Regulations 2
Z Example of a Classification Rationale Statement CE Marking (Conformité Européene) / CB Scheme 3
J Sampling Plan Justification and Rationale Inspection, Prints (Drawings), Testing, Sampling and Related Topics 9
R Frequency Level Rationale - Nonconformance warrants a Root Cause Investigation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
Q Justification/Rationale for not investigating a nonconformance ISO 13485:2016 - Medical Device Quality Management Systems 19
P Reference Only Gages - "For Reference Only" Rationale (Reasoning/Justification) General Measurement Device and Calibration Topics 25
A Whats the rationale for a Sample size of 30 for Capability Analysis? Statistical Analysis Tools, Techniques and SPC 5
Marc The History and Rationale of the MIL-STD-810 testing standard Book, Video, Blog and Web Site Reviews and Recommendations 0
B AS9102 - 3D printing a special tool required for assembly (counterfeit risk?) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 10
K Defining risk control measures IEC 62304 - Medical Device Software Life Cycle Processes 13
U Supply risk management Manufacturing and Related Processes 4
T Biological Evaluation (10993) & Risk Management ISO 14971 - Medical Device Risk Management 9
D Cybersecurity and Risk Management: Loss of confidentiality IEC 62304 - Medical Device Software Life Cycle Processes 4
Q FMEA and Risk assessment in Microsoft Access FMEA and Control Plans 6
I Realization processes input into overall risk ISO 14971 - Medical Device Risk Management 2
M Need Help With Information Security Asset Risk Register IEC 27001 - Information Security Management Systems (ISMS) 2
thisby_ Post Market/Production Risk Assessment ISO 14971 - Medical Device Risk Management 0
S Risk Management Review ISO 14971 - Medical Device Risk Management 4
D Low risk IVD study in the UK, do I need MHRA approval? UK Medical Device Regulations 1
S Risk Management and other Files ISO 14971 - Medical Device Risk Management 8
silentmonkey Overall Benefit/Risk Analysis - Risk Management VS Clinical Evaluation ISO 14971 - Medical Device Risk Management 3
N ISO 27001 for Jumb Burger - Risk Assessment sheet IEC 27001 - Information Security Management Systems (ISMS) 11
C Risk Assessment Tools ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
qualprod Examples to mitigate risk from Covid ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
G Risk of stopping your customer's line IATF 16949 - Automotive Quality Systems Standard 4
C Risk Matrix vs FMEAs ISO 14971 - Medical Device Risk Management 11
S IVD risk class II devices for Brazil and MDSAP Other Medical Device Regulations World-Wide 0
M ISO 14971:2019: Criteria for overall residual risk ISO 14971 - Medical Device Risk Management 11
M ISO14971:2019 - Verification of implementation and effectiveness of risk control ISO 14971 - Medical Device Risk Management 3
Aymaneh Medical Device Cybersecurity Risk Management IEC 27001 - Information Security Management Systems (ISMS) 2
S Traceability of requirements to design and risk Design and Development of Products and Processes 3
R Risk control measures as per ISO 14971 ISO 14971 - Medical Device Risk Management 6
D Deciding whether or not pre-market clinical investigation is required for low risk device EU Medical Device Regulations 5
R The term "Benefit Risk Ratio" in EU MDR, do I need to present benefit risk analysis as a RATIO Risk Management Principles and Generic Guidelines 4
_robinsingh Security Risk Assessment Tool IEC 27001 - Information Security Management Systems (ISMS) 0
A 21 CFR 820 - Risk Management - Looking for some guidance US Food and Drug Administration (FDA) 3
bryan willemot Contract Review and risk managment AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
D Risk Analysis using Monte Carlo Simulation instead of Scoring and Heat Map Risk Management Principles and Generic Guidelines 2
Sravan Manchikanti Software Risk Management & probability of occurrence as per IEC 62304 IEC 62304 - Medical Device Software Life Cycle Processes 8
E Normal Condition Hazards in Risk Analysis ISO 14971 - Medical Device Risk Management 3
silentmonkey Rationalising the level of effort and depth of software validation based on risk ISO 13485:2016 - Medical Device Quality Management Systems 10

Similar threads

Top Bottom