How to qualitatively assess initial probability

[MrTetris]

Hello, quality people
An interesting problem on a risk assessment, I am curious on how you would approach it.
We offer surgical planning service to surgeons. Surgeons make their surgical proposal on the basis of patient's images (ct scans), then we have service engineers controlling their plans and eventually adapting it.
One of the risks listed in our RMC is the possibility that anatomical obstacles are incorrectly indicated in the plan by SE, resulting in false positive and/or negative.
This does not happen very often (very rarely, actually), since our Service Engineers are adequately trained. The problem is how to handle this case in the RMC.
1- We assume that the initial probability of indicating incorrect obstacles is high, "likely" (as if a non-trained person would do the exercise - this never happens in reality), and then we decrease the index in the final RMC since our Service Engineers are actually trained (3 --> 2).
2- Since our SE never work on surgical plans before being trained, we assume a low probability ("unlikely") in the initial RMC, and then we do not decrease the risk any further (2 --> 2).
I believe that option 2 reflects our reality, so I would proceed with it, but I am curious to read your opinions.

 

[Tidge]

My preference is to go with option (1). I am motivated in this direction because reducing the probability from X -> 2 will clearly motivate some element of analysis that justifies your assessment that you have actually arrived at a probability of 2. Generally, if a team (qualitatively) starts at some lower pre-determined assessment of probability and doesn't think that it can be reduced there is likely to be no analysis before or after the implementation of any possible risk controls.

I encourage you to review a classic page from AAMI HE75:2009 in section 5.2 "Types of Use Errors". The chart of interest is also found in Figure B.1 of IEC 62366:2007; I don't think it survived in either 62366-1:2015 or 62366-2:2016. The diagram does a good job of (graphically) describing how using equipment for its intended use can lead to use errors, and how these are distinct from "inadequately trained or unqualified use". Examples of use errors that can be made by an adequately trained user is the misapplication of a good rule, or applying a well-meant "optimization".

 

[MrTetris]

My preference is to go with option (1). I am motivated in this direction because reducing the probability from X -> 2 will clearly motivate some element of analysis that justifies your assessment that you have actually arrived at a probability of 2. Generally, if a team (qualitatively) starts at some lower pre-determined assessment of probability and doesn't think that it can be reduced there is likely to be no analysis before or after the implementation of any possible risk controls.

I encourage you to review a classic page from AAMI HE75:2009 in section 5.2 "Types of Use Errors". The chart of interest is also found in Figure B.1 of IEC 62366:2007; I don't think it survived in either 62366-1:2015 or 62366-2:2016. The diagram does a good job of (graphically) describing how using equipment for its intended use can lead to use errors, and how these are distinct from "inadequately trained or unqualified use". Examples of use errors that can be made by an adequately trained user is the misapplication of a good rule, or applying a well-meant "optimization".

Thank you for your answer Tidge. Regarding option 1, would you not consider training as Information for Safety and as such not suitable to decrease the occurrence probability (3-->2)? The new 14971 clearly states that "Training to users can be an important aspect of delivering information for safety", so I am reluctant to use this control measure to decrease risk index values.

 

[Tidge]

I wouldn't necessarily consider training to be an IFS risk control, unless the manufacturer of record controls all aspects of training (materials, trainer certifications, student assessments, etc.) and somehow guarantees that only trained and competent users are the ones using the device. It is rare that a manufacturer takes such a complete ownership of the training for a medical device in this manner.

I want to contrast this with the situation in which the device manufacturer has formally identified a specific class of users with particular training specifically applicable to the given device. The identification of user classes happens early in risk assessments and could impact the (initial) ratings of certain risks, but their specific training still would not be IFS. Most likely this would be in the "P2" rating (probability that a hazardous situation leads to a harm), but YMMV. If this pre-assessment factor is keeping risks acceptable, the medical device manufacturer has some obligation to ensure that only such formally trained users access the device, otherwise there will be a need to consider the class of untrained users and we will be back where we started.

I wouldn't use IFS to reduce the assessed value of risk, but I do account for IFS when performing risk benefit analyses.

I'm a little old school, with a hopefully solid bit of new school: The IFS is obligated to disclose risks that the manufacturer has (in the absence of a positive-outcome RBA) determined would otherwise be unacceptable, but it also can contain specific information that (when adhered to) contributes to the continued safe use of the device. An example of the former might be disclosing the potential allergy-sensitive materials in an implantable device where there is no alternate but to use such materials. An example of the latter might be listing cleaning products that are not to be used on a device because they could compromise one or more aspects of the device. In neither case would the manufacturer have fully 'designed away' certain specific risks, but in each case IFS can contribute to an improved RBA leading to acceptability.

 

[Ronen E]

I think that in this case the training discussed is not "normal" training provided to users.
In this case the "product" is a service. The training discussed is part of "manufacturing the product" (the service), because it's not provided to users of the service (e.g. surgeons), but to employees providing/enabling/creating the service (service engineers).

Please also don't mix provision of information for safety with disclosure of residual risks. They are not exactly in the same regulatory status.