How to qualitatively assess initial probability

MrTetris

Involved In Discussions
#1
Hello, quality people :)
An interesting problem on a risk assessment, I am curious on how you would approach it.
We offer surgical planning service to surgeons. Surgeons make their surgical proposal on the basis of patient's images (ct scans), then we have service engineers controlling their plans and eventually adapting it.
One of the risks listed in our RMC is the possibility that anatomical obstacles are incorrectly indicated in the plan by SE, resulting in false positive and/or negative.
This does not happen very often (very rarely, actually), since our Service Engineers are adequately trained. The problem is how to handle this case in the RMC.
1- We assume that the initial probability of indicating incorrect obstacles is high, "likely" (as if a non-trained person would do the exercise - this never happens in reality), and then we decrease the index in the final RMC since our Service Engineers are actually trained (3 --> 2).
2- Since our SE never work on surgical plans before being trained, we assume a low probability ("unlikely") in the initial RMC, and then we do not decrease the risk any further (2 --> 2).
I believe that option 2 reflects our reality, so I would proceed with it, but I am curious to read your opinions.
 
Elsmar Forum Sponsor

Tidge

Trusted Information Resource
#2
My preference is to go with option (1). I am motivated in this direction because reducing the probability from X -> 2 will clearly motivate some element of analysis that justifies your assessment that you have actually arrived at a probability of 2. Generally, if a team (qualitatively) starts at some lower pre-determined assessment of probability and doesn't think that it can be reduced there is likely to be no analysis before or after the implementation of any possible risk controls.

I encourage you to review a classic page from AAMI HE75:2009 in section 5.2 "Types of Use Errors". The chart of interest is also found in Figure B.1 of IEC 62366:2007; I don't think it survived in either 62366-1:2015 or 62366-2:2016. The diagram does a good job of (graphically) describing how using equipment for its intended use can lead to use errors, and how these are distinct from "inadequately trained or unqualified use". Examples of use errors that can be made by an adequately trained user is the misapplication of a good rule, or applying a well-meant "optimization".
 

MrTetris

Involved In Discussions
#3
My preference is to go with option (1). I am motivated in this direction because reducing the probability from X -> 2 will clearly motivate some element of analysis that justifies your assessment that you have actually arrived at a probability of 2. Generally, if a team (qualitatively) starts at some lower pre-determined assessment of probability and doesn't think that it can be reduced there is likely to be no analysis before or after the implementation of any possible risk controls.

I encourage you to review a classic page from AAMI HE75:2009 in section 5.2 "Types of Use Errors". The chart of interest is also found in Figure B.1 of IEC 62366:2007; I don't think it survived in either 62366-1:2015 or 62366-2:2016. The diagram does a good job of (graphically) describing how using equipment for its intended use can lead to use errors, and how these are distinct from "inadequately trained or unqualified use". Examples of use errors that can be made by an adequately trained user is the misapplication of a good rule, or applying a well-meant "optimization".
Thank you for your answer Tidge. Regarding option 1, would you not consider training as Information for Safety and as such not suitable to decrease the occurrence probability (3-->2)? The new 14971 clearly states that "Training to users can be an important aspect of delivering information for safety", so I am reluctant to use this control measure to decrease risk index values.
 

Tidge

Trusted Information Resource
#4
I wouldn't necessarily consider training to be an IFS risk control, unless the manufacturer of record controls all aspects of training (materials, trainer certifications, student assessments, etc.) and somehow guarantees that only trained and competent users are the ones using the device. It is rare that a manufacturer takes such a complete ownership of the training for a medical device in this manner.

I want to contrast this with the situation in which the device manufacturer has formally identified a specific class of users with particular training specifically applicable to the given device. The identification of user classes happens early in risk assessments and could impact the (initial) ratings of certain risks, but their specific training still would not be IFS. Most likely this would be in the "P2" rating (probability that a hazardous situation leads to a harm), but YMMV. If this pre-assessment factor is keeping risks acceptable, the medical device manufacturer has some obligation to ensure that only such formally trained users access the device, otherwise there will be a need to consider the class of untrained users and we will be back where we started.

I wouldn't use IFS to reduce the assessed value of risk, but I do account for IFS when performing risk benefit analyses.

I'm a little old school, with a hopefully solid bit of new school: The IFS is obligated to disclose risks that the manufacturer has (in the absence of a positive-outcome RBA) determined would otherwise be unacceptable, but it also can contain specific information that (when adhered to) contributes to the continued safe use of the device. An example of the former might be disclosing the potential allergy-sensitive materials in an implantable device where there is no alternate but to use such materials. An example of the latter might be listing cleaning products that are not to be used on a device because they could compromise one or more aspects of the device. In neither case would the manufacturer have fully 'designed away' certain specific risks, but in each case IFS can contribute to an improved RBA leading to acceptability.
 

Ronen E

Problem Solver
Moderator
#5
I think that in this case the training discussed is not "normal" training provided to users.
In this case the "product" is a service. The training discussed is part of "manufacturing the product" (the service), because it's not provided to users of the service (e.g. surgeons), but to employees providing/enabling/creating the service (service engineers).

Please also don't mix provision of information for safety with disclosure of residual risks. They are not exactly in the same regulatory status.
 
Thread starter Similar threads Forum Replies Date
A ISO 13485 Calibration Requirements- Assess and Record validity Previous measuring results ISO 13485:2016 - Medical Device Quality Management Systems 5
E ISO 13485 7.6 Control of monitoring and measuring equipment - Assess the Validity ISO 13485:2016 - Medical Device Quality Management Systems 4
B How to assess which directives and standards apply - Brewing and distilling products CE Marking (Conformité Européene) / CB Scheme 1
S Medical Device Changes and How to Assess Their Impacts to Essential Requirements EU Medical Device Regulations 2
T Method to assess Measurement System of a Continuous Process Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
Ronen E Independence on Notified Bodies, inter alia of Manufacturers they Assess EU Medical Device Regulations 7
M BS 10008:2014 Registration - UKAS accredited company in the UK to assess our company IEC 27001 - Information Security Management Systems (ISMS) 4
G How to assess Customer Satisfaction without using surveys ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 14
L How to Assess Adhesion of Stickers on Painted Steel Manufacturing and Related Processes 5
V How to Assess Process Capability at the Development or Optimization Phase Capability, Accuracy and Stability - Processes, Machines, etc. 6
A Cnpk (Non-Parametric Capability Analysis) to assess whether the Process is Stable Capability, Accuracy and Stability - Processes, Machines, etc. 7
S Required to Assess and Evaluate all Suppliers? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
R How to assess capability for parameters which has got either Min values or Max values Statistical Analysis Tools, Techniques and SPC 2
R CQI-15 Welding System Assess. & CQI-17 Special Process: Soldering Various Other Specifications, Standards, and related Requirements 6
Q Maintenance Process Audit Check List to Assess Maintenance Process ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
V Criteria used to assess the Validity of a Complaint - Insurance Company Customer Complaints 4
somashekar Non Moving Electronic components - How to assess its fitness for use. Manufacturing and Related Processes 3
B GM's QSB - Requirement, Checklists needed - Checklist used by GM to assess & sample Customer and Company Specific Requirements 9
S Advice on how to assess competency in the absence of academic qualifications? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 24
Z How to implement "assess" mentioned in the clause 7.1.4? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
B Designing a Test Part to Assess CMM Operator's Knowledge Design and Development of Products and Processes 9
M Method to assess / calculate a design wrt DFM/DFA - Seeking template Design and Development of Products and Processes 4
B How to sufficiently self-assess a QM (Quality Management) system? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
P How to Assess the Effectiveness of the QMS (Quality Management System) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
apestate Assess effects of MT&E found out of tolerance - ISO9001 Section 7.6 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
K What are You doing to assess Customer Satisfaction? Please help! ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 24
S 17025 Flexible Scopes of Accreditation - Laboratory to 'self-assess' new test method ISO 17025 related Discussions 2
D Minimum number of Subgroups to assess Process Capability Statistical Analysis Tools, Techniques and SPC 3
C How to assess Process Capability for parallelism Statistical Analysis Tools, Techniques and SPC 8
M Ford QOS - 3rd Party Auditing body able to assess the QOS Assessment? QS-9000 - American Automotive Manufacturers Standard 1
Y Must Initial importers have any QMS? US Food and Drug Administration (FDA) 1
M How to apply IEC 62304 Problem Resolution Process for bugs discovered prior to release i.e. during initial development? IEC 62304 - Medical Device Software Life Cycle Processes 2
S Initial PFMEA FMEA and Control Plans 6
M MDSAP required for Device Initial Importer/Distributor into Canada? Other Medical Device Regulations World-Wide 11
M Initial Importer/Distributor and Software Validation IEC 62304 - Medical Device Software Life Cycle Processes 1
J Stage 2 audit initial cert, few data points ISO 13485:2016 - Medical Device Quality Management Systems 4
K Initial capability studies on similar parts APQP and PPAP 7
I If i do not want to be an initial importer should i register with FDA? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
U Is Initial Importer Status Required if a Medical Device is Manufactured and Sterilized by an OEM in the US Other US Medical Device Regulations 1
J Furnace repaired - Do I need a new initial TUS? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
I IQOQ or just initial calibration required? General Measurement Device and Calibration Topics 3
K FDA - Can we have more than 1 Initial Importer Medical Device and FDA Regulations and Standards News 4
J How much to charge for helping a startup company with initial ISO 13485 certification? Consultants and Consulting 3
J ISO 13485 System 'soft start' - How to best reflect this in initial audits, management review minutes and other records? ISO 13485:2016 - Medical Device Quality Management Systems 3
H CQI-15 2nd Edition - Date of initial assessment? IATF 16949 - Automotive Quality Systems Standard 3
S Private Label Requirements, Manufacturer's Registration & Initial Importers 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
S Medical Device Initial Importer (DII) Requirements - Delivery Address 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
J AIAG PPAP: 2.2.11.3 Acceptance Criteria for Initial Study (page 9, 4th edition) APQP and PPAP 1
L Why is initial production units, lots, or batches, or their equivalents required for Design Validation? US Food and Drug Administration (FDA) 3
S Initial Importer and ISO 13485 (MDSAP) - Will the US company be included? ISO 13485:2016 - Medical Device Quality Management Systems 1

Similar threads

Top Bottom