What do you think of Chat GPTs answer to this Risk Acceptability question?

[d_addams]

Specifically how ISO 14971 isn't the end all be all to risk acceptability. I think many young engineers really struggle with there not being an explicit answer even though ISO 14971 seems to imply there is. Thoughts?

Question Posed to Chat GPT:
Does the manufacturer of the <product name> have to set a predetermined risk acceptability level? I thought ISO 14971 required setting risk acceptability criteria?

Chat GPTs Answer:
ISO 14971 is an international standard that provides guidance on the application of risk management to medical devices. The standard does require that manufacturers of medical devices establish and maintain a risk management process, which includes identifying hazards, assessing the risks associated with those hazards, and implementing measures to control or mitigate those risks.

As part of this process, manufacturers are expected to establish risk acceptability criteria, which are used to determine whether the residual risks associated with a medical device are acceptable. These criteria will depend on factors such as the intended use of the device, the population being treated, and the benefits of the device. In the case of <product name>, the manufacturer will have to follow the ISO 14971 standard and establish the risk acceptability criteria for this device.

It's also worth mentioning that the FDA in the US has similar regulations for medical devices, where manufacturers are expected to follow the Quality System Regulation (QSR) which includes a risk management process, similar to what ISO 14971 standard requires.

It's important to note that the risk acceptability criteria set by the manufacturer are not necessarily the only criteria used to evaluate the safety and effectiveness of a medical device. They will also be reviewed and evaluated by regulatory bodies such as the FDA and other relevant authorities, to ensure that the device is safe and effective for its intended use.

 

[Ed Panek]

ChatGPT does better the more structured the subject is, so topics like legal, medical and regulatory are usually well presented. ChatGPT has access to the standards as well as blogs, presentations, etc on those topics. This creates some variance on the topic but generally regulatory answers are better than more subjective things like questions about rapidly evolving events like the war in Ukraine. If you were to ask about the history of Ukraine though it would again be solid.

V 4.0 is due out and it will have something like 10 trillion parameters while v 3.0 has only billions.

When thinking about risk consider something like a CT machine. Every time a person uses a CT machine they are exposed to ionizing radiation increasing their risk of cancer, and yet CT machines are used everywhere. How is that possible if they cause cancer?

The final product has something called residual risk that cannot be 100% eliminated as radiation is a core imaging mechanism for CT, so a risk benefit is the final decision tool.

Going backwards from that are all the mitigations to reduce exposure (There are many other risks electrical, physical but lets focus on radiation) If this is the first-ever CT created some of the risks are estimates but if this is the 20th generation there is a ton of actual data to calculate risks.

Also, the risk file is living set of documents. Some risks may be over or understated previously and need more or new controls.

Regulatory bodies may disagree with your risk values but keep the rankings logical (Radiation is a higher risk than an unpleasant noise) (A false negative vs false positive from imaging) etc.

 

[Tidge]

The "answer" from ChaptGPT sounds a little too wishy-washy... that is, a human expert might share the same details but would have answered "yes" to the first question. Any equivocating by a human on this question would have their phone number removed from speed-dial.

Out of curiosity, what does ChaptGPT have to say if you cut-and-paste the OP into it? That is: what does it say when it is asked to self-evaluate its own response?

 

[RoxaneB]

ChatGPT will refine its answer if you continue your query within the thread - it's like having a debate or discussion with it. This is an opportunity to refine your query and also to point flaws in ChatGPT's responses

I recently did this with a query on a particular term - ChatGPT indicated that my term was not typically used. I pointed out that it was typically used within a particular geographic zone which it then acknowledged. I then said that its first response then could have indicated this and ChatGPT's reponse was "I apologize for the confusion. You are correct that the...I should have made it clear that <the term> is an abbreviation used by <geographic area>...Thank you for bringing this to my attention."

If you are dissatisfied with the response from ChatGPT, you may wish to consider continuing the conversation within its thread, pointing out possible flaws in its logic, and/or refining your question.

 

[Ed Panek]

The challenge neural networks have is in balancing speed and learning rate. A very large network may provide more accurate results but as they are very large finding the best answer will take a longer time. A very low learning rate will mean that outliers are never addressed in the answers. A high learning rate means older information is replaced quite easily so the AI presents its most recent data as correct. Because laws and regulations always change that might be a better approach.

I think in the future what will happen is a network will offload the question to another network specialized in that subject. Maybe a legal/regulatory network focused just on that specific topic that is very responsive to that subject area but knows nothing about anything else. That may already be in place though, I haven't looked

 

[GalitSa]

See what i got from ChatGPT:
Relationship between ISO 14971 and ISO 13485
ISO 14971 and ISO 13485 are two international standards that are closely related to each other, but they have different purposes and scopes.
ISO 14971:2019 specifies the requirements for a risk management process for medical devices. It provides guidance on how to identify, assess, and control risks associated with the use of medical devices throughout their lifecycle.
On the other hand, ISO 13485:2016 specifies the requirements for a quality management system (QMS) for medical devices. It provides guidance on how to establish and maintain a QMS that ensures the safety and effectiveness of medical devices.
ISO 14971 and ISO 13485 are related in that ISO 14971 can be used as a tool to meet the risk management requirements of ISO 13485. ISO 13485 requires medical device manufacturers to establish and maintain a risk management process that is appropriate for the nature of the medical device and that meets the requirements of applicable regulatory authorities. By following the risk management process outlined in ISO 14971, medical device manufacturers can demonstrate compliance with the risk management requirements of ISO 13485.
In addition, ISO 14971 can be used as a tool to support compliance with regulatory requirements in various countries, including the European Union's Medical Device Regulation (MDR) and In Vitro Diagnostic Regulation (IVDR) and the US Food and Drug Administration (FDA) guidance documents.
Overall, while ISO 14971 and ISO 13485 have different scopes and purposes, they are closely related and complementary to each other. Medical device manufacturers should ensure that their risk management process aligns with their QMS to ensure the safety and effectiveness of their medical devices.
Risk-benefit analysis
Risk-benefit analysis is an essential component of the risk management process outlined in ISO 14971:2019. It requires medical device manufacturers to assess the potential benefits of the device in addition to its risks. The risk-benefit analysis helps to ensure that the benefits of the device outweigh its risks.
The risk-benefit analysis involves the following steps:
Identify the potential benefits of the device: Medical device manufacturers must identify all potential benefits associated with the use of the device. These benefits can be clinical or non-clinical, and they must be identified based on the intended use of the device.
Identify the potential risks of the device: Medical device manufacturers must identify all potential risks associated with the use of the device. These risks can be clinical or non-clinical, and they must be identified based on the intended use of the device.
Evaluate the benefits and risks: Medical device manufacturers must evaluate the potential benefits of the device in relation to its potential risks. This evaluation requires a comprehensive analysis of the potential benefits and risks, considering all relevant factors.
Determine the overall acceptability of the risks: Medical device manufacturers must determine the overall acceptability of the risks associated with the device. This determination requires an evaluation of the severity of the risks and the likelihood of their occurrence.

Determine the need for risk control measures: Based on the evaluation of the risks and benefits, medical device manufacturers must determine the need for risk control measures. If the risks outweigh the benefits, risk control measures must be implemented to mitigate the risks.
Overall, the risk-benefit analysis is a critical component of the risk management process, ensuring that the potential benefits of the device outweigh its risks. Medical device manufacturers must conduct a comprehensive risk-benefit analysis to ensure that their devices are safe and effective for their intended use.
Risk-benefit determination
Benefit-risk determination is the process of evaluating the potential benefits and risks associated with the use of a medical device and determining whether the benefits outweigh the risks. This process is a critical component of the risk management process outlined in ISO 14971:2019.
The benefit-risk determination process involves the following steps:
Identify the potential benefits: Medical device manufacturers must identify all potential benefits associated with the use of the device. These benefits can be clinical or non-clinical and must be identified based on the intended use of the device.
Identify the potential risks: Medical device manufacturers must identify all potential risks associated with the use of the device. These risks can be clinical or non-clinical and must be identified based on the intended use of the device.
Evaluate the benefits and risks: Medical device manufacturers must evaluate the potential benefits of the device in relation to its potential risks. This evaluation requires a comprehensive analysis of the potential benefits and risks, considering all relevant factors.
Determine the overall acceptability of the risks: Medical device manufacturers must determine the overall acceptability of the risks associated with the device. This determination requires an evaluation of the severity of the risks and the likelihood of their occurrence.
Determine the need for risk control measures: Based on the evaluation of the risks and benefits, medical device manufacturers must determine the need for risk control measures. If the risks outweigh the benefits, risk control measures must be implemented to mitigate the risks.
Document the benefit-risk determination: Medical device manufacturers must document the benefit-risk determination process and the conclusions reached, including the basis for the determination.
The benefit-risk determination process is essential for ensuring that medical devices are safe and effective for their intended use. By conducting a comprehensive benefit-risk analysis, medical device manufacturers can identify potential risks and benefits, determine the overall acceptability of the risks, and implement appropriate risk control measures to ensure that the benefits of the device outweigh its risks.
Production and post-production activities
ISO 14971:2019 outlines the risk management process for medical devices, including the production and post-production phases. The production phase includes all activities related to the manufacture and assembly of the medical device, while the post-production phase includes activities related to the distribution, use, and maintenance of the device.

During the production phase, medical device manufacturers must follow the risk management process outlined in ISO 14971:2019, including the following steps:

1. Identify hazards: Medical device manufacturers must identify all potential hazards associated with the production of the device, including hazards related to manufacturing processes and materials used.
2. Estimate the probability and severity of harm: Medical device manufacturers must estimate the probability and severity of harm associated with each identified hazard.
3. Evaluate the risks: Medical device manufacturers must evaluate the risks associated with each identified hazard to determine the overall risk associated with the production of the device.
4. Implement risk control measures: Medical device manufacturers must implement risk control measures to mitigate the identified risks associated with the production of the device.
5. Verify risk control measures: Medical device manufacturers must verify that the implemented risk control measures are effective in mitigating the identified risks.

Overall, the risk management process outlined in ISO 14971:2019 must be followed throughout the production and post-production phases of medical devices to ensure that they are safe and effective for their intended use. Medical device manufacturers must conduct a comprehensive risk management process, including identifying hazards, estimating the probability and severity of harm, evaluating the risks, implementing risk control measures, and verifying their effectiveness.
Usability and human factors
Usability and human factors are important considerations in the risk management process outlined in ISO 14971:2019. Usability is the degree to which a medical device can be used by specific users to achieve specific goals with effectiveness, efficiency, and satisfaction in a specific context of use. Human factors refer to the interaction between the user and the medical device, including design, layout, and other user-related factors.
In the risk management process, medical device manufacturers must consider usability and human factors to ensure that the device is safe and effective for its intended use. This includes the following steps:

1. Identify potential use errors: Medical device manufacturers must identify potential use errors that can occur as a result of poor usability or human factors.
2. Evaluate the severity and probability of harm: Medical device manufacturers must evaluate the severity and probability of harm associated with each potential use error.
3. Implement risk control measures: Medical device manufacturers must implement risk control measures to mitigate the identified risks associated with poor usability or human factors. These risk control measures may include design changes, labeling changes, or user training.
4. Verify the effectiveness of risk control measures: Medical device manufacturers must verify that the implemented risk control measures are effective in mitigating the identified risks associated with poor usability or human factors.
5. Document the usability and human factors assessment: Medical device manufacturers must document the assessment of usability and human factors and the risk control measures implemented to mitigate the identified risks.

Overall, considering usability and human factors is essential for ensuring that medical devices are safe and effective for their intended use. By conducting a comprehensive assessment of usability and human factors, medical device manufacturers can identify potential use errors, evaluate the associated risks, and implement appropriate risk control measures to ensure that the device is safe and effective for its intended use.
Labeling and instructions for use
Labeling and instructions for use are important considerations in the risk management process outlined in ISO 14971:2019. Medical device labeling provides information to users about the device, including its intended use, indications for use, contraindications, warnings, precautions, and instructions for use. Instructions for use provide guidance to users on how to properly use the device and should be clear and concise to ensure safe and effective use of the device.
In the risk management process, medical device manufacturers must consider the labeling and instructions for use to ensure that the device is safe and effective for its intended use. This includes the following steps:

1. Identify potential use errors related to labeling and instructions for use: Medical device manufacturers must identify potential use errors that can occur as a result of unclear or inadequate labeling and instructions for use.
2. Evaluate the severity and probability of harm: Medical device manufacturers must evaluate the severity and probability of harm associated with each potential use error related to labeling and instructions for use.
3. Implement risk control measures: Medical device manufacturers must implement risk control measures to mitigate the identified risks associated with unclear or inadequate labeling and instructions for use. These risk control measures may include revising the labeling and instructions for use, providing additional training to users, or conducting human factors testing.
4. Verify the effectiveness of risk control measures: Medical device manufacturers must verify that the implemented risk control measures are effective in mitigating the identified risks associated with unclear or inadequate labeling and instructions for use.
5. Document the labeling and instructions for use assessment: Medical device manufacturers must document the assessment of labeling and instructions for use and the risk control measures implemented to mitigate the identified risks.

Overall, considering labeling and instructions for use is essential for ensuring that medical devices are safe and effective for their intended use. By conducting a comprehensive assessment of labeling and instructions for use, medical device manufacturers can identify potential use errors, evaluate the associated risks, and implement appropriate risk control measures to ensure that the device is safe and effective for its intended use.


Post-market surveillance
Post-market surveillance is an important aspect of the risk management process outlined in ISO 14971:2019. It involves the ongoing monitoring of medical devices after they have been placed on the market to identify any potential safety issues or performance problems that were not identified during the pre-market testing and evaluation.
In the risk management process, medical device manufacturers must consider post-market surveillance to ensure that the device continues to be safe and effective for its intended use. This includes the following steps:

1. Establish a post-market surveillance plan: Medical device manufacturers must establish a post-market surveillance plan that outlines the activities to be conducted, including the methods for collecting and analyzing data, the frequency of data collection, and the criteria for evaluating the data.
2. Collect and analyze data: Medical device manufacturers must collect and analyze data from various sources, including complaints, adverse event reports, and other sources of information, to identify any potential safety issues or performance problems.
3. Evaluate the data: Medical device manufacturers must evaluate the data collected through post-market surveillance to determine if any changes need to be made to the device or its labeling or if any additional risk control measures need to be implemented.
4. Implement any necessary changes or risk control measures: Medical device manufacturers must implement any necessary changes to the device or its labeling or implement additional risk control measures to ensure that the device continues to be safe and effective for its intended use.
5. Document the post-market surveillance activities: Medical device manufacturers must document the post-market surveillance activities conducted, including any changes or risk control measures implemented.

Overall, post-market surveillance is an important aspect of the risk management process that helps ensure the ongoing safety and effectiveness of medical devices. By establishing a post-market surveillance plan, collecting and analyzing data, evaluating the data, and implementing any necessary changes or risk control measures, medical device manufacturers can identify and address any potential safety issues or performance problems that may arise after the device is placed on the market.
Consideration of other risks and other aspects of risk
ISO 14971:2019 requires medical device manufacturers to consider not only the clinical risks associated with the use of the device but also other risks and aspects of risk. These include:
Non-clinical risks: These are risks that may not directly affect the patient but can affect the safety and effectiveness of the medical device. For example, risks associated with the device's design, manufacture, labeling, or instructions for use.
Residual risks: These are risks that remain after risk control measures have been implemented. Medical device manufacturers must evaluate the overall residual risk and determine whether it is acceptable.
Benefits and risks: Medical device manufacturers must consider the benefits of the device in addition to its risks. This requires a benefit-risk analysis, where the potential benefits of the device are compared to its risks.
Usability and human factors: Medical device manufacturers must consider the usability of the device, including its design, labeling, and instructions for use, to ensure that users can use the device safely and effectively.
Labeling and instructions for use: Medical device manufacturers must provide clear and concise labeling and instructions for use to ensure that users can use the device safely and effectively.

Post-market surveillance: Medical device manufacturers must establish a system for monitoring the performance of the device in the post-market phase to identify and evaluate any new risks or issues that may arise.
Overall, the consideration of other risks and aspects of risk in addition to clinical risks is essential for ensuring the safety and effectiveness of medical devices throughout their lifecycle. Medical device manufacturers must take a comprehensive approach to risk management and consider all potential risks and benefits associated with the use of their device.
Interfaces between producers and other parties
In the context of ISO 14971:2019, there are several interfaces between medical device producers and other parties that need to be considered as part of the risk management process. These interfaces include the following:

1. Suppliers: Medical device producers must consider the risks associated with any components or materials supplied by third-party suppliers. This includes evaluating the supplier's quality system, assessing the potential risks associated with the supplied components or materials, and implementing risk control measures as necessary.
2. Users: Medical device producers must consider the risks associated with the use of the device by intended users. This includes evaluating the user's knowledge, skills, and experience, and providing appropriate labeling and instructions for use to minimize the risk of use errors.
3. Patients: Medical device producers must consider the risks associated with the use of the device by patients. This includes evaluating the potential risks associated with the device's design, labeling, and instructions for use, and implementing risk control measures to minimize the risk of harm to patients.
4. Regulatory authorities: Medical device producers must comply with regulatory requirements and communicate with regulatory authorities as necessary. This includes providing information on the risks associated with the device, implementing risk control measures, and reporting adverse events or other safety-related information as required.
5. Healthcare providers: Medical device producers must consider the risks associated with the use of the device by healthcare providers. This includes evaluating the healthcare provider's knowledge, skills, and experience, and providing appropriate training and education to minimize the risk of use errors.

Overall, considering the interfaces between medical device producers and other parties is essential for ensuring the safety and effectiveness of the device. By evaluating the risks associated with these interfaces and implementing appropriate risk control measures, medical device producers can minimize the risk of harm to users, patients, and other stakeholders, and ensure regulatory compliance.