How to Validate Effectiveness of Mitigations?

david316

Involved In Discussions
#11
Earlier this year I prepared training material on the changes between the 2000 and 2007 version of ISO 14971. During this process one of things that I realised was that the standard has carefully worded requirements for records. In some places you need objective evidence or more details, in others, just the results (yes/no) are OK.

For example Clause 6.3 covers both "implementation" and "effectiveness" of risk control measures. Both need to be "verified" (which is a defined term, meaning gaining "objective evidence").

However, while you need to keep objective evidence for implementation as a record, for effectiveness only the result (effective / not effective) needs to be recorded. The objective evidence of effectiveness can be thrown away and you still comply with ISO 14971.

I believe this is deliberate. While keeping objective evidence of effectiveness sounds nice, it quickly gets messy in practice. I guess from a regulatory point of view what they are saying is that at least someone should take responsibility for saying that a risk control measure is effective.

I would agree with Anna, a system for linking to objective evidence is reasonable, but keep it optional, used when it makes sense to keep more records, such as borderline cases, or cases which effectiveness might be reasonably questioned by a regulatory reviewer.
Thanks Peter. So to record effectiveness once verification that the risk mitigator are implemented, can it be as simple as a stating in a design review that there is agreement that the risk mitigators are effective?
 
Elsmar Forum Sponsor

yodon

Staff member
Super Moderator
#12
Great discussion! For some controls, it may be possible to have objective evidence of effectiveness before going to the field, particularly those that are 'inherent safety by design' (e.g., controls to prevent free-flow when opening the door on an infusion pump can be readily demonstrated as effective through verification testing). It certainly gets cloudier with protective measures and completely impossible with information for safety (even a summative study is just limited in terms of study subjects and conditions).

Seems like, before release, when no clinicals were required, you can conclude they're believed to be effective based on existing test evidence, reviewers' opinions, and maybe some expert feedback. Continual assessment of postmarket data is then needed to substantiate the claim.
 

david316

Involved In Discussions
#13
I guess my question is what is the most practical way of recording the risk mitigators are effective and how much effort needs to go into this. For example, if we follow a particular standard do we just assume it is effective or do we have to explicitly state it is effective based on the assumption the particular standard is effective (even if it may not be recognised by the FDA)? Alternatively what approach do we take if we rely on the expertise of the risk management team as it is difficult to objectively demonstrate effectiveness. For example, a moving part represents a crush hazard to a users hand so the risk management team decides adding a hydraulic shock absorber to slow down the moving part down to specific speed would reduce the risk to an acceptable level. We can verify the shock absorber reduces the speed appropriately but the hazard is still there even if there is sufficient time for the user to remove their hand, how would we document the risk control is effective in this case and how much effort should we put in?

Thanks
 

Peter Selvey

Staff member
Super Moderator
#14
Most risk management tables have "pre" and "post" values for the estimated probability/severity/risk. The "post" values act as the formal record that the risk control was effective (since the numbers there are usually below the criteria for acceptable risk).

Keep in mind this is the minimum requirements from ISO 14971 which has to cover a huge variety of situations.

In the case of critical situations or borderline it makes sense to keep some more records, especially to make a clear record of who is really taking responsibility. Ironically though, those are the very cases where manufacturer tends to keep a hidden folder under the desk ;)
 
Thread starter Similar threads Forum Replies Date
A How to verify and validate the effectiveness of CAPA with regards to employees ? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
L How to determine / validate Medical Device Storage Conditions ISO 13485:2016 - Medical Device Quality Management Systems 1
Anonymous16-2 21 CFR Part 11 - Steps to take if we want to validate an electronic system Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 2
T How to “validate/qualify” a new measurement equipment? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
R Validation of Medical Device Hardware containing Software - How many to Validate ISO 13485:2016 - Medical Device Quality Management Systems 1
S Do we need to validate Software used in Drug discovery and development process? Qualification and Validation (including 21 CFR Part 11) 2
kisxena Memo to File - Failure to Validate Supply Chain Security Management Systems 1
A ISO 13485:2016 - Validate our use of software that impacts on the QMS ISO 13485:2016 - Medical Device Quality Management Systems 8
A Process to Validate the Accuracy of Test Reports (AS9100) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 38
C Do we need to Validate Off the Shelf Software? Manufacturing and Related Processes 5
M How to validate hundreds or even thousands of parts? Qualification and Validation (including 21 CFR Part 11) 7
B How to validate Microsoft Excel Spreadsheet Calculations for the Cpk value Capability, Accuracy and Stability - Processes, Machines, etc. 9
M How to validate a system to make it compliant with 21 CFR Part 11 Qualification and Validation (including 21 CFR Part 11) 2
B Do I need to validate my database software? Calibration and Metrology Software and Hardware 1
K COTS Validation: Should we validate Windows after each Update? IEC 62304 - Medical Device Software Life Cycle Processes 3
S How to validate a homegrown software 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
A When to verify and when to validate Design Changes 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
H How To Validate Electronic Part From Non Authorized Distributor Process Maps, Process Mapping and Turtle Diagrams 7
G Prototype Manufacturing Equipment - Validate or not to validate? Quality Manager and Management Related Issues 2
S How to validate Device Shelf Life? CE Marking (Conformité Européene) / CB Scheme 8
M Destructive VTMV Subgroups - A Destructive Variable Test Method to Validate Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
M How to validate a Dye Test method used to evaluate Package Integrity Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 8
I When to Validate and when not to Validate US Food and Drug Administration (FDA) 24
C How do you Validate Solderability of Components Prior to Soldering Operations Manufacturing and Related Processes 5
M Welding - to Validate or Not? If so, How? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
V Do we need to Validate a Server/Back-End System? Qualification and Validation (including 21 CFR Part 11) 2
I Using validated SAS JMP to validate Excel for Statistical Analysis? Qualification and Validation (including 21 CFR Part 11) 11
E How to Validate Master Equipment that has been brought to onsite General Measurement Device and Calibration Topics 12
E When to Validate a Process - 21 CFR Part 820.75.(a) 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 8
R How to bond PVC (Tygon) Tubing and Validate the Process ISO 13485:2016 - Medical Device Quality Management Systems 13
T Do I need to validate a basic Excel spreadsheet Quality Assurance and Compliance Software Tools and Solutions 10
R Process Validation - Adhesive Bonding, what to validate and how? Qualification and Validation (including 21 CFR Part 11) 10
N How to Validate a Gage R&R Spread Sheet General Measurement Device and Calibration Topics 3
C Validate the idea of a Composite Performance Index Quality Tools, Improvement and Analysis 4
M How to determine/validate the "REQUIRED MANPOWER" in an office environment? Service Industry Specific Topics 4
R How to Validate Vision Inspection Systems Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
C Third-party lab needed to validate cleaning & disinfection? EU Medical Device Regulations 12
Proud Liberal How to validate the geometry of a 8 mm pitch, 22 tooth Gates HTD sprocket Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
T How does a tool-shop validate molds? Verification vs. Validation ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Validate 99.5% reliability 90% confidence - Sample size calculation Reliability Analysis - Predictions, Testing and Standards 4
J Protocol to validate the use of Electronic Records like DHRs, NCMRs, Validations ISO 13485:2016 - Medical Device Quality Management Systems 3
N Periodically validate test reports for raw material - AS9100 7.4.3 AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 15
V Calibration of a Digital Vernier - Need to check/calibrate/validate all 3 functions? General Measurement Device and Calibration Topics 9
P Last-off-part comparison used to validate the set-up (TS16949 Clause 7.5.1.3) IATF 16949 - Automotive Quality Systems Standard 3
E How to validate product if gage is determined out of calibration Document Control Systems, Procedures, Forms and Templates 19
M Digital Filling System - What do I need to Validate Here? Software Quality Assurance 1
M Seat Cover Product Validation for PPAP - How can I validate my product? APQP and PPAP 12
N Do we need to Validate Outsouced Processes? ISO13485 ISO 13485:2016 - Medical Device Quality Management Systems 12
E GMW3172 - Thermal Shock and PTC cycles required to validate an electronic system Reliability Analysis - Predictions, Testing and Standards 2
A Software Observation during ISO 9001 Audit - Validate and Re-validate Software ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13

Similar threads

Top Bottom