How to Validate Effectiveness of Mitigations?

[david316]

Earlier this year I prepared training material on the changes between the 2000 and 2007 version of ISO 14971. During this process one of things that I realised was that the standard has carefully worded requirements for records. In some places you need objective evidence or more details, in others, just the results (yes/no) are OK.

For example Clause 6.3 covers both "implementation" and "effectiveness" of risk control measures. Both need to be "verified" (which is a defined term, meaning gaining "objective evidence").

However, while you need to keep objective evidence for implementation as a record, for effectiveness only the result (effective / not effective) needs to be recorded. The objective evidence of effectiveness can be thrown away and you still comply with ISO 14971.

I believe this is deliberate. While keeping objective evidence of effectiveness sounds nice, it quickly gets messy in practice. I guess from a regulatory point of view what they are saying is that at least someone should take responsibility for saying that a risk control measure is effective.

I would agree with Anna, a system for linking to objective evidence is reasonable, but keep it optional, used when it makes sense to keep more records, such as borderline cases, or cases which effectiveness might be reasonably questioned by a regulatory reviewer.

Thanks Peter. So to record effectiveness once verification that the risk mitigator are implemented, can it be as simple as a stating in a design review that there is agreement that the risk mitigators are effective?

 

[yodon]

Great discussion! For some controls, it may be possible to have objective evidence of effectiveness before going to the field, particularly those that are 'inherent safety by design' (e.g., controls to prevent free-flow when opening the door on an infusion pump can be readily demonstrated as effective through verification testing). It certainly gets cloudier with protective measures and completely impossible with information for safety (even a summative study is just limited in terms of study subjects and conditions).

Seems like, before release, when no clinicals were required, you can conclude they're believed to be effective based on existing test evidence, reviewers' opinions, and maybe some expert feedback. Continual assessment of postmarket data is then needed to substantiate the claim.

 

[david316]

I guess my question is what is the most practical way of recording the risk mitigators are effective and how much effort needs to go into this. For example, if we follow a particular standard do we just assume it is effective or do we have to explicitly state it is effective based on the assumption the particular standard is effective (even if it may not be recognised by the FDA)? Alternatively what approach do we take if we rely on the expertise of the risk management team as it is difficult to objectively demonstrate effectiveness. For example, a moving part represents a crush hazard to a users hand so the risk management team decides adding a hydraulic shock absorber to slow down the moving part down to specific speed would reduce the risk to an acceptable level. We can verify the shock absorber reduces the speed appropriately but the hazard is still there even if there is sufficient time for the user to remove their hand, how would we document the risk control is effective in this case and how much effort should we put in?

Thanks

 

[Peter Selvey]

Most risk management tables have "pre" and "post" values for the estimated probability/severity/risk. The "post" values act as the formal record that the risk control was effective (since the numbers there are usually below the criteria for acceptable risk).

Keep in mind this is the minimum requirements from ISO 14971 which has to cover a huge variety of situations.

In the case of critical situations or borderline it makes sense to keep some more records, especially to make a clear record of who is really taking responsibility. Ironically though, those are the very cases where manufacturer tends to keep a hidden folder under the desk