SBS - The Best Value in QMS software

Evaluation of the effectiveness of correction actions

Uriel Alejandro

Involved In Discussions
#1
Hello Everybody,

I´m making some updates to my corrective actions procedure and during this process I´m having some problems trying to define a correct method for the evaluation of the efectiveness of a implemented action.

I the past, I managed to solve the effectiveness requirement declaring that the way to measure it was through recurrency: If the problem never happened again the action was efective.

I think that approach was correct, but there was always a debate about the evidence to prove it. I used to show the master list of correction action as an evidence.

Recently, I received a suggestion of an auditor to include as a part of the SCAR form an space to capture de criteria of effectivenes for each action and then ask for evidence of them. For me it seems like too much.

Is there any advice you can give me on this topic?

Thank you in Advance
 
Elsmar Forum Sponsor

normzone

Trusted Information Resource
#2
One persons effective corrective action is another person's "we been lucky so far".

Each case may merit it's own rationale - if you're never going to buy that component again then the effectiveness may be solid.

If the process was not changed the likelihood of effectiveness is lessened.

I just stick my neck out and write a brief summary of my opinion of how effective it appears to be, based on evidence available.

Occasionally I am forced to admit that the C/A was ineffective, or only partially so, because we could not obtain enough data and were reduced to addressing likely or potential root causes in an effort to reduce risk.
 

qualprod

Trusted Information Resource
#3
It depends of the situation.

For example a CA for a machine failure, root cause was found, determined action plans to fix the problem. Was the criteria to close out this CA, it could be that machine will be running a complete week in order to be convinced actions were effective.

Other examples:
Lack of competence in some people, problem found in an audit. Well root cause was found, and plans were implemented, when to evaluate effectiveness.
It could be measure it two months after having verified problems were not present due to people competence.
Hope this helps.
 

Uriel Alejandro

Involved In Discussions
#4
It depends of the situation.

For example a CA for a machine failure, root cause was found, determined action plans to fix the problem. Was the criteria to close out this CA, it could be that machine will be running a complete week in order to be convinced actions were effective.

Other examples:
Lack of competence in some people, problem found in an audit. Well root cause was found, and plans were implemented, when to evaluate effectiveness.
It could be measure it two months after having verified problems were not present due to people competence.
Hope this helps.
Thank you, for your feed back

Just a question for curiosity, in the second scenario, would you leave the CA opened for the two months period defined? or close it after make the verification as an external process?
 

qualprod

Trusted Information Resource
#5
Sometimes cas, remain open for several months.
The idea is to keep them opened til you are convinced
Action were implemented and resulted effective after a verification.
Normally after the process was running enough time as you are convinced now runs well.
Regards
 

Randy

Super Moderator
#6
How about you do what works for you that meets the requirements of your system and the standard in question and tell the auditor if he wants more to provide a requirement to do so?
 

John Predmore

Trusted Information Resource
#7
If the problem never happened again the action was effective. Thank you in Advance
How long do you wait to prove something can’t happen again?

If problem occurrences follow a Poisson distribution, and if the past is indicative of the future, the length of time between occurrences can be predicted using the Exponential function. For example, if the Mean Time Between Failures (MTBF) is 123 hours, the likelihood of a failure in any one hour (the lambda parameter, 1/MTBF) is 0.00813. In Excel, the cumulative probability that a problem will occur in the next 123 hours is EXPONDIST(123,0.00813,TRUE)=63.2%. (Because the Exponential is not bell-shaped, the probability of the mean is not 50%.)

You want the inverse problem, for a given level of confidence, what is the length of time when at least one event is statistically expected to occur? Excel does not provide an inverse function for EXPONDIST, but my textbook states the inverse function is -LN(alpha)/lambda. For my example, for 90% confidence, -LN(1-0.9)/.00813 = 283 hours.

If your MTBF is in weeks rather than hours, you may not want to wait 283 weeks (5.5 years) for an effectiveness review to close your CAPA. In that case, if you know enough about the problem to recreate the problem occurrence, you could validate your solution with a designed experiment to show your solution was effective.
 
#8
Hello Everybody,

I´m making some updates to my corrective actions procedure and during this process I´m having some problems trying to define a correct method for the evaluation of the efectiveness of a implemented action.

I the past, I managed to solve the effectiveness requirement declaring that the way to measure it was through recurrency: If the problem never happened again the action was efective.

I think that approach was correct, but there was always a debate about the evidence to prove it. I used to show the master list of correction action as an evidence.

Recently, I received a suggestion of an auditor to include as a part of the SCAR form an space to capture de criteria of effectivenes for each action and then ask for evidence of them. For me it seems like too much.

Is there any advice you can give me on this topic?

Thank you in Advance
It's simple! Take a look at the problem statement in the corrective action request. Then, answer, has this problem gone away? If time shows no recurrence, the CA was good. If your local town has a problem with speeding drivers and installs traffic calming devices - rumble strips, narrow lanes, signs etc. then the speed of the cars will drop if these measures are effective, as long as they remain in place.
 

QuinnM

Involved In Discussions
#9
Hi Uriel,

If I understand correctly the auditor suggested to capture de criteria of effectiveness of each action. From this statement, I do not think the auditor was questioning your approach, but rather the documentation activities. Capturing the effectiveness of each activity, and over all CAPA should be documented. I do not know what quality system you use, and my experience is from 13485 and 21 CFR 820. See below.

ISO 13485:2016, section 8.5.2 states: “The organization shall document a procedure to define requirements for: …f) reviewing the effectiveness of corrective action taken. Records of the results of any investigation and of action taken shall be maintained (see 4.2.5)”

21 CFR 820.100 Corrective and Preventative Action: “(4) Verifying or validating the corrective and preventive action to ensure that such action is effective and does not adversely affect the finished device;” and “(b) All activities required under this section, and their results, shall be documented.”

Maybe you do not have an issue with defining the correct method, but only need to document the activities?

All the best,
Quinn
 

Uriel Alejandro

Involved In Discussions
#10
How long do you wait to prove something can’t happen again?

If problem occurrences follow a Poisson distribution, and if the past is indicative of the future, the length of time between occurrences can be predicted using the Exponential function. For example, if the Mean Time Between Failures (MTBF) is 123 hours, the likelihood of a failure in any one hour (the lambda parameter, 1/MTBF) is 0.00813. In Excel, the cumulative probability that a problem will occur in the next 123 hours is EXPONDIST(123,0.00813,TRUE)=63.2%. (Because the Exponential is not bell-shaped, the probability of the mean is not 50%.)

You want the inverse problem, for a given level of confidence, what is the length of time when at least one event is statistically expected to occur? Excel does not provide an inverse function for EXPONDIST, but my textbook states the inverse function is -LN(alpha)/lambda. For my example, for 90% confidence, -LN(1-0.9)/.00813 = 283 hours.

If your MTBF is in weeks rather than hours, you may not want to wait 283 weeks (5.5 years) for an effectiveness review to close your CAPA. In that case, if you know enough about the problem to recreate the problem occurrence, you could validate your solution with a designed experiment to show your solution was effective.
Thank you John, that´s an awesome way to manage it and a good advice. I will take a look to my records to see if have enough data to calculate something like this.
 
Thread starter Similar threads Forum Replies Date
A CAPA - Reasonable Timeframes for Effectiveness Evaluation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
H LPA's (Layered Process Audits) - Evaluation of Effectiveness Process Audits and Layered Process Audits 22
B Training Effectiveness Evaluation - Different Styles - In-House and External Document Control Systems, Procedures, Forms and Templates 21
Raffy How do I measure Measuring Training Effectiveness? Performance evaluation? Other ISO and International Standards and European Regulations 26
M Selling non-CE marked devices for evaluation EU Medical Device Regulations 4
M Procedure for clinical evaluation according to new MDR EU Medical Device Regulations 1
Q Process map Evaluation and Analysis Method Process Maps, Process Mapping and Turtle Diagrams 5
M Supplier evaluation Supplier Quality Assurance and other Supplier Issues 5
I QMS monitoring, measurement, analysis and evaluation requirement - Template ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
T Biological Evaluation (10993) & Risk Management ISO 14971 - Medical Device Risk Management 9
shimonv Clinical evaluation report for class I device EU Medical Device Regulations 3
A Applicability of Photobiological Safety Evaluation for LED used in medical devices Reliability Analysis - Predictions, Testing and Standards 2
P GSPRs / Clinical Evaluation EU Medical Device Regulations 3
silentmonkey Overall Benefit/Risk Analysis - Risk Management VS Clinical Evaluation ISO 14971 - Medical Device Risk Management 3
M NICE Medical Technology Evaluation Programme - Recommendations Service Industry Specific Topics 0
Q Summative Usability Evaluation Testing: prior or during Clinical Investigation? Human Factors and Ergonomics in Engineering 9
S Regular updates of clinical evaluation report EU Medical Device Regulations 6
K Surface finish (Evaluation Length) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
M Clinical Evaluation Benchmark vs. Equivalent EU Medical Device Regulations 2
M IVDR and Performance Evaluation Plan CE Marking (Conformité Européene) / CB Scheme 2
S User evaluation for self monitoring blood glucose test systems US Food and Drug Administration (FDA) 4
S Australia TGA Clinical Evaluation Report (CER) Other Medical Device Regulations World-Wide 0
S API Spec. Q1 clause 5.6.1.2 On site evaluation Oil and Gas Industry Standards and Regulations 10
B Biological evaluation plan and report Other Medical Device Related Standards 6
F Biocompatibility evaluation for Hardware/Interface Components? Other Medical Device Related Standards 12
B Clinical Evaluation Expert Panels - MDR EU Medical Device Regulations 1
T Clinical evaluation of a new medical device EU Medical Device Regulations 0
dgrainger Informational MDCG 2020:13 - Clinical evaluation assessment report template EU Medical Device Regulations 0
S A clinical performance evaluation study with an IVD product as Investagional Use product - Clinical Monitor requirements 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
B AS9100 8.4.1 Supplier Selection/Evaluation criteria and reevaluations AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 13
R Clinical evaluation without clinical data - MDR Article 61(10) EU Medical Device Regulations 8
M Clinical Evaluation Plan vs. PMCF Plan EU Medical Device Regulations 33
K Biocompatibility evaluation of gas pathways Medical Device and FDA Regulations and Standards News 5
T HF testing / Summative evaluation for MDDS class I necessary? Human Factors and Ergonomics in Engineering 2
N Evaluation of service providers Supplier Quality Assurance and other Supplier Issues 2
M Can someone share a scrubbed version of Clinical Evaluation Plan (CEP) EU Medical Device Regulations 8
S EU MDR Annex XIV - Clinical Evaluation Plan - What do these methods mean? EU Medical Device Regulations 16
C Material from outside CER evaluation period CE Marking (Conformité Européene) / CB Scheme 8
DitchDigger IEC 60601-1 subclause 5.1 - Adequate evaluation in lieu of testing IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
S Clinical Evaluation - Is this an ISO 13485:2016 requirement? ISO 13485:2016 - Medical Device Quality Management Systems 4
P SOUP anomaly evaluation for MMA (Mobile Medical Application) IEC 62304 clause 7.1.3 IEC 62304 - Medical Device Software Life Cycle Processes 6
M Informational How to perform a clinical evaluation of medical devices – Part 2 – Level of clinical evidence and what sufficient clinical evidence means Medical Device and FDA Regulations and Standards News 9
P IEC 62304 - evaluation of integration and system testing IEC 62304 - Medical Device Software Life Cycle Processes 4
R Supplier evaluation and business needs in the context of ISO 13485 ISO 13485:2016 - Medical Device Quality Management Systems 2
U Changes to Internal Processes and Risk Evaluation - Mitigations Risk Management Principles and Generic Guidelines 10
M Informational Work in progress at the FDA for biological evaluation – Color Hazard and RISk calculator (CHRIS) Medical Device and FDA Regulations and Standards News 0
M Informational How to perform a clinical evaluation of medical devices – Part 1 – Overview and sample of activities Medical Device and FDA Regulations and Standards News 0
T EU MDR Article 61- Clinical Evaluation EU Medical Device Regulations 2
pashah Looking for Clinical Evaluation SOP acc. MEDDEV and EU MDR Other Medical Device Related Standards 1
K Supplier re-evaluation (API Q1) Manufacturing and Related Processes 9

Similar threads

Top Bottom