Evaluation of the effectiveness of correction actions

[Uriel Alejandro]

Hello Everybody,

I´m making some updates to my corrective actions procedure and during this process I´m having some problems trying to define a correct method for the evaluation of the efectiveness of a implemented action.

I the past, I managed to solve the effectiveness requirement declaring that the way to measure it was through recurrency: If the problem never happened again the action was efective.

I think that approach was correct, but there was always a debate about the evidence to prove it. I used to show the master list of correction action as an evidence.

Recently, I received a suggestion of an auditor to include as a part of the SCAR form an space to capture de criteria of effectivenes for each action and then ask for evidence of them. For me it seems like too much.

Is there any advice you can give me on this topic?

Thank you in Advance

 

[normzone]

One persons effective corrective action is another person's "we been lucky so far".

Each case may merit it's own rationale - if you're never going to buy that component again then the effectiveness may be solid.

If the process was not changed the likelihood of effectiveness is lessened.

I just stick my neck out and write a brief summary of my opinion of how effective it appears to be, based on evidence available.

Occasionally I am forced to admit that the C/A was ineffective, or only partially so, because we could not obtain enough data and were reduced to addressing likely or potential root causes in an effort to reduce risk.

 

[qualprod]

It depends of the situation.

For example a CA for a machine failure, root cause was found, determined action plans to fix the problem. Was the criteria to close out this CA, it could be that machine will be running a complete week in order to be convinced actions were effective.

Other examples:

Lack of competence in some people, problem found in an audit. Well root cause was found, and plans were implemented, when to evaluate effectiveness.
It could be measure it two months after having verified problems were not present due to people competence.
​

Hope this helps.

 

[Uriel Alejandro]

It depends of the situation.

For example a CA for a machine failure, root cause was found, determined action plans to fix the problem. Was the criteria to close out this CA, it could be that machine will be running a complete week in order to be convinced actions were effective.

Other examples:

Lack of competence in some people, problem found in an audit. Well root cause was found, and plans were implemented, when to evaluate effectiveness.
It could be measure it two months after having verified problems were not present due to people competence.
​

Hope this helps.

Thank you, for your feed back

Just a question for curiosity, in the second scenario, would you leave the CA opened for the two months period defined? or close it after make the verification as an external process?

 

[qualprod]

Sometimes cas, remain open for several months.
The idea is to keep them opened til you are convinced
Action were implemented and resulted effective after a verification.
Normally after the process was running enough time as you are convinced now runs well.
Regards

 

[Randy]

How about you do what works for you that meets the requirements of your system and the standard in question and tell the auditor if he wants more to provide a requirement to do so?

 

[John Predmore]

If the problem never happened again the action was effective. Thank you in Advance

How long do you wait to prove something can’t happen again?

If problem occurrences follow a Poisson distribution, and if the past is indicative of the future, the length of time between occurrences can be predicted using the Exponential function. For example, if the Mean Time Between Failures (MTBF) is 123 hours, the likelihood of a failure in any one hour (the lambda parameter, 1/MTBF) is 0.00813. In Excel, the cumulative probability that a problem will occur in the next 123 hours is EXPONDIST(123,0.00813,TRUE)=63.2%. (Because the Exponential is not bell-shaped, the probability of the mean is not 50%.)

You want the inverse problem, for a given level of confidence, what is the length of time when at least one event is statistically expected to occur? Excel does not provide an inverse function for EXPONDIST, but my textbook states the inverse function is -LN(alpha)/lambda. For my example, for 90% confidence, -LN(1-0.9)/.00813 = 283 hours.

If your MTBF is in weeks rather than hours, you may not want to wait 283 weeks (5.5 years) for an effectiveness review to close your CAPA. In that case, if you know enough about the problem to recreate the problem occurrence, you could validate your solution with a designed experiment to show your solution was effective.

 

[AndyN]

Hello Everybody,

I´m making some updates to my corrective actions procedure and during this process I´m having some problems trying to define a correct method for the evaluation of the efectiveness of a implemented action.

I the past, I managed to solve the effectiveness requirement declaring that the way to measure it was through recurrency: If the problem never happened again the action was efective.

I think that approach was correct, but there was always a debate about the evidence to prove it. I used to show the master list of correction action as an evidence.

Recently, I received a suggestion of an auditor to include as a part of the SCAR form an space to capture de criteria of effectivenes for each action and then ask for evidence of them. For me it seems like too much.

Is there any advice you can give me on this topic?

Thank you in Advance

It's simple! Take a look at the problem statement in the corrective action request. Then, answer, has this problem gone away? If time shows no recurrence, the CA was good. If your local town has a problem with speeding drivers and installs traffic calming devices - rumble strips, narrow lanes, signs etc. then the speed of the cars will drop if these measures are effective, as long as they remain in place.

 

[QuinnM]

Hi Uriel,

If I understand correctly the auditor suggested to capture de criteria of effectiveness of each action. From this statement, I do not think the auditor was questioning your approach, but rather the documentation activities. Capturing the effectiveness of each activity, and over all CAPA should be documented. I do not know what quality system you use, and my experience is from 13485 and 21 CFR 820. See below.

ISO 13485:2016, section 8.5.2 states: “The organization shall document a procedure to define requirements for: …f) reviewing the effectiveness of corrective action taken. Records of the results of any investigation and of action taken shall be maintained (see 4.2.5)”

21 CFR 820.100 Corrective and Preventative Action: “(4) Verifying or validating the corrective and preventive action to ensure that such action is effective and does not adversely affect the finished device;” and “(b) All activities required under this section, and their results, shall be documented.”

Maybe you do not have an issue with defining the correct method, but only need to document the activities?

All the best,
Quinn

 

[Uriel Alejandro]

How long do you wait to prove something can’t happen again?

If problem occurrences follow a Poisson distribution, and if the past is indicative of the future, the length of time between occurrences can be predicted using the Exponential function. For example, if the Mean Time Between Failures (MTBF) is 123 hours, the likelihood of a failure in any one hour (the lambda parameter, 1/MTBF) is 0.00813. In Excel, the cumulative probability that a problem will occur in the next 123 hours is EXPONDIST(123,0.00813,TRUE)=63.2%. (Because the Exponential is not bell-shaped, the probability of the mean is not 50%.)

You want the inverse problem, for a given level of confidence, what is the length of time when at least one event is statistically expected to occur? Excel does not provide an inverse function for EXPONDIST, but my textbook states the inverse function is -LN(alpha)/lambda. For my example, for 90% confidence, -LN(1-0.9)/.00813 = 283 hours.

If your MTBF is in weeks rather than hours, you may not want to wait 283 weeks (5.5 years) for an effectiveness review to close your CAPA. In that case, if you know enough about the problem to recreate the problem occurrence, you could validate your solution with a designed experiment to show your solution was effective.

Thank you John, that´s an awesome way to manage it and a good advice. I will take a look to my records to see if have enough data to calculate something like this.