How to measure CAPA (Corrective Action Preventive Action) System Performance

How would one measure the performance of a CAPA system? Many individuals say you measure the time it takes to do a root cause and move to verification of effectiveness. But how do you answer that question to upper management who live in a very matrix numbers world? Each CAPA is different and unique. Would you want to measure the number of CAPAs opened in a period? Why would you look at that? What would you gain?

Re: CAPA System Performance

QA Memphis said:

How would one measure the performance of a CAPA system? Many individuals say you measure the time it takes to do a root cause and move to verification of effectiveness. But how do you answer that question to upper management who live in a very matrix numbers world? Each CAPA is different and unique. Would you want to measure the number of CAPAs opened in a period? Why would you look at that? What would you gain?

Click to expand...

First separate CA and PA. If it's CA you're concerned with, the only rational yardstick (from a high level) is effectiveness--what's the likelihood that the CA process, once invoked, will result in keeping the same (and similar) things from happening again?

Re: CAPA System Performance

I am having the same discussions at work, I am the current "CAPA process champion"

Another way of asking this question is,:truce: why do you have a CAPA system?

In non regulated industries CAPA = continous "or continual" improvement process. In that case the metric for the system is improvement on process and products. Do you have process metrics (yields) do you have internal or external product metrics (complaints).

Could you establish the corelation of how does your CAPA systems affects these metrics?
Measuring # of CAPAs or time to close them is done as a necessary process management tool, but should not be your only metric.

I agree with you. In our case we have to do CAPA because of the CFR.

QA Memphis said:

How would one measure the performance of a CAPA system? Many individuals say you measure the time it takes to do a root cause and move to verification of effectiveness. But how do you answer that question to upper management who live in a very matrix numbers world? Each CAPA is different and unique. Would you want to measure the number of CAPAs opened in a period? Why would you look at that? What would you gain?

Click to expand...

In our world (internally) we measure it by looking at the recurrence of the same problem (feature) on repeat parts or similar parts! Also, for each corrective action we attempt to provide preventative measures so we do not have the same repeat issues. We know in some worlds that it is considered all corrective action. So, in determining the effectiveness of a CAPA activity, We query out rejection Database for repeat feature rejections on same part number same feature. Then we query the ERP system to determine how many pieces we have shipped, compare the two and the results are clear become clear.

We also treat external CAPA in the same manner.

In the end if you have repeat NC, then the CAPA has not been effective. However, one must be careful, because often time when you correct one problem, it may create another feature issue. So, in the case where it is the same part number, but a different feature one has to be sure that you look for any correlations to the new feature NC.

IMHO, if you track CA in this manner then you can have the following data;

Open Corrective Actions
Closed Corrective Actions - Need Proof of Closure and Effectiveness
Delinquent Open Corrective Action - Past Date For Closure
Repeat Corrective Actions - Highlighting that Previous Efforts Did Not Work
Deferred Corrective Actions - Until Next Production Run

In the past when I was a Suit, I was very interested in how much effort we were putting into CAPA, with the above data it was simple to allow QA to report on this activity and hold process holders accountable.

Typically, I asked for the same information on PA system!

Final comments, sometimes if maybe helpful to look at CAPA from an age standpoint, how long is it acceptable for a CAPA to remain open........

I know it's a little overkill but we measure CAPA by the following:

1. Action completion on time: We set a date of completion based on the complexity of the project and hold ourselves to that deadline.

2. CAPA FTT (First time through): We measure how many action reports are rejected back to the appointed team due to incomplete information or insufficient investigation, action, or steps to prevent.

3. Repeat or recurrence of issue: We measure this by way of Paynter chart.

4. Corrective action vs Preventive action ratio: IE In theory, the more planning, risk assessment, and prevention activities you institute, the amount of corrective actions should exponentially reduce. Our APQP process is a huge input to this measurable.

Boscoeee said:

Final comments, sometimes if maybe helpful to look at CAPA from an age standpoint, how long is it acceptable for a CAPA to remain open........

Click to expand...

In the biotech industry, we definately measure the age of the CA and PA, since this is something that the FDA would also look at. If aging isn't watched closely, it can be shocking the amount of time that a CA or PA will linger, making it very difficult to defend to the FDA.

Sandra

It seems to be handled differently everywhere I go. One thing I can comment on that's very consistent is the way FDA investigators go about analyzing a CAPA system.

It seems that almost every organization I've been in has always been concerned with reports showing how many issues were opened, closed, are overdue, etc. but few seem interested in the actual performance indicators of the system.

I tend to try to shift the thinking and approach it from the Field Investigator's perspective. I don't see anything wrong (at least on the surface) with a large quantity of open items, or items that take a "long time" to resolve.

I'd rather see some of the detail. I think charts that illustrate

Source of issue
Type of issue (general category)
Quality System element
Specific processes
Cause (general category)

Then, set expectations based on the organization, its size, culture, quality awareness, stage of development, etc. In other words, I'd expect to see more issues related to employees not following procedures issued into the CAPA system in large quantities and in large blocks at most start-up companies. Many incorporate the nonconforming materials and/or complaint handling procedures in the same document. I typically find that the majority of the items relate to nonconforming material consistently through time, but quality system issues are raised primarily during internal audits. As an auditor, it tells me that I can most likely expect to find many instances of employees not following procedures, because they don't pay attention to them until they're audited.

If on the other hand, I see consistent entries in the CAPA system for quality system issues, it tells me that the organization is consistently monitoring their processes.

If I see very few CAPA entries, it tells me that the CAPA system is probably not effective, or the entire quality system is ineffective.

Whenever I've hosted FDA or other agency inspection, the investigators have always looked at quantity relative to the organization. They look at the same criteria I mentioned above. It gives them a bird's eye view of the entire business, and they're able to identify areas of concern very quickly.

In my opinion, the only true measure of the performance of a CAPA system is to measure what (and how) it's doing, rather than "how many."
(that was just too much for two pennies)

Chris Ford

Chris Ford said:

It seems that almost every organization I've been in has always been concerned with reports showing how many issues were opened, closed, are overdue, etc. but few seem interested in the actual performance indicators of the system.

I tend to try to shift the thinking and approach it from the Field Investigator's perspective. I don't see anything wrong (at least on the surface) with a large quantity of open items, or items that take a "long time" to resolve.

In my opinion, the only true measure of the performance of a CAPA system is to measure what (and how) it's doing, rather than "how many."
(that was just too much for two pennies)

Chris Ford

Click to expand...

I would agree. It seems like most of the companies I've worked for in the past were more interested in the aging report and little emphasis was place on the effectiveness of the corrective action. The mindset seems to be quantity over quality.

In past FDA audits, there have been issues around CA's remaining open for an excessive amount of time without any or little activity....it becomes difficult to demonstrate that you've taken timely action. The length of time that a CA remains open needs to be relative to the risk level and the complexity to implement.

Sandra

Sandra Gauvin said:

I would agree. It seems like most of the companies I've worked for in the past were more interested in the aging report and little emphasis was place on the effectiveness of the corrective action. The mindset seems to be quantity over quality.

In past FDA audits, there have been issues around CA's remaining open for an excessive amount of time without any or little activity....it becomes difficult to demonstrate that you've taken timely action. The length of time that a CA remains open needs to be relative to the risk level and the complexity to implement.

Sandra

Click to expand...

Exactly... the length of time will always raise an eyebrow. It should. At first glance, it can raise a red flag... is the CAPA system ineffective? Are resources too tight? I like the CAPA systems that require regular updates and handle CAPA events as projects. They usually require updates to the plan with explanations for changes and requests for extensions, etc. A well-documented CAPA system as well as the thought process in the action taken will go a long way later on.

It's not just during an inspection that a company can benefit from doing this. These systems seem to be more successful in actually resolving and preventing recurrence when there is an active, frequent participation - when the system is viewed as a day-to-day monitoring and management procedure, rather than a response system.

Chris Ford