How to measure CAPA (Corrective Action Preventive Action) System Performance

Q

QA Memphis

#1
How would one measure the performance of a CAPA system? Many individuals say you measure the time it takes to do a root cause and move to verification of effectiveness. But how do you answer that question to upper management who live in a very matrix numbers world? Each CAPA is different and unique. Would you want to measure the number of CAPAs opened in a period? Why would you look at that? What would you gain?:biglaugh:
 
Elsmar Forum Sponsor

Jim Wynne

Staff member
Admin
#2
Re: CAPA System Performance

How would one measure the performance of a CAPA system? Many individuals say you measure the time it takes to do a root cause and move to verification of effectiveness. But how do you answer that question to upper management who live in a very matrix numbers world? Each CAPA is different and unique. Would you want to measure the number of CAPAs opened in a period? Why would you look at that? What would you gain?:biglaugh:
First separate CA and PA. If it's CA you're concerned with, the only rational yardstick (from a high level) is effectiveness--what's the likelihood that the CA process, once invoked, will result in keeping the same (and similar) things from happening again?
 
T

Trackerii

#3
Re: CAPA System Performance

I am having the same discussions at work, I am the current "CAPA process champion" :notme:

Another way of asking this question is,:truce: why do you have a CAPA system?

In non regulated industries CAPA = continous "or continual" improvement process. In that case the metric for the system is improvement on process and products. Do you have process metrics (yields) do you have internal or external product metrics (complaints).

Could you establish the corelation of how does your CAPA systems affects these metrics?
Measuring # of CAPAs or time to close them is done as a necessary process management tool, but should not be your only metric.:2cents:
 
B

Boscoeee

#5
How would one measure the performance of a CAPA system? Many individuals say you measure the time it takes to do a root cause and move to verification of effectiveness. But how do you answer that question to upper management who live in a very matrix numbers world? Each CAPA is different and unique. Would you want to measure the number of CAPAs opened in a period? Why would you look at that? What would you gain?:biglaugh:
In our world (internally) we measure it by looking at the recurrence of the same problem (feature) on repeat parts or similar parts! Also, for each corrective action we attempt to provide preventative measures so we do not have the same repeat issues. We know in some worlds that it is considered all corrective action. So, in determining the effectiveness of a CAPA activity, We query out rejection Database for repeat feature rejections on same part number same feature. Then we query the ERP system to determine how many pieces we have shipped, compare the two and the results are clear become clear.

We also treat external CAPA in the same manner.

In the end if you have repeat NC, then the CAPA has not been effective. However, one must be careful, because often time when you correct one problem, it may create another feature issue. So, in the case where it is the same part number, but a different feature one has to be sure that you look for any correlations to the new feature NC.

IMHO, if you track CA in this manner then you can have the following data;

Open Corrective Actions
Closed Corrective Actions - Need Proof of Closure and Effectiveness
Delinquent Open Corrective Action - Past Date For Closure
Repeat Corrective Actions - Highlighting that Previous Efforts Did Not Work
Deferred Corrective Actions - Until Next Production Run

In the past when I was a Suit, I was very interested in how much effort we were putting into CAPA, with the above data it was simple to allow QA to report on this activity and hold process holders accountable.

Typically, I asked for the same information on PA system!

Final comments, sometimes if maybe helpful to look at CAPA from an age standpoint, how long is it acceptable for a CAPA to remain open........
 
M

Matt Swartwood

#6
I know it's a little overkill but we measure CAPA by the following:

1. Action completion on time: We set a date of completion based on the complexity of the project and hold ourselves to that deadline.

2. CAPA FTT (First time through): We measure how many action reports are rejected back to the appointed team due to incomplete information or insufficient investigation, action, or steps to prevent.

3. Repeat or recurrence of issue: We measure this by way of Paynter chart.

4. Corrective action vs Preventive action ratio: IE In theory, the more planning, risk assessment, and prevention activities you institute, the amount of corrective actions should exponentially reduce. Our APQP process is a huge input to this measurable.
 
S

Sandra Gauvin

#7
Final comments, sometimes if maybe helpful to look at CAPA from an age standpoint, how long is it acceptable for a CAPA to remain open........
In the biotech industry, we definately measure the age of the CA and PA, since this is something that the FDA would also look at. If aging isn't watched closely, it can be shocking the amount of time that a CA or PA will linger, making it very difficult to defend to the FDA.

Sandra
 
Last edited by a moderator:
C

Chris Ford

#8
It seems to be handled differently everywhere I go. One thing I can comment on that's very consistent is the way FDA investigators go about analyzing a CAPA system.

It seems that almost every organization I've been in has always been concerned with reports showing how many issues were opened, closed, are overdue, etc. but few seem interested in the actual performance indicators of the system.

I tend to try to shift the thinking and approach it from the Field Investigator's perspective. I don't see anything wrong (at least on the surface) with a large quantity of open items, or items that take a "long time" to resolve.

I'd rather see some of the detail. I think charts that illustrate

Source of issue
Type of issue (general category)
Quality System element
Specific processes
Cause (general category)

Then, set expectations based on the organization, its size, culture, quality awareness, stage of development, etc. In other words, I'd expect to see more issues related to employees not following procedures issued into the CAPA system in large quantities and in large blocks at most start-up companies. Many incorporate the nonconforming materials and/or complaint handling procedures in the same document. I typically find that the majority of the items relate to nonconforming material consistently through time, but quality system issues are raised primarily during internal audits. As an auditor, it tells me that I can most likely expect to find many instances of employees not following procedures, because they don't pay attention to them until they're audited.

If on the other hand, I see consistent entries in the CAPA system for quality system issues, it tells me that the organization is consistently monitoring their processes.

If I see very few CAPA entries, it tells me that the CAPA system is probably not effective, or the entire quality system is ineffective.

Whenever I've hosted FDA or other agency inspection, the investigators have always looked at quantity relative to the organization. They look at the same criteria I mentioned above. It gives them a bird's eye view of the entire business, and they're able to identify areas of concern very quickly.

In my opinion, the only true measure of the performance of a CAPA system is to measure what (and how) it's doing, rather than "how many."
:2cents::2cents::2cents: (that was just too much for two pennies)

Chris Ford
 
S

Sandra Gauvin

#9
It seems that almost every organization I've been in has always been concerned with reports showing how many issues were opened, closed, are overdue, etc. but few seem interested in the actual performance indicators of the system.

I tend to try to shift the thinking and approach it from the Field Investigator's perspective. I don't see anything wrong (at least on the surface) with a large quantity of open items, or items that take a "long time" to resolve.

In my opinion, the only true measure of the performance of a CAPA system is to measure what (and how) it's doing, rather than "how many."
:2cents::2cents::2cents: (that was just too much for two pennies)

Chris Ford
I would agree. It seems like most of the companies I've worked for in the past were more interested in the aging report and little emphasis was place on the effectiveness of the corrective action. The mindset seems to be quantity over quality.

In past FDA audits, there have been issues around CA's remaining open for an excessive amount of time without any or little activity....it becomes difficult to demonstrate that you've taken timely action. The length of time that a CA remains open needs to be relative to the risk level and the complexity to implement.

Sandra
 
Last edited by a moderator:
C

Chris Ford

#10
I would agree. It seems like most of the companies I've worked for in the past were more interested in the aging report and little emphasis was place on the effectiveness of the corrective action. The mindset seems to be quantity over quality.

In past FDA audits, there have been issues around CA's remaining open for an excessive amount of time without any or little activity....it becomes difficult to demonstrate that you've taken timely action. The length of time that a CA remains open needs to be relative to the risk level and the complexity to implement.

Sandra
Exactly... the length of time will always raise an eyebrow. It should. At first glance, it can raise a red flag... is the CAPA system ineffective? Are resources too tight? I like the CAPA systems that require regular updates and handle CAPA events as projects. They usually require updates to the plan with explanations for changes and requests for extensions, etc. A well-documented CAPA system as well as the thought process in the action taken will go a long way later on.

It's not just during an inspection that a company can benefit from doing this. These systems seem to be more successful in actually resolving and preventing recurrence when there is an active, frequent participation - when the system is viewed as a day-to-day monitoring and management procedure, rather than a response system.

Chris Ford
 
Thread starter Similar threads Forum Replies Date
C First Article Inspections - 'how to measure' General Measurement Device and Calibration Topics 11
lanley liao Question regarding the calibration of monitoring and measure equipment. Oil and Gas Industry Standards and Regulations 0
J Should a Class 1 medical device with an option to measure body weight be considered Class 1m? EU Medical Device Regulations 0
M Gage R&R and right way to measure Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 16
P How to measure DPPM in Chemical batch manufacturing Quality Tools, Improvement and Analysis 1
silentmonkey How to measure severity if my product is designed for emergency use and failure would result in death? ISO 14971 - Medical Device Risk Management 9
A Looking for a third party to calibrate our measure equipment - South Carolina General Measurement Device and Calibration Topics 6
J Validity / outcomes measure for custom made medical device ISO 13485:2016 - Medical Device Quality Management Systems 2
D Performance specification as a Risk Control Measure, EN 14971 ISO 14971 - Medical Device Risk Management 7
J KPIs or Metrics to Measure a New Complaint Handling Process 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 9
D Torque fractional unit of measure; cN-m or N-cm? General Measurement Device and Calibration Topics 2
qualprod ISO 9001 5.1.1 - How to measure System effectiveness ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
J How to measure a Critical Characteristic that is a Basic Dimension Misc. Quality Assurance and Business Systems Related Topics 6
I Can we use pin gauges to measure an accurate Cpk ? Capability, Accuracy and Stability - Processes, Machines, etc. 9
qualprod Best criteria to measure Corrective Action effectiveness - Poor Maintenance ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
1 How to measure straightness on a HEX Long Bar Manufacturing and Related Processes 7
Ron Rompen Measurement Method Challenge - Measure feature #91 Inspection, Prints (Drawings), Testing, Sampling and Related Topics 9
G Measure 3 times, for calibration report? Customer Owned Reference Standard General Measurement Device and Calibration Topics 4
M Informational The ANSM decides, as a precautionary measure, to withdraw from the market macrotextured breast implants and breast implants with polyurethane-coated s Medical Device and FDA Regulations and Standards News 0
M Informational Health Canada advises Allergan of its intent to suspend its licences for Biocell breast implants as a precautionary measure Medical Device and FDA Regulations and Standards News 0
U SI 13 - Not Every Process Requires an Efficiency Measure IATF 16949 - Automotive Quality Systems Standard 1
P NPR Metric - What if Customer has relaxed measure? TL 9000 Telecommunications Standard and QuEST 1
T Tape Measure, the right device for 2 or 3 decimals accuracy on prints General Measurement Device and Calibration Topics 8
M How do we document and measure a service we do not provide? IATF 16949 - Automotive Quality Systems Standard 1
M For a Type 1 Gage R&R I want to use 1 unit and measure it 30 times Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
M How to Measure Paper considering Variation and Humidity General Measurement Device and Calibration Topics 10
rob73 Advice/recommendations for equipment to measure a soft pvc tube - Optical Measuring? General Measurement Device and Calibration Topics 4
R Height gauge to measure short hard to access heights General Measurement Device and Calibration Topics 3
D Proper Tool to Measure Wall Thickness (ISO 3611 - Micrometers) Manufacturing and Related Processes 7
S Quality Measure for Engineering Services Misc. Quality Assurance and Business Systems Related Topics 1
A What are methods and tools to measure Workload analysis ? Human Factors and Ergonomics in Engineering 1
M How do I measure variation in my process ? Capability, Accuracy and Stability - Processes, Machines, etc. 3
J How to measure process quality? Is DPPM the correct metric? Capability, Accuracy and Stability - Processes, Machines, etc. 5
M Micrometer Anvil Placement to measure the diameter of Steel Wires Measurement Uncertainty (MU) 6
S Where to Get SPC Training to Measure our Processes? Training - Internal, External, Online and Distance Learning 12
M A question on Measurement System Analysis - Stability to measure got Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
Jim Wynne Measure Twice, Cut Once: The French Railroad Snafu World News 18
S Measure water activity for fresh raw chicken meat and giblets Food Safety - ISO 22000, HACCP (21 CFR 120) 1
O Metrology Devices to Measure the Basic Dimensions of Plastic Specimens General Measurement Device and Calibration Topics 6
L How to Measure Paint Thickness for an Enclosure Manufacturing and Related Processes 5
C Chamfer Measurements - Do you measure the height and width or the hypotenuse? Manufacturing and Related Processes 15
R Repeatability in MSA - Is repeatability a measure of Location Error Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
E How can I measure three true positions? GD&T basic question General Measurement Device and Calibration Topics 5
V How to use a Calibration Certificate in Uncertainty Calculation for a Test Measure Measurement Uncertainty (MU) 13
R How do companies measure QC Productivity? Quality Manager and Management Related Issues 3
V How to measure radius without radius gages APQP and PPAP 4
A Device to measure "Straightness & Deformation" of a metal that's under water Manufacturing and Related Processes 4
M How to measure effectiveness and efficiency of the established QMS as Auditors ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 14
F Specification for how to measure lengths of hose assemblies Various Other Specifications, Standards, and related Requirements 1
P MSA requirements for Systems that Measure Process Parameters Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3

Similar threads

Top Bottom