I am working to set up the effectiveness verification process for corrective and preventive actions at my work. All we are doing now is validating the completion of actions by looking at documents, installing of new tools, etc but not effectiveness. The CAPA are categorized into different groups such as documentation updates, Improve Mfg. Process/Recipe, Poka Yoke, and so on. Is there a guideline as to what kind of verification methods can be used based on action type?
CAPA effectiveness verification methods for different types of CA
- Thread starter Hnin MO
- Start date
Hi Hnin MO,
The verification of effectiveness will very much depend what your original problem was. You can use any method you like to verify that your actions have been effective in preventing reoccurrence or potential occurrence. Of course, if it is possible to use statistics this is even better but not mandatory.
I have seen some instances where each for each individual action there is an effectiveness check, or all of them together as a whole.
Personally, I don't like to write myself into a corner so I provide space for a plan to be documented, how that is completed will very much depend on circumstance.
The verification of effectiveness will very much depend what your original problem was. You can use any method you like to verify that your actions have been effective in preventing reoccurrence or potential occurrence. Of course, if it is possible to use statistics this is even better but not mandatory.
I have seen some instances where each for each individual action there is an effectiveness check, or all of them together as a whole.
Personally, I don't like to write myself into a corner so I provide space for a plan to be documented, how that is completed will very much depend on circumstance.
Thee Bouyyy
Multiple Personalities
There is no such guideline exist. As @Alan_DB said in above post, the verification of effectiveness will very much depend what your original problem was. Adding more,
Still as an example what my clients are doing as of now, below are some activities that are performed to verify the effectiveness of the corrective actions.
1. Corrective action type - Providing resources
Method of evaluation of effectiveness of corrective action - Physical verification shall be done to ensure the effective utilization of resources provided.
2. Corrective action type - Changes/ revision in QMS
Method of evaluation of effectiveness of corrective action - Auditing shall be done to ensure the effective implementation of revised QMS and/or interviewing the concern personnel to understand whether the changes have been properly communicated.
3. Corrective action type - Imparted training
Method of evaluation of effectiveness of corrective action - Training effectiveness evaluation shall be done by self and by superior to assess the effectiveness of the training.
While I agree that there are some circumstances where it wouldn't be worth the effort to construct a study design for Corrective Actions (and frankly, I am not sure that it is mathematically possible, absent mental gymnastics, to do so for Preventive Actions), I do encourage CAPA teams to consider what sort of study design could be created for every action item in a CAPA. My motivation is that by trying to construct a study design to gage the effectiveness of an action item, the team will have to make some meaningful assessments about the seriousness of the problem and the seriousness of the proposed action item.
Usually by the time a problem has escalated to CA, someone has already made a determination that "this must be fixed", but occasionally the thing "to be fixed" isn't as well understood as folks presumed. I have occasionally seen corrective actions implemented that would be impossible to prove are doing anything. In such cases, I'd rather not implement those... but at least with math I can argue that we don't have to spend much effort on an effectiveness check.
The company I work for got a finding for not having the Trend 88 reporting requirements (MDR) identified within our QMS. Our correction action is to train the responsible employees to that section of the standard. What effectiveness could be done for this? As the finding is not that we did not perform the trending, it is that we did not instruct ourselves to perform the training within our QMS. This has always been a difficult type of corrective action to navigate.
This seems rather straightforward: the planned CA plan would probably be something like "by a specified date, training ____ will be delivered to 100% of identified associates" and the effectiveness check would be to review the training records.
There are all sorts of subtleties, but typically the effectiveness checks are established before implementing the correction. There is a spectrum of usefulness (of both CA and effectiveness checks) but what I suggested above is something I have commonly seen. I can attest that I've been party to more (bluntly applied) mass retraining as a result of CA than I could possibly recall.
Tagin
Trusted Information Resource
We can use the definition of 'effectiveness' from 9000:2015 as a guide:
Breaking this down, we have these elements:
More colloquially, we would probably phrase this as something like:
What is not an explicitly stated part of effectiveness is side effects. For example, if our form achieves the intended results, but its a 30-page checklist that takes 2 hours to fill out, causing worker unhappiness and missed work order deadlines, those are undesirable side effects. Side effects could be positive or negative (e.g., an improved process might make workers happier as a side effect), but in either case are missing from effectiveness. One could argue that 'planned results' should encompass the risk of side effects, so that changes our effectiveness statement to:
Breaking this down, we have these elements:
- extent that planned activities are realized
- extent that planned results are achieved
- (implied) 'extent' indicates a dataset that is a defensible sample size across applicable activities and time
- extent that planned activities are realized:
- checklist form F-0123 exists and is available to applicable workers
- checklist form F-0123 is being filled out correctly
- checklist form F-0123 is being filled out on every work order
- checklist form F-0123 is being saved in the right location
- extent that planned results are achieved (remember, we did this because of a process deficiency)
- process demonstrates improvement of x% from the deficiency (or elimination of the deficiency)
- extent:
- across this process, for at least 15 work orders (or for minimum of 1 months, etc.).
More colloquially, we would probably phrase this as something like:
- Effectiveness: Monitor for a minimum of 25 work orders as of 6/10/2022 that form F-0123 is being correctly filled out, signed, and saved as planned on every work order in process X123, and that rejection rates from process X123 are below 1.5% for a minimum of 3 months beginning with July 2022.
What is not an explicitly stated part of effectiveness is side effects. For example, if our form achieves the intended results, but its a 30-page checklist that takes 2 hours to fill out, causing worker unhappiness and missed work order deadlines, those are undesirable side effects. Side effects could be positive or negative (e.g., an improved process might make workers happier as a side effect), but in either case are missing from effectiveness. One could argue that 'planned results' should encompass the risk of side effects, so that changes our effectiveness statement to:
- Effectiveness: Monitor for a minimum of 25 work orders as of 6/10/2022 that form F-0123 is being correctly filled out, signed, and saved as planned on every work order in process X123, and that rejection rates from process X123 are below 1.5% for a minimum of 3 months beginning with July 2022, and there is no negative impact on process X123 on-time performance and no notable worker complaints relative to these changes.
My understanding of effectiveness would not be to verify the training had occurred, but to establish the effectiveness of the training. In other words, has the issue reoccurred, because the corrective action (i.e., training) was to keep this from happening. OR to mandate tests/quizzes to establish if the trainee retained the information.
I am recommending an effectiveness check for the action item taken. If there were other issues, perhaps more action items were needed.
Enghabashy
Quite Involved in Discussions
- verifying the effectiveness should be initiated initially from each dept. head , the department is responsible to initiate & apply effective corrective action according root causes investigation/ determination within the agreed time 'approved before ; we shall refer to documented action plan also ,the management representative /QA / --shall receive in the same report / form a statement that :completion & close out of corrective action as effectively applied , sample of evidences should be provided also; the management representative can allow the report to be still open till make independent investigation regarding the result of internal & external evidences like : internal & external filed failure, customers complain rates " the worst indications " or NC rate through last 3 months in the process through this period of the implementation " , ---etc. -- accordingly he can closeout by satisfaction implementation result status / -- or leave the report still open till reinvestigation / determination by the same committee who schedule the action plan
