Does anyone have a scoring/grading scheme for Effectiveness Reviews that they could share? Our CAMS folks are considering upgrading our scheme from a simple pass/fail as no one is willing to actually consider themselves as "failed".

Does anyone have a scoring/grading scheme for Effectiveness Reviews that they could share? Our CAMS folks are considering upgrading our scheme from a simple pass/fail as no one is willing to actually consider themselves as "failed".

How is it possible for it to be objective?

How is it possible for it to be objective?

I am sure any such scheme would be some combination of subjective and objective.

I am sure any such scheme would be some combination of subjective and objective.

How could you keep the subjective from coloring the outcome? I don't think it's a good idea to try and assign arbitrary numbers to things, and then use the outcome to describe performance of a person or system.

Does anyone have a scoring/grading scheme for Effectiveness Reviews that they could share? Our CAMS folks are considering upgrading our scheme from a simple pass/fail as no one is willing to actually consider themselves as "failed".


Then have them start to use "Not achieving desired level of success"

I agree with Jim. We used to have a scoring system but it was seen as a failure indicator by managers. If anyone fell below a certain percentage then then were deemed to be failures. No matter how many times that we told them that it was only to show areas for improvement they still felt it was a black mark against them.
As an auditor, coming into that scheme, I had reservations with the system. It used to apply points for audit questions/items passed. The result was, basically, questions/items passed against total questions giving a percentage value. i.e if you asked 20 questions and 5 had observations then the audittee would get a value of 75%. My real problem was the number of questions changed depending on the process etc. One audit I asked 150 questions and their were 8 observations noted - 94.7%...the next audit was 15 questions and three observations - 80%. There was no weight on the questions and therefore the entire system was faulty but it ran for five years before I finally got it thrown out and set about repairing the damage that it caused.
NOTE: All audits had preset questions pulled from a data base written by the Quality manager - very laborious and not very flexible

I feel it's difficult to get it.

Interesting, thanks for the comments so far. The difficulty is that we have to address the question - were the corrective actions "effective"? But this is indeed being played out in an environment where the straight answer of "no" is seen as a "Black mark" against you, as was eloquently put.

Interesting, thanks for the comments so far. The difficulty is that we have to address the question - were the corrective actions "effective"? But this is indeed being played out in an environment where the straight answer of "no" is seen as a "Black mark" against you, as was eloquently put.

What if you were to just summarize the results without identifying individuals or projects, and then deal with the individuals one-to-one? You could have some sort of chart that shows the number of "opportunities" and the ones that were successful, and people can do the math on their own with regard to the "unsuccessful" quantity: 100 CAs initiated, and 57 resulted in effective resolution. We then know, without trumpeting, that 43 were not successful, and we can do what's necessary to improve the situation.

What if you were to just summarize the results without identifying individuals or projects, and then deal with the individuals one-to-one? You could have some sort of chart that shows the number of "opportunities" and the ones that were successful, and people can do the math on their own with regard to the "unsuccessful" quantity: 100 CAs initiated, and 57 resulted in effective resolution. We then know, without trumpeting, that 43 were not successful, and we can do what's necessary to improve the situation.


Actually, that is the way I would prefer our CAMS folks go - more frequent, less extensive verifications. Unfortunately, this has become a "big ticket" item, with only "Significant Issues" getting formal verfications, so that in our 3,000 employee company, only one or two verifications get done per month.

If I remember correctly, Deming never failed anyone in his classes but gave them incompletes. The exception would be if the work was not done in a reasonable time. The problem with grading the completion is that it does not take into account the priorities and resource availability for the work. Giving someone 10 things to do when they only have time for 5 will cause them to either pick what they think is the highest priority or attempt to do all 10 poorly. Some items also take more time so an in progress category is needed if monthly or weekly reviews are being done.

If only 1 or 2 verifications are getting done per month and the list continues to grow, either you need to evaluate what is on the list or assign more people. You may be surprised at how many things make the list, stay on it forever but are still considered significant even though they are never done.

Bill Pflanz

Scoring a qualitative matter like effectiveness can be challenging.

I am looking to use a method like the attachment shows, where different types of findings can be scored and the results trended.

Thanks, Jennifer. Though - we don't do FMEA (perhaps we should, but that is a different discussion).

The problem here (and this is a goverment project, so we fall under Price Anderson) is that we already have folks who avoid entering things into CAMS at all costs. Yet, we have to show that we have CAMS, and that the Corrective Actions are effective. So we are always trying to balance doing the right thing with not creating a huge beauracracy. Well, here is what I have suggested to the CAMS group:


EXCELLENT - Performance measurement data related to the action have been identified, and shows a significant improving trend. The system is also judged to be in an improved state in comparison to the state prior to identification of the original issue.

VERY GOOD - Performance measurement data have been identified and are being monitored. The system is judged to be returned to the state prior to identification of the original issue and the issues raised are unlikely to recur.

GOOD - The corrective actions have been implemented as originally intended. The issues raised are unlikely to recur. No new follow on actions have been identified.

ACCEPTABLE - The corrective actions have been implemented as originally intended. The issues raised are unlikely to recur. Additional Opportunities for Improvement have been identified and are entered into CAMS.

MARGINAL - Some corrective actions were not implemented as originally intended. The issues raised may recur. Additional Low Threshold Deficiencies have been identified and are entered into CAMS.

INEFFECTIVE - The corrective actions were not implemented effectively. The issue will likely recur. A new Significant Issue has been entered into CAMS