QMS Performance Based on how many NCRs were generated month to month

[DenRM]

Hello, All.

I'm looking for opinions or any published articles on the problem of organizations who measure the performance of their QMS solely by recording and comparing how many NCRs were generated month to month.

In my opinion, the simplistic task of counting the number of NCRs is not a measure of performance. Indeed, if this is the goal, then the perception that NCRs are a bad thing is perpetuated leading to the possibility that they will be under reported in order not to upset the monthly statistics. Sometimes, employees get sneaky and try to rationalize and categorize a quality issue as something other than a NCR so that the matter is "technically" not part of the NCR count.

I have counselled on the benefits and learning opportunities that NCRs offer and that we need to encourage employees and managers to report them without hesitation. I tell them that we need to care less about how many NCRs are reported and care more about and measure the timeliness and effectiveness of the correction and corrective actions that were implemented for each NCR. That, I believe, is a more valuable and insightful metric to base one's performance.

Any thoughts, links to publications, or existing discussion threads on this topic would be appreciated.

 

[qualprod]

Dear DenRM

The performance of a QMS, could be measured in several aspects.

It is considered you have implemented a QMS with several practices
which together with the people, will lead the company to the Objectives the Top management defined, a special document which can be used is the Quality objectives.

Some of them:
-Complaints reduction
-Rejects reduction
-Increase productivity
-increase market share
-Increase Customer satisfaction

I you defined goals for your processes and they are in compliment, that means your QMS has good performance.

Not necessarily quantity of NCRS means poor performance.

I think you have to consider several aspects in order to have a wider vision of your QMS.

Hope this helps

 

[DenRM]

Thanks qualprod. Great comments.

Just to be clear, my question is not about my own QMS but rather what I have encountered in the QMS of other organizations. They will, most often, have many other quality objectives to define and measure their performance. However, they often appear to rely too heavily on simply counting and reporting the number of NCRs. Their Management Review presentations or report often rely heavily on showing numerous graphs solely on this statistic. I guess perhaps because it is an easy metric to gather and report on.

 

[AndyN]

Thanks qualprod. Great comments.

Just to be clear, my question is not about my own QMS but rather what I have encountered in the QMS of other organizations. They will, most often, have many other quality objectives to define and measure their performance. However, they often appear to rely too heavily on simply counting and reporting the number of NCRs. Their Management Review presentations or report often rely heavily on showing numerous graphs solely on this statistic. I guess perhaps because it is an easy metric to gather and report on.

The management of these organizations don't grasp the QMS purpose. Of course, they may not be sophisticated to understand that you measure performance to plan. If they simply react to undesirable situations, they will not be in business much longer...

 

[Dr.Quality]

Hi there DenRM, let me throw my 5 cents on this topic.

Firstly, I think we shall clarify if we are talking for internal or external NCRs. Let's say your organization has identified 10 internal NCRs while customer issued "only" 1. I am aware even 1 is too many for the customer, but at least you are showing a "positive" ratio. It will be totally different story if internal NCRs are close to 0 but the voice of the customer says otherwise.

Another thing we need to understand is if the count of NCRs is real. Just because someone issued a NCR, this doesn't necessarily identify a problem. It often turns out to be due to lack of understanding/interpretation.

What is the severity level of the NCRs? Are we shutting line down or we are talking for a small visual imperfection. There should be some kind of ranking for the purpose of reporting.

I tell them that we need to care less about how many NCRs are reported and care more about and measure the timeliness and effectiveness of the correction and corrective actions that were implemented for each NCR.

I do agree with you that "timeliness" and "effectiveness" are key words in Quality world. However, I disagree on the point you should care less about how many are reported. You can have the best problem solving team in the world, robust corrective actions done in a day, but if you keep getting NCRs how is this doing any good? QMS purpose is to eliminate the need of NCRs to begin with. Being proactive rather than reactive is key.

To finish my long, boring post I am throwing some ideas for QMS KPIs:
- voice of the customer/customer scorecards
- scrap reduction/PPM/MRB
- continuous improvements implementation
- cost of (poor) quality reduction

 

[Jen Kirley]

Finding more nonconformities can be expected in a less mature organization, but there are too many variables to use that as an indicator of the organization's performance. If it was, I could go to a site and make them look great by going around the place with a bag over my head. "Fresh eyes" can find more nonconformities if those eyes are paired with a skilled brain and their owner is forthcoming. Then we have the question of quantity versus quality... If I find nonconformities, it could be considered a very good thing because better me than the customer or a regulator... we can learn about the system and the requirements together. Learning is the entire point.

 

[Randy]

Contributes the same value to quality and quality management as having a ticket quota does to motor vehicle safety, absolute total waste of resources and it paints a pretty, but worthless picture.

 

[AndyN]

Hi there DenRM, let me throw my 5 cents on this topic.

Firstly, I think we shall clarify if we are talking for internal or external NCRs. Let's say your organization has identified 10 internal NCRs while customer issued "only" 1. I am aware even 1 is too many for the customer, but at least you are showing a "positive" ratio. It will be totally different story if internal NCRs are close to 0 but the voice of the customer says otherwise.

Another thing we need to understand is if the count of NCRs is real. Just because someone issued a NCR, this doesn't necessarily identify a problem. It often turns out to be due to lack of understanding/interpretation.

What is the severity level of the NCRs? Are we shutting line down or we are talking for a small visual imperfection. There should be some kind of ranking for the purpose of reporting.




I do agree with you that "timeliness" and "effectiveness" are key words in Quality world. However, I disagree on the point you should care less about how many are reported. You can have the best problem solving team in the world, robust corrective actions done in a day, but if you keep getting NCRs how is this doing any good? QMS purpose is to eliminate the need of NCRs to begin with. Being proactive rather than reactive is key.

To finish my long, boring post I am throwing some ideas for QMS KPIs:
- voice of the customer/customer scorecards
- scrap reduction/PPM/MRB
- continuous improvements implementation
- cost of (poor) quality reduction

Maybe it was a confusion over the American misuse of the phrase "Couldn't care less" which has become the meaningless "Could care less"? It would be appropriate to care less about NCRs than performance, as a means to elevate understanding of performance. Not that NCRs are not to be cared about at all...

 

[somashekar]

Hello, All.

I'm looking for opinions or any published articles on the problem of organizations who measure the performance of their QMS solely by recording and comparing how many NCRs were generated month to month.

In my opinion, the simplistic task of counting the number of NCRs is not a measure of performance. Indeed, if this is the goal, then the perception that NCRs are a bad thing is perpetuated leading to the possibility that they will be under reported in order not to upset the monthly statistics. Sometimes, employees get sneaky and try to rationalize and categorize a quality issue as something other than a NCR so that the matter is "technically" not part of the NCR count.

I have counselled on the benefits and learning opportunities that NCRs offer and that we need to encourage employees and managers to report them without hesitation. I tell them that we need to care less about how many NCRs are reported and care more about and measure the timeliness and effectiveness of the correction and corrective actions that were implemented for each NCR. That, I believe, is a more valuable and insightful metric to base one's performance.

Any thoughts, links to publications, or existing discussion threads on this topic would be appreciated.

Opinions Thoughts ..
Finding more nonconformities can be expected in a less mature organization.. @Jen
Perfectly said.
Focus on improving the quality of internal audits as the organization matures.
Into the management review, the input from Audit results must trend the areas and processes that show up concerns and are effecting the policy and objectives adversely.
This should trigger focus areas for CA / improvement as well as more audits further.
If one is just counting without knowing what counts, then you are not evaluating the QMS performance at all.