Criteria to raise a Nonconformance based on KPI values

[John Broomfield]

Who set the KPIs? We’re they set by top management as system objectives?

How do you know the KPIs are not just key performance indicators or aspirations?

Better to use the term: objectives.

As we know an ineffective system is a nonconformity and effectiveness is determined by the system’s objectives.

KPIs are not the same as objectives unless your top management makes it so.

 

[Bev D]

I would suggest thinking about this a bit differently. you should absolutely use SPC to determine if your KPIs are meeting the goals. simply have some arbitrary red yellow green range is not going to be helpful. it will only drive people even more against the whole goal thing.

There are two types of corrective actions: one against the KPI/Goal system itself and one against the process that is not meeting the goal.

You might consider a CA against the KPI/goal system itself fro the leadership team. In my experience I have found this to be very successful. Typically this comes down to a better goal setting and measuring process as well as understanding what the teams need to actually achieve the goal. (resources, time, methods, etc.)

Then you can launch a 'project' (maybe not call it a CA) to improve the performance on one or more of your poor performing KPIs.

 

[Jim Wynne]

The impetus for any objective is to be better at something than you presently are. Improvement, in other words. Improvement doesn't happen by just setting objectives. There has to be a plan. The idea should be to optimize processes--to make them perform as well as possible, given the known constraints. If you don't have a plan for each of your objectives--and a method for measuring performance--you don't have objectives. You have arbitrary mandates, and such things never end well.

 

[qualprod]

Who set the KPIs? We’re they set by top management as system objectives?

How do you know the KPIs are not just key performance indicators or aspirations?

Better to use the term: objectives.

As we know an ineffective system is a nonconformity and effectiveness is determined by the system’s objectives.

KPIs are not the same as objectives unless your top management makes it so.

John
The kpis I refer to are key kpis, supporting one main adjetive.
e.g. Increase client satisfaction from 65 to 80 %
For this I defined kpis to be monitored
-Reduce client rejects by 15% kpis,
-Improve OTD by 10%
-Speed up quotations by 20%

Bev D

When doing this analysis I found what you say.
When there is an uncompliment, not reaching the goal
One could be totally technical, (machine failure, people competency, material, etc.)
while other could be poor analysis to define the goal, setting the kpi in the wrong process or level, etc.

Thanks also to Jim

 

[RoxaneB]

1. How did you determine that, using your example, those 3 KPIs are the ones that will increase client satisfaction from 65% to 80%?
2. If you have historical data, you can easily calculate your control limits and then determine your "yellow" and "red" zones using a 6 sigma approach.
3. I'll wager to say that Management Review happens more than twice a year, just not in a formal setting that you've document. Who owns the process related to customer satisfaction? Who owns the process(es) where client rejects, OTD, and quote time are reported? A mature system has the process owners keeping their eye on these metrics on a much more frequent basis
4. The idea of a preventive action after one yellow - something you mentioned earlier in thread - comes across as causing more work than adding value. Think of it this way...you find yourself speeding while driving, you take your foot off of the accelerator. You find yourself speeding a lot, now you start to analyze what's going on. We went with 3 consecutive yellows before a response was required from the owner of the metric.
5. We tried to set upper control limits, as well. These generated "good" reds. A red isn't necessarily bad - it simply means that your result is outside of the expected control limits. Sometimes we would discuss if the good red was sustainable or did we spend too much time on getting that result that we failed elsewhere?

 

[qualprod]

1. How did you determine that, using your example, those 3 KPIs are the ones that will increase client satisfaction from 65% to 80%?
2. If you have historical data, you can easily calculate your control limits and then determine your "yellow" and "red" zones using a 6 sigma approach.
3. I'll wager to say that Management Review happens more than twice a year, just not in a formal setting that you've document. Who owns the process related to customer satisfaction? Who owns the process(es) where client rejects, OTD, and quote time are reported? A mature system has the process owners keeping their eye on these metrics on a much more frequent basis
4. The idea of a preventive action after one yellow - something you mentioned earlier in thread - comes across as causing more work than adding value. Think of it this way...you find yourself speeding while driving, you take your foot off of the accelerator. You find yourself speeding a lot, now you start to analyze what's going on. We went with 3 consecutive yellows before a response was required from the owner of the metric.
5. We tried to set upper control limits, as well. These generated "good" reds. A red isn't necessarily bad - it simply means that your result is outside of the expected control limits. Sometimes we would discuss if the good red was sustainable or did we spend too much time on getting that result that we failed elsewhere?

Thanks RoxaneB
in Point one, really Is there a Math fórmula or equation to get this objective? Do you think all organizations make this calculation?
Honestly, I didn't, it was an objective set with no calculations, the idea was to measure it at the end of the year, it complies or not . Never have seen an auditor to ask at this deep level.
Could you provide an example in detail of objective calculation?
In the next points your are right, managing kpis by zones/ranges is complex , I think I'll stop to use this approach.
Other last point, I'm agree, while responsibles have compromise with the system, they must do a close follow up of their kpis, taking actions and not to wait the colored zones of colors to do something.
Thanks, I'm very interested in your feedback of objectives calculations.
Thanks

 

[Bev D]

Qualprod. - it’s not really a calculation of ranges. That is just too complex a mathematical exercise for judging when the focus should be on helping teams establish appropriate goals for the business and then helping them achieve those goals. More complicated ways to slap them with a ‘traffic ticket’ is not helpful.

Basic SPC to trend to the results is all you need. (Probably just an I, MR chart).
If you see action to improve the system and the results are trending in the positive direction - All is well.
(Green)

If you see no action and/or no improvement trend you need to adjust the actions to be effective (you may call this yellow or caution)

If you see no action and the trend is to a worse level - an out of control condition - then a course corrective is definitely needed (you may call this red)

Please note that you must use SPC thinking to determine a trend. One point up or down is not sufficient. Don’t confuse random variation for directed action...

 

[RoxaneB]

Thanks RoxaneB
in Point one, really Is there a Math fórmula or equation to get this objective? Do you think all organizations make this calculation?
Honestly, I didn't, it was an objective set with no calculations, the idea was to measure it at the end of the year, it complies or not . Never have seen an auditor to ask at this deep level.
Could you provide an example in detail of objective calculation?

In the next points your are right, managing kpis by zones/ranges is complex , I think I'll stop to use this approach.

As Bev indicated, there isn't really a formula for calculating what your KPI target should, but there is good business sense/knowledge. If, for example, your company is going to focus all of their resources (e.g., people, time, money) on increasing better quality products being made out on the shop floor, then there may not be anything left to find ways to help Sales improve their quotation speed.

You could also look as historical performance as a way to perhaps estimate what your organization is capable of this year. We would look at our previous results, look at what our objectives and goals were for the year and start to plan out our resources. If resources simply could not be allocated towards a goal, we did not set an improvement on it (or we kept it very low).

The whole idea is to set SMART targets. The R stands for Realistic. If no thought or consideration was given into setting the targets they aren't realistic. And if there is no plan to help the organization try to achieve them, then the targets are not A (Achievable) either.

You don't set targets for auditors. You set them for the company. If targets are selected with little to no consideration of what the company is trying to achieve, then you're setting yourselves up for failure AND people will not believe in the system.

In the next points your are right, managing kpis by zones/ranges is complex , I think I'll stop to use this approach.

It isn't too complex or complicated once you understand it. You don't need to start this approach with all of your indicators all at once. Start with a small group of metrics, the ones with the most visibility, and try it out.

 

[John Broomfield]

qualprod,

KPI monitoring seems to part of your preventive action

John

 

[Kronos147]

I would suggest thinking about this a bit differently. you should absolutely use SPC to determine if your KPIs are meeting the goals. simply have some arbitrary red yellow green range is not going to be helpful.

If you have historical data, you can easily calculate your control limits and then determine your "yellow" and "red" zones using a 6 sigma approach.

The idea of a preventive action after one yellow - something you mentioned earlier in thread - comes across as causing more work than adding value. Think of it this way...you find yourself speeding while driving, you take your foot off of the accelerator. You find yourself speeding a lot, now you start to analyze what's going on. We went with 3 consecutive yellows before a response was required from the owner of the metric.

This is a great discussion. One point I will add, in the scope of any registrar assessment, it should be stated in the opening meeting (ISO 19011:2018 6.4.3) scope and criteria should be established, which should include that the system conforms to (ISO 19011:2018 5.2) all relevant requirements e.g. statutory and regulatory, compliance, reqs for certification to a management system.

So if you need to put KPI's\objectives in place to meet the standard, and they have goals, and you don't hit goals, it IS non-conforming to your own requirements.

I'm in love with the green, yellow, red, and reviewing metrics over time in a simple chart format to identify trends. It is a firm demonstration to the top management commitment to the system that should engage all empoyees.