ISO9000 Performance Metrics

[Marc]

Subject: Re: Q: Metrics For ISO Performance /../Peter/Russo/Hodges
Date: Wed, 30 Dec 1998 12:17:26 -0600
From: ISO Standards Discussion

From: Jrhodges
Subject: Re: Q: Metrics For ISO Performance /../Peter/Russo/Hodges

It is true that 'quality systems' and 'business management systems' are inseparatable; however, no matter what terminology you call it there is a need to measure performance. Project manager's use cost, schedule, and performance to ascertain their failures and success. Why shouldn't the ISO Representative do the same -- just ensure the metrics he/she uses is not based on emotionalism but fact of data?

I use several metrics to ascertain the performance of those elements in which I am ultimately responsible (i.e., Management Responsibility, Quality System, Corrective and Preventive Action, and Internal Audits).

Management Systems - bar graphs showing the attendance of functional
areas and project/program management representatives at quality
meetings.

Quality System - Pareto charts showing the findings related to element
4.2 (e.g., finding not related to a specific element like 15,
procedures/processes not being used, etc)

Corrective and Preventive Action - A cumulative bar chart showing
(by month) the number of corrective and preventive actions opened,
overdue, and closed as well as a chart of all elements regarding Root
Cause quantities.

Internal Audits - A pareto analysis of last year's findings compared
to this year's finding and each audit showing the areas of findings.
This provides a clear picture -- try it.

Last but not least, I developed a nine question opinion survey which I provide to all of the management and project representatives. This opinion survey addresses all of the aforementioned areas (with validity questions) as well as Training. I ask their opinion and then develop a Radar Chart for presentation.

I find the results (anonymous) usually match the metrics -- what people say in private matches the results. If they don't truly support the ISO program, the metrics and radar chart clearly shows it.

If you have any questions, feel free to contact me.

Jake Hodges
Quality Manager & Adjunct Faculty

[Marc]

Subject: Re: Q: Metrics For ISO Performance /../Peter/Russo/Hodges/Scalies
Date: Wed, 6 Jan 1999 11:04:59 -0600
From: ISO Standards Discussion

From: Charley Scalies
Subject: Re: Q: Metrics For ISO Performance /../Peter/Russo/Hodges/Scalies

SNIP....ISO Standards Discussion wrote:

> I use several metrics to ascertain the performance of those elements in which
> I am ultimately responsible (i.e., Management Responsibility, Quality System,
> Corrective and Preventive Action, and Internal Audits).

Are these the only metrics you are using for these elements? My comments (following) assume they are.

> Management Systems - bar graphs showing the attendance of functional
> areas and project/program management representatives at quality
> meetings.

I once worked for a company where very little real progress was made because everyone was always in meetings. How does this metric confirm/refute that you're system is effective at meeting the intent of 4.1?

> Quality System - Pareto charts showing the findings related to element
> 4.2 (e.g., finding not related to a specific element like 15,
> procedures/processes not being used, etc)

This has some limited value, but IMHO an effective quality system is not a collection of 20 disparate elements, but is one thing. Are your inquiries limited to "documenting what we do" or do they extend to measuring the effectiveness of the documented system at achieving the desired results? Can we assume that there are others in your organization who are responsible for other micro elements? And who is responsible for determining if the whole thing works? That's what it really is all about.

> Corrective and Preventive Action - A cumulative bar chart showing
> (by month) the number of corrective and preventive actions opened,
> overdue, and closed as well as a chart of all elements regarding Root
> Cause quantities.

Again this has some limited value. However, how would you interpret a sharp reduction in the number of CARs/PARs opened? Is that "good" - the system is performing better - or "bad" - we stopped counting?

> Internal Audits - A pareto analysis of last year's findings compared
> to this year's finding and each audit showing the areas of findings.
> This provides a clear picture -- try it.

Limited value. Findings without verified causes are just clues.

> Last but not least, I developed a nine question opinion survey which I provide
> to all of the management and project representatives. This opinion survey
> addresses all of the aforementioned areas (with validity questions) as well as
> Training. I ask their opinion and then develop a Radar Chart for
> presentation.
> I find the results (anonymous) usually match the metrics -- what people say in
> private matches the results. If they don't truly support the ISO program, the
> metrics and radar chart clearly shows it.

If people don't truly support the quality program, the nonfulfillment of the quality policy will show it.That's the Prize! My experience demonstrates that unless you are measuring at the macro level (i.e the forest) you will get lost in the trees.

Here is my challenge to anyone who cares to accept it.
1. Boil your quality policy down to its essence: 25 words or less.
2. Then list no more than five "discernables" you would assess or measure
that tell how well you are achieving that policy.

Charley

[Don Winton]

Once again, Charley has made very valid points. I agree.

Regards,
Don

[Marc]

Yeah - I like Charley's comments a lot of the time, too. He generally has a come back.

Metrics issues interest me a lot right now, which is why I threw this up for comment.

[barb butrym]

If people don't truly support the quality program, the nonfulfillment of the quality policy will show it.That's the Prize! My experience demonstrates that unless you are measuring at the macro level (i.e the forest) you will get lost in the trees.

Here is my challenge to anyone who cares to accept it.
1. Boil your quality policy down to its essence: 25 words or less.
2. Then list no more than five "discernables" you would assess or measure
that tell how well you are achieving that policy.
----------------------------------------

YUP...thats the way I do it all the time...any more and you get lost...Less is More....my favorite quote

[Kevin Mader]

Charlies comments are appropriate. I also especially like the comments made on the creation of the Quality Policy and the discernables. Short, realistic, and belivable would be my three key inputs in the creation of the Quality Policy and the key Quality System measurables. Keep it simple but effective (to Barb's point, less is more!).