Departmental Performance Metrics - What's appropriate for Quality Assurance?

[normzone]

There's a movement afoot to choose some non-financial performance metrics that the QA department will track. I'm under pressure to pick something simple that is meaningless but easily trackable.

I'd like to choose something that we could actually work to improve to our benefit, but everything I'd like to improve is an issue that is outside of our direct control.

Do any of you have any metrics regarding your department's performance that you're tracking that you are content with or happy about?

Thanks
Norman Lutes

[wweng7]

Your departmental PPM.

[normzone]

Well, we're not exactly a million output kind of outfit. Ruggedized computers for the military.

I already track flaws found at inspection, and flaws found at test. I have considered tracking flaws that get past inspection that COULD have been caught by inspection but got found in testing, but that would generate a blip every few months and nothing more.

[gvalenti9]

Metrics should be 1) quantifiable, 2) observable and 3) meaningful.

If the metric is not meaningful, then why bother doing it?

I've used non-financial metrics to highlight processes that need improvement, for example. How many hours are being spent writing up defects--can that process be improved to reduce that burden? What is the DPMO by product--is one particular product in need of some help? What is the warranty return rate--are things that should be caught before shipment being missed, does a test that is being performed need to be changed or added?

All of the root-causes of these data may be out of your control, but if you can highlight the need for change (and something is done to address that need), then the metric is successful.

Greg

[wweng7]

Other options for performance metrics we are using is:

Target percentage of corrective and preventive action requests per:
• Reporting Source
• Type
• Subject

[Steve Prevette]

Originally Posted by normzone

There's a movement afoot to choose some non-financial performance metrics that the QA department will track. I'm under pressure to pick something simple that is meaningless but easily trackable.

I'd like to choose something that we could actually work to improve to our benefit, but everything I'd like to improve is an issue that is outside of our direct control.

Do any of you have any metrics regarding your department's performance that you're tracking that you are content with or happy about?

Thanks
Norman Lutes

In these days, there are very few efforts in a corporation that are within anyone's "direct control". If your measure is "meaningul" it is not likely under your direct control. So, go with it. If the measure reflects something important that QA has a role in, measure and learn from it. You may need to team with another department in order to deal with it, but "it" will become a win for the company.

[normzone]

Well, perhaps I've not expressed my original sentiment very clearly. Let me try again.

I believe that tracking quantifiable issues and seeking to improve trends is a good thing. I do it all the time.

My boss has been asked to set some criteria for his own performance that he can be measured against, and the same for his direct reports. I'm not certain if the goal of the exercise is to improve where improvements are possible, or if it's a holdover from the "merit pay will be based on performance against metrics" philosophy.

My boss is encouraging us to choose metrics within our control. My problem is, everything that is within my control is mundane and of small value to improve. For example, conformance to a three day time window to get my data entry and analysis done AFTER I receive the raw data from Production.

The exercise is academic. If I get it done in three days, it's because the rush is over and there was time to do it. If I fail to get it done in three days, it's because the rush was still on and I'd have to stop shipments to do the data entry and analysis.

There are plenty of things I can and do track that involve other departments, but I've been asked to choose goals germane to the QA department.

[Helmut Jilling]

Originally Posted by normzone

My boss has been asked to set some criteria for his own performance that he can be measured against, and the same for his direct reports. I'm not certain if the goal of the exercise is to improve where improvements are possible, or if it's a holdover from the "merit pay will be based on performance against metrics" philosophy.

There are plenty of things I can and do track that involve other departments, but I've been asked to choose goals germane to the QA department.

I would say that selecting metrics in a vacumn is futile. The process approach would have you define the important criteria for this Quality process first (4.1.c). When you define what is important, those cirteria will essentially define for you what should be measured (4.1.e). For some reason, we frequently get it in the wrong order, and it becomes somewhat undefined or even arbitrary.

If you can tell me the important criteria, then I can easily tell you what should be measured.