I'm putting together a scorecard for my company and I was wondering if there were any key metrics I was missing. I know this can vary quite a bit, depending on the business and what the current needs are, but I wondered if we could come up with a list of core/"must haves" for a manufacturing company. Here is what I have so far:

1) #Field Claims / #Shipments (% by month)
2) #Production Nonconformances
3) Scrap$
4) #Safety Incidents
5) #Supplier Line Items Returned / Line Items Received (%)
6) #ECNs Created
7) #Waste or Error Correction ECNs Created
8) #Failed First Articles / #First Articles Performed (%)
9) #Internal Audit Nonconformances

I have a few others that are more custom to our business, but here I thought I'd start with the core from manufacturing (Purchasing, Inventory, Production, Shipping, Engineering, etc). I know service related businesses would probably have even more metrics - but I'd like to limit this thread to manufacturing. Sorry service folks - start your own thread.

I don't care for 6 and 7, but I'm looking for a good way to measure how Design Engineering is doing.

Fire away!
Thanks,
Andy 7/13

A biggie for us is machine/line uptime or utilization.

I'm putting together a scorecard for my company and I was wondering if there were any key metrics I was missing. I know this can vary quite a bit, depending on the business and what the current needs are, but I wondered if we could come up with a list of core/"must haves" for a manufacturing company.

Ahhh....Key Performance Indicators....something my life appear to revolve around of late. The core or "must haves" for KPI's are up to you, or rather your organization, to determine. But when developing them, ensure you ask the following:


What are we measuring?
Why are we measuring it?
What does it mean to us?
What do we hope to gain from measuring this?
How will we measure it?
How will we use the data?
Who is responsible for generating/collecting/analyzing/responding to the data?
How frequently will be measure/collect/analyze/discuss the data?
Does this impact our ability to meet requirements (be it Safety, Environment, or Quality)?
Is the process under control at this time? Do we have acceptable limits to indicate when we are under control and when we are not?
How will we take action on KPI's not under control?


Andy, I don't know who developed your list, and perhaps you have started off with a high-level organization KPI list, but my suggestion would be to start at the bottom and work your way up. Talk with the the departments on an individual basis. Find out what they feel is vital to keeping their own department/process alive and under control. Find out what seems to be triggers to them that all is not as it should be.

From these lists and discussion, you might that your core list of KPI's is substantially different than what you have at this time.

1) #Field Claims / #Shipments (% by month)

Assuming "Field Claims" equate to "Customer Complaints", I like this. We do something very similar - we then break it down to 1 Complaint / X tons shipped. The higher the X value, the better we have done. But we also break down this KPI into four categories. Invoice Complaints and Service-Sales Complaints are the responsibility of Sales to resolve. Service-Mill and Quality Complaints are the mills' responsibility to resolve.

2) #Production Nonconformances

At a high level this is a number that is almost meaningless until you:


Have previous years' data so that you can compare where you are know to where you were then (assuming you have conistent cycles).
Filter it down to the crew/department level, so you can analyze if there is a particular problem area.


3) Scrap$

Cost of unQuality basically, right?

4) #Safety Incidents

We break ours down into Medical Aids and Lost Time Accidents. We also then do the frequency rate and compare ourselves to those organizations in the same industry. Yes, one accident is one too many, but it is good to see how we do in comparison to others.

5) #Supplier Line Items Returned / Line Items Received (%)

Unsure what this means...could be that you use difference terminology than what I am used to.

6) #ECNs Created

What about ECNs closed? Or ECNs open > X days? Or ECNs open > X days with Actions Pending? "#ECNs Created" does not provide much information on the status or effectiveness of Engineering. The others help to show process efficiency...or prompt discussion on the allocation of resources.

7) #Waste or Error Correction ECNs Created

We look at similar items on a category basis. Safety, department, crew, particular process aspect...

8) #Failed First Articles / #First Articles Performed (%)

Don't know what this is.

9) #Internal Audit Nonconformances

What about External? What about Opportunities for Improvement? What about breaking it down to minors and majors, instead of lumping them all together? What about time to resolve? Internal Audits are supposed to be beneficial and help the organization...their KPI should show the improvement. :)

I have a few others that are more custom to our business, but here I thought I'd start with the core from manufacturing (Purchasing, Inventory, Production, Shipping, Engineering, etc). I know service related businesses would probably have even more metrics - but I'd like to limit this thread to manufacturing.

We also show such KPI's like:


Utilization
Energy consumption metrics
Waste to Landfill ratio
% Overtime
...and so on...


Like I said....KPI's are my life, of late....learning to love them as a tool of visual management.

How will you be communicating the KPI's? The results, the status, etc.? Stop-light colouring? Okay, okay...getting all excited here...

Great questions. Thanks.
Some more thoughts or philosophies, if you will....

The list of questions you would ask about a metric is a good one, but before I ask any of those I ask, "is improving this metric wildly important to our company?" Taken from a Franklin/Covey seminar on execution. The list is intended to be high level, something for management review. Other departmental metrics like % utilization are also important and we track that too, but I left that off as something that Manufacturing would monitor but we would not focus on for management review.

I also agree you need to break down each metric into categories. Each metric I listed is going to be displayed on a one page "football field," as an overview since we are a lighting company. And then each metric will also have a tab in the spreadsheet (see sample attached) showing top level Pareto and current corrective actions in place.

I disagree with starting at the bottom and working up to identify metrics. I think you come up with too many, maybe not meaningless but not critical to the company. I believe the list should be short as possible. Each department manager should probably not have more than two key ones to focus on at any given time otherwise effective execution is not possible.

Andy

1) #Field Claims / #Shipments (% by month)
Assuming "Field Claims" equate to "Customer Complaints", I like this. We do something very similar - we then break it down to 1 Complaint / X tons shipped. The higher the X value, the better we have done. But we also break down this KPI into four categories. Invoice Complaints and Service-Sales Complaints are the responsibility of Sales to resolve. Service-Mill and Quality Complaints are the mills' responsibility to resolve.

2) #Production Nonconformances
At a high level this is a number that is almost meaningless until you:

Have previous years' data so that you can compare where you are know to where you were then (assuming you have conistent cycles).
Filter it down to the crew/department level, so you can analyze if there is a particular problem area.


3) Scrap$
Cost of unQuality basically, right?

4) #Safety Incidents
We break ours down into Medical Aids and Lost Time Accidents. We also then do the frequency rate and compare ourselves to those organizations in the same industry. Yes, one accident is one too many, but it is good to see how we do in comparison to others.

5) #Supplier Line Items Returned / Line Items Received (%)
Unsure what this means...could be that you use difference terminology than what I am used to.

6) #ECNs Created
What about ECNs closed? Or ECNs open > X days? Or ECNs open > X days with Actions Pending? "#ECNs Created" does not provide much information on the status or effectiveness of Engineering. The others help to show process efficiency...or prompt discussion on the allocation of resources.

7) #Waste or Error Correction ECNs Created
We look at similar items on a category basis. Safety, department, crew, particular process aspect...

8) #Failed First Articles / #First Articles Performed (%)
Don't know what this is.

9) #Internal Audit Nonconformances
What about External? What about Opportunities for Improvement? What about breaking it down to minors and majors, instead of lumping them all together? What about time to resolve? Internal Audits are supposed to be beneficial and help the organization...their KPI should show the improvement. :)

1) Agree with breaking it down, see my last post.
2) Agree
3) Basically yes, I should also include rework.
4)
5) Line item is P.O. line item, basically lots nonconforming vs lots received
6) 2 schools of thought on ECNs, one is that all ECNs represent waste -- "why wasn't it designed right in the first place?", and two is that all ECNs are value added -- you're making a change that is an improvement. In truth, both are correct, so what should you measure? I don't like #created or #closed, because who is to say what is the right number? Cycle time is good but also difficult because some changes could be done in minutes, but other design changes could take months.
7) I like to focus on waste ECNs: those needed to correct a tolerance, properly define a datum, correct an error, etc. Anything that should have been caught during a properly run design review.
8) First articles are 100% dimensional inspections we perform on new piece parts or significant changes or new tooling. It is basically to qualify the tooling and sometimes we accept the supplier's measurements.
9) Agree externals should be included as well.

Thanks,
Andy

Always open to more thoughts and philosophies! :D What's works for one organization does not necessarily work for another...

The list of questions you would ask about a metric is a good one, but before I ask any of those I ask, "is improving this metric wildly important to our company?" Taken from a Franklin/Covey seminar on execution. The list is intended to be high level, something for management review. Other departmental metrics like % utilization are also important and we track that too, but I left that off as something that Manufacturing would monitor but we would not focus on for management review.

So what is management to review? What do you expect the outputs to be from their review? And do the KPI's you listed address these outputs?

I also agree you need to break down each metric into categories. Each metric I listed is going to be displayed on a one page "football field," as an overview since we are a lighting company. And then each metric will also have a tab in the spreadsheet (see sample attached) showing top level Pareto and current corrective actions in place.

If that works for your organization, great! My experience has show that a simple chart using the stop-light colouring method allows all people at all levels to understand at a glance how things are going. Green means the results demonstrate a controlled process. Yellow means we're outside of control and steps are being taken, where practical, to get back under control. Red means we're outside of control for the third month in a row and some serious root cause analysis is required - no more band-aid fixes.

I disagree with starting at the bottom and working up to identify metrics. I think you come up with too many, maybe not meaningless but not critical to the company. I believe the list should be short as possible. Each department manager should probably not have more than two key ones to focus on at any given time otherwise effective execution is not possible.

Why must the department manager focus on the KPI? Why not have team leaders do this? Why not have people from the floor spearhead a focus on a KPI? This helps in morale, involvement, and communication.

The reason I prefer a "bottom's up!" approach is because the guys on the floor know the process the best. They know what is important. They can tell us what impacts the process. Of course, this is for production-based KPI's only....you probably know best what to report for audits, etc.

From their input, supervisors and managers can filter through and determine those KPI's that have a strong financial, safety, environmental, quality focus.

From that point, upper management can focus on the KPI's that best show the overall health of the organization.

This approach works for us. It may not work for everyone. I do know, however, that implementing the team-based environment, along with stronger communication tools, between all levels has helped improve not only morale, but it has helped our KPI's and our understanding of our own processes.

However, if what you have works for you, great! :agree1: Just don't let that stop you from considering alternate ways of collecting/analyzing/presenting the data. Even our method does not remain stagnant - every year there is at least one minor change to our KPI process.

Andy[/QUOTE]

So what is management to review? What do you expect the outputs to be from their review? And do the KPI's you listed address these outputs?

Management reviews all of the KPI's, and the corrective actions in place to drive improvement in those metrics. They also look for weaknesses or improvements that could be made to the overall Quality Management System.

What do you think of the metric #7, # of waste ECN's created?
Is this a good one to track for design engineering?

Andy

Management reviews all of the KPI's, and the corrective actions in place to drive improvement in those metrics. They also look for weaknesses or improvements that could be made to the overall Quality Management System.

We have several levels of KPIs and several levels of review. At the department level, a department reviews all of their specific KPIs and details associated with them. Usually, there a few members of Management there, as well, to keep up to speed on what's happening.

At the Management level, they review the KPIs they feel are the best to demonstrate how the company is doing. The do not review all KPIs - no time to do that and it would not be an effective/efficient useo f their time.

What do you think of the metric #7, # of waste ECN's created?

How will it be determined that an ECN is due to an improperly run design review or otherwise?
Is this a good one to track for design engineering?

Andy[/QUOTE]

Andy

Are you still looking for ideas for design metrics?

If you have member access to AIAG please consider looking at:

AIAG D-13 Product Development Metrics
AIAG D-7 A snapshot of product development practices in automotive supply chains, and
AIAG PM-1 Automotive project management guide

There are numerous well thought out metrics included in these.

If you don't have access, I could try to summarize some of the ideas.
Cheers
Joe

I don't have member access to AIAG. A brief summary of some design engineering stats would be helpful.

Thanks,
Andy

At the Management level, they review the KPIs they feel are the best to demonstrate how the company is doing. The do not review all KPIs - no time to do that and it would not be an effective/efficient use of their time.

Agreed.


How will it be determined that an ECN is due to an improperly run design review or otherwise?
Is this a good one to track for design engineering?

I guess it would depend how you would define a 'waste ECN.' Many times ECN's are to increase a tolerance on a dimension that failed an inspection. Other times a part is changed to make it assemble easier. If it is a new design, I would expect the design and mfg engineers to do a better job during review to make sure the tolerances are reasonable, and then test the design in a pilot run to verify that it would work for assembly.
My thoughts would be the in the database when the ECN is created, the engineer could mark it as 'waste, improvement, or other,' and then the 'waste' ECN's could be queried and analyzed each month.

Andy

I'm putting together a scorecard for my company and I was wondering if there were any key metrics I was missing. I know this can vary quite a bit, depending on the business and what the current needs are, but I wondered if we could come up with a list of core/"must haves" for a manufacturing company. Here is what I have so far:

1) #Field Claims / #Shipments (% by month)
2) #Production Nonconformances
3) Scrap$
4) #Safety Incidents
5) #Supplier Line Items Returned / Line Items Received (%)
6) #ECNs Created
7) #Waste or Error Correction ECNs Created
8) #Failed First Articles / #First Articles Performed (%)
9) #Internal Audit Nonconformances




I have a few others that are more custom to our business, but here I thought I'd start with the core from manufacturing (Purchasing, Inventory, Production, Shipping, Engineering, etc). I know service related businesses would probably have even more metrics - but I'd like to limit this thread to manufacturing. Sorry service folks - start your own thread.

I don't care for 6 and 7, but I'm looking for a good way to measure how Design Engineering is doing.

Fire away!
Thanks,
Andy 7/13


Andy, I think these are all good ideas. I dont do #7 but #6 is really big! We use it as a measure of Engineering Dept. efficiency. Further, we target the cost of production time as a result of poor design.
Poor design usually is first discovered on the production floor when the operator says there is a problem. This is usually in the form of interference type problems and/or dimensional.
After we initiate an ECR we record the time it takes to correct the problem plus the $$ it cost the company for each job number. These cost are kept on an Excel log and reviewed by Engineering & Production Management.
The log also records the designer's initials, a brief description of the problem, and references the ECR#.
We also track how much time it takes Engineering to issue the ECN once the ECR has been submitted.
:agree1:

Andy,
I will get back to you on this thread with details during the weekend.

One quick note though...
I saw your Sample.xls file (Supplier Quality story board). Here is a suggestion:
We use a presentation called "Quality Operating System" (QOS), the story board is presented similar to the form of a Paynter Chart.
1st Quadrant with the historic trend (12 month rolling average), 2nd Quadrant is this month break-up of defect pareto. Third and forth Quadrant occupied by a table that monitors the Top candles of the pareto on a weekly basis. Responsibility assigned in one column, CA identified in another column. I understand this approach is widely used in automotive manufacturing. Check it out.
Govind.

Andy
As a newbie here (and from england), excuse me for coming in half way through the debate but i would also suggest that you also include Company financial performance indicators (profit, loss, turnover etc) . Seeing how the "management" is performing is vital as they are responsible for having enough cash to provide the "resources" you need.

Andy, I think these are all good ideas. I dont do #7 but #6 is really big! We use it as a measure of Engineering Dept. efficiency. Further, we target the cost of production time as a result of poor design.
Poor design usually is first discovered on the production floor when the operator says there is a problem. This is usually in the form of interference type problems and/or dimensional.
After we initiate an ECR we record the time it takes to correct the problem plus the $$ it cost the company for each job number. These cost are kept on an Excel log and reviewed by Engineering & Production Management.
The log also records the designer's initials, a brief description of the problem, and references the ECR#.
We also track how much time it takes Engineering to issue the ECN once the ECR has been submitted.
:agree1:

So are the ECRs that are tracked in the spreadsheet just the ones that were discovered as problems on the production floor and initiated by Production?
I think that would be a good way for me to identify what I call 'waste' ECNs. Thanks. :thanx:

Andy

Andy
As a newbie here (and from england), excuse me for coming in half way through the debate but i would also suggest that you also include Company financial performance indicators (profit, loss, turnover etc) . Seeing how the "management" is performing is vital as they are responsible for having enough cash to provide the "resources" you need.
Thanks wynyards. I think you are very correct that we would want to definitely also see improvements in the financials, and if not, maybe it would indicate that we might not be looking at the right QMS performance indicators.
Andy

You got it! Actually the idea was floated by the Engineering Manager in an attempt to improve the quality of his output! I thought it was a great idea! I might add that I track all ECR's on a seperate spread sheet that is reviewed by the VP's.

we've talked heartily about What to track, but not really what to do with the numbers as received. So much of the time (in my plant at least) these 'numbers' become fodder for unrestrained management by objective.

the problem is no matter how cleverly we try to pick the right numbers to look at, pushing the plant towards the highest achievement (and keep the axe sharp when goals are missed) breeds fear, shortcuts, lack of long term planning, and mistakes.

Also variation is normal (we're QA people and we know that), management DOESN'T seem to understand though- and any lower number results in people be blamed/'accountability' occurring. Instead of letting our process sit controlled in ever lessening common cause variation. management starts swinging with an axe--
One months missed production quotas becomes next months customer complaint as people are pushed into shipping questionable material.

You've asked the question, what's the best metrics but my feeling is the focus should be more on how to use them correctly; if you can record numbers correctly and use them to better you company-- lots of obvious, useful metrics are all about you. and I might say-probably your continuos improvement strategies should point you to the very best ones.

Just my :2cents: and I'll try to stay off that high :horse: for the rest of conversation.

You've asked the question, what's the best metrics but my feeling is the focus should be more on how to use them correctly; if you can record numbers correctly and use them to better you company-- lots of obvious, useful metrics are all about you. and I might say-probably your continuos improvement strategies should point you to the very best ones.

If you read Post#3 in this thread, how to use them is part of the list of questions I think organizations should ask themselves in determining if something should be a performance indicator or not.

Part of using them correctly is understanding why goals/objectives were not achieved and allocating the resources to help them be achieved next time.

We focus primarily on the indicators that have a direct impact on our ability to meet the company objectives like safety, environment, customer satisfaction, $ gaps, process development/implementation...and while we don't jump on the manager's who do not meet the goals, we do expect them to come to the meeting with answers as to what happened, why it happened, and what will be done so that it doesn't happen again.

It's an evolution, though, for Management to do this...to step back and let the people who do the work, actually do their jobs. Some management people still think old-school, sometimes, and feel they must harangue and question and restrict creativity in order for the company to succeed.

Amazingly enough, however, when people have leadership that simply involves resource allocation, communication, and an end goal....usually we are much better at achieving the goal than we would if the old-school style were still in place here.

As for using the indicators for CI, that is all part of our Annual Planning Process. We review the year's data...we review past data...we look at what we done, what works, what doesn't work. We use what our organization's management technology calls "7 Quality Tools" and "7 Planning Tools".

The following is a story about the approach to kaizen (Continuous Improvement) observed in a Japanese company. Just some food for thought about improvement & analysis!

"Years ago when we first visited ____ in Japan, we were struck by something that seemed out of keeping with their continuing success. They seemed to worry all the time. We met managers who had just accomplished remarkable feats of muda (waste) removal during kaizen events and yet they couldn't seem to just relax and enjoy it. Instead they were busy analyzing what they had just done and trying to think of ways it could be even better. We began to say to ourselves, "Even smiling is muda at _____."

By contrast, kaizen events in other firms we have visited over the years have often been marked by celebrations and self-congratulation, no matter how much was really accomplished or how modest the improvement goal.

Recently, while reading's Jeff Liker's excellent new book "The ____ Way", I came across the section on "hansei" or reflection, which for _____ is the third step in their PDCA process for every improvement. It helped me put in words what I've been feeling for years about the difference between kaizen at ____ and at most other firms.

_____'s idea is simply that every time we analyze a situation (the "P" for "plan" in PDCA) and then try a new way (the "D" for "do"), it's time to reflect very carefully (hansei) on what we have just done. (This, of course, is the "C" for "check".) In most companies the fact that the new performance of a system met expectations is the end of the discussion. And if it didn't meet expectations, this is only to be expected with some experiments. (Remember that controlled experimentation using the scientific
method is what kaizen really is.)

But for _____ it seems to be very different. In their view, if the performance met expectations, surely they could have done even better. The performance objective was too modest. And if the performance did not meet expectations, something was wrong with the original plan and it is important to understand what and why. Indeed, this is yet another use for the "five whys", but this time applied to the improvement process itself.

But note that in either case there is no room for celebrating what has been achieved even if the results are substantial or for celebrating the conduct of a noble experiment even if it failed. The objective must be to either find out why it didn't work or why it didn't work even better. In short, with hansei _____ has a formula for standardized worrying!

Surely this seems harsh. Can't these guys just have some fun? But the sobering reality of life, at least for me, is that folks who worry every day about every thing are very likely to have little to worry about in the long run. By contrast, those who grade their kaizen performance on how hard they tried -- whatever the results -- are likely to be very happy in the short term but soon may be looking for work.

So please give some thought to how you approach kaizen. If no improvement in performance is ever good enough and if every kaizen failure undergoes meticulous analysis to understand exactly why it didn't work, you too will become a standardized worrier. And, very likely, you will become a leader in a continuously successful enterprise."

First off, great article, gerrybean! :agree1:

But note that in either case there is no room for celebrating what has been achieved even if the results are substantial or for celebrating the conduct of a noble experiment even if it failed. The objective must be to either find out why it didn't work or why it didn't work even better. In short, with hansei _____ has a formula for standardized worrying!

Surely this seems harsh. Can't these guys just have some fun? But the sobering reality of life, at least for me, is that folks who worry every day about every thing are very likely to have little to worry about in the long run. By contrast, those who grade their kaizen performance on how hard they tried -- whatever the results -- are likely to be very happy in the short term but soon may be looking for work.

So please give some thought to how you approach kaizen. If no improvement in performance is ever good enough and if every kaizen failure undergoes meticulous analysis to understand exactly why it didn't work, you too will become a standardized worrier. And, very likely, you will become a leader in a continuously successful enterprise."

How to explain what it is we do at my organization? We plan. We start planning in September. We look at the data from previous years. We look at the data from the current year. We look at the abnormalities, the causes, the actions plans...we look at the available resources...and we plan for improvement.

When we achieve the goal, we celebrate. To constantly strive for a goal and not reward those who worked to achieve is unacceptable. A pizza party, a pen, a word of thanks from the CEO...all are simple ways of acknowleding a job well done...something that we need for personal satisfaction.

When we do not acheive the goal, we ask why 5 times, we do Ishikawa, we do more planning. We do not spend time pointing fingers or dancing around the fire of 'he-said-she-said'...we analzye, we plan, we execute.

We are still in 2004...I do not know how it will end...if our goals will be met. But we start planning for 2005 in just over a month. Hopefully we will celebrate in January that we met our goals for 2004, but there will be little time for much fun, as we have actions to be done to help us achieve our goals in 2005.

But we still acknowledge the people who got us to our goal.

But for _____ it seems to be very different. In their view, if the performance met expectations, surely they could have done even better. The performance objective was too modest. And if the performance did not meet expectations, something was wrong with the original plan and it is important to understand what and why. Indeed, this is yet another use for the "five whys", but this time applied to the improvement process itself.

But note that in either case there is no room for celebrating what has been achieved even if the results are substantial or for celebrating the conduct of a noble experiment even if it failed. The objective must be to either find out why it didn't work or why it didn't work even better. In short, with hansei _____ has a formula for standardized worrying!"

A standardized worryer in the managerial ranks equates to a lack of appreciation on the shop floor. Here the process owners ,whom are usually your lowest paid workers have put out effort to accomodate, assist and/or even perform 'miraculous improvements of Kaizen.' Instead of rewards and appreciation is criticism and fault finding. People do best when they feel they are winning (not comfortable but winning none the less). remember, as well that management has its own rewards when profit soars. for the shop floor owner nothing is really different unless management is willing to share the accomplishment.
in short I feel this attitude is unfair to the efforts of the lower ranks of the company. no process improvement should lack appreciation for the workers / and technical staff when improvements are made.

Follow the link:

http://clubpmi.cuoa.it/

I'm afraid it's just in italian!

"Club PMI" stays for "small- size enterprises club".
Here is listed a series of indicators, not just about effectiveness but efficiency too.

They are classified for:
Gestione dell'ordine: order management;
Offerta: offer;
Acquisizione ordine: Order booked;
Sviluppo ordine AND programmazione della produzione: prduction planning and scheduling;
Acquisti: purchasing;
Produzione: production;
Fatturazione attiva e passiva: active and passive invoicing;
Post vendita: post sales;
Logistica in entrata, in uscita ,in produzione: incoming, shipment, in production logistic.


For each indicator is given:
Codice: Code;
Nome Indicatore: indicator name;
Definizione indicatori: it's a indicator's description;
Tipologia indicatore AND Unità di misura: are the metrics used.

I found it usefull, and maybe too complete :D .
Bye bye

9) #Internal Audit Nonconformances

What about External? What about Opportunities for Improvement? What about breaking it down to minors and majors, instead of lumping them all together? What about time to resolve? Internal Audits are supposed to be beneficial and help the organization...their KPI should show the improvement. :)
Sorry All, but back on the topic of the original post (I'm still looking to improve my top level, mangement review list of metrics), and I'm considering removing Audit Nonconformances from the list.

Don't get me wrong, I would still like to review internals, externals, and observations during management review, but I'm thinking of just using them as a supplement to the review of the other metrics, and not have a separate metric for # of nonconformances.
What would constitute improvement in this metric? If the # of nonconformances decreases month-to-month, does that really mean the system is improving?

Great questions. Thanks.
Some more thoughts or philosophies, if you will....

The list of questions you would ask about a metric is a good one, but before I ask any of those I ask, "is improving this metric wildly important to our company?" Taken from a Franklin/Covey seminar on execution. The list is intended to be high level, something for management review. Other departmental metrics like % utilization are also important and we track that too, but I left that off as something that Manufacturing would monitor but we would not focus on for management review.

I also agree you need to break down each metric into categories. Each metric I listed is going to be displayed on a one page "football field," as an overview since we are a lighting company. And then each metric will also have a tab in the spreadsheet (see sample attached) showing top level Pareto and current corrective actions in place.

I disagree with starting at the bottom and working up to identify metrics. I think you come up with too many, maybe not meaningless but not critical to the company. I believe the list should be short as possible. Each department manager should probably not have more than two key ones to focus on at any given time otherwise effective execution is not possible.

Andy


Dear Andy Nutt:

I am new to the Cove and getting familiar with my surroundings. I couldn't help looking at the Excel file you had attached. And, if you do not know already, I got totally NUTS (no pun intended with your last name) whenever I see a table of x and y values. I immediately start looking for a pattern. So, I prepared a graph of the Items Received (x) and Items Returned (y) in your Excel file. I have pasted the values from your Excel file below. I sorted the data by Items Received, x. The percent returned is simply the ratio y/x = m multiplied by 100. Guess what? I find something interesting, just like I suspected.


Month Percent Items Items
Returned Received, x Returned, y


Nov-03 3.71 1131 42
Jan-04 1.46 1305 19

Feb-04 3.67 1361 50
Aug-03 1.47 1565 23
Sep-03 1.56 1604 25
Oct-03 1.98 1619 32
Mar-04 3.84 1641 63
Dec-03 2.42 1697 41
Jul-03 2.15 1817 39


When we calculate percentages, as you have, and this is also what most of us do, we are looking at each (x, y) pair in isolation. Let's take the first point for Nov 2003. The slope of the straight line joining the point P (x, y) back to the origin of the x-y graph has a slope m = y/x = 42/1131 = 0.0371. Multiply by 100 and you get the percent. Now, we do the same with the second point for January 2004. The slope m = y/x = 19/1305 = 0.0146, or 1.46%. So, we conclude that Jan 2004 was a better month than November 2003. Of course, we don't know why. Is this all just "statistical fluctuation"? We need to analyze the data much more carefully. Then we get into sample sizes and the like and normality and many other things. A few years ago, I thought of, what I believe is, a simpler approach. Let's see if this works here.

I prepared a graph of x versus y instead of calculating the percentages. It is so easy to do this Excel. Then I stare at the graph for a few minutes. This usually tells me something and quite ofen I start seeing points falling on roughly parallel lines. I tried this with the above and BINGO! Three points fall on, or very close, to the line A with the equation y = hx + c where h = 0.041 and c = - 4.57. These three points are:

1131 42
1361 50
1641 63

I fixed the value of h by considering the two extreme points here. Then, I found three more points falling on a parallel line with the equation y = 0.041x - 35.82. These points are:

1305 19
1619 32
1817 39

One point (1697, 41) stands all alone. The remaining two points seem to fall on another parallel. They are really close together on the graph as you can see from the x and y values given below.

1565 23
1604 25

What does all this mean? As I interpret it, we are merely seeing what common sense tells us. Are these "events" truly isolated? I guess not. The Items that you receive and the items that are returned, month after month, have something in common. Your company is probably manufacturing the same widget (please correct me, if I am wrong). Some "defects" occur and the customer is unhappy and returns some of them. So, the "events" are kind of related. This is why we see the law y = hx + c. What this tells us is that as the number of items received (x) increases, the number of items returned (y) also increases. However, the items returned must be less than items received. So the constant h < 1 and we fix the numerical value by considering the (x, y) pairs as we have just discussed. The nonzero c simply means that when the number of items received is doubled, the number of items returned does not double. In other words, the straight line on our x-y graph does not pass through the origin. Instead of different "m" for each point, we have a different "c" and a single value of h.

Take the straight line y = 0.041x - 4.57. The ratio y/x = 0.041 - (4.57/x). As x increases, the ratio y/x, or the percent returned will increase because of the correction term - 4.57/x from the nonzero c. Although the percent returned is increasing the slope h is fixed, which means that the number returned is increasing at a fixed rate (we call it the derivative) as the number received increases. When c is more negative, you have "good" months and the number of items returned decreases. When c increases, you have "bad" months and the number of returns increases. Instead, of calling all this just the intercept c, I created a real fancy term called the "work function". This is discussed in more detail in my August 2004 article, Is there a law relating Views and Replies?

Is there a law relating Items Received and Items Returned? I think so. Let me know what you think. Once, we begin to get a handle on this "work function", profits should start soaring! You can start focussing on why the work function keeps fluctuating from one month to the other. Might sound like old wine in a new bottle, but we all fall for NEW BOTTLES and Labels, don't we? But, seriously, I think, there may just be a lot more to all this. I see y = hx + c in too many places, often in the most unexpected places. If you are still reading this ... let me know.

Charmed :) :thanx:

P. S. Einstein was a very smart guy, RIGHT. It was he who first thought of thing called the work function. This is just a nice explanation for the intercept c in the equation K = hf - W which describes photoelectricity. Now, we say y = hx + c = hx - W also implies a work function! I better take a few days off now.

Dear Andy Nutt:

I am new to the Cove and getting familiar with my surroundings. I couldn't help looking at the Excel file you had attached. And, if you do not know already, I got totally NUTS (no pun intended with your last name) whenever I see a table of x and y values. I immediately start looking for a pattern. So, I prepared a graph of the Items Received (x) and Items Returned (y) in your Excel file. I have pasted the values from your Excel file below. I sorted the data by Items Received, x. The percent returned is simply the ratio y/x = m multiplied by 100. Guess what? I find something interesting, just like I suspected.


Month Percent Items Items
Returned Received, x Returned, y


Nov-03 3.71 1131 42
Jan-04 1.46 1305 19

Feb-04 3.67 1361 50
Aug-03 1.47 1565 23
Sep-03 1.56 1604 25
Oct-03 1.98 1619 32
Mar-04 3.84 1641 63
Dec-03 2.42 1697 41
Jul-03 2.15 1817 39


When we calculate percentages, as you have, and this is also what most of us do, we are looking at each (x, y) pair in isolation. Let's take the first point for Nov 2003. The slope of the straight line joining the point P (x, y) back to the origin of the x-y graph has a slope m = y/x = 42/1131 = 0.0371. Multiply by 100 and you get the percent. Now, we do the same with the second point for January 2004. The slope m = y/x = 19/1305 = 0.0146, or 1.46%. So, we conclude that Jan 2004 was a better month than November 2003. Of course, we don't know why. Is this all just "statistical fluctuation"? We need to analyze the data much more carefully. Then we get into sample sizes and the like and normality and many other things. A few years ago, I thought of, what I believe is, a simpler approach. Let's see if this works here.

I prepared a graph of x versus y instead of calculating the percentages. It is so easy to do this Excel. Then I stare at the graph for a few minutes. This usually tells me something and quite ofen I start seeing points falling on roughly parallel lines. I tried this with the above and BINGO! Three points fall on, or very close, to the line A with the equation y = hx + c where h = 0.041 and c = - 4.57. These three points are:

1131 42
1361 50
1641 63

I fixed the value of h by considering the two extreme points here. Then, I found three more points falling on a parallel line with the equation y = 0.041x - 35.82. These points are:

1305 19
1619 32
1817 39

One point (1697, 41) stands all alone. The remaining two points seem to fall on another parallel. They are really close together on the graph as you can see from the x and y values given below.

1565 23
1604 25

What does all this mean? As I interpret it, we are merely seeing what common sense tells us. Are these "events" truly isolated? I guess not. The Items that you receive and the items that are returned, month after month, have something in common. Your company is probably manufacturing the same widget (please correct me, if I am wrong). Some "defects" occur and the customer is unhappy and returns some of them. So, the "events" are kind of related. This is why we see the law y = hx + c. What this tells us is that as the number of items received (x) increases, the number of items returned (y) also increases. However, the items returned must be less than items received. So the constant h < 1 and we fix the numerical value by considering the (x, y) pairs as we have just discussed. The nonzero c simply means that when the number of items received is doubled, the number of items returned does not double. In other words, the straight line on our x-y graph does not pass through the origin. Instead of different "m" for each point, we have a different "c" and a single value of h.

Take the straight line y = 0.041x - 4.57. The ratio y/x = 0.041 - (4.57/x). As x increases, the ratio y/x, or the percent returned will increase because of the correction term - 4.57/x from the nonzero c. Although the percent returned is increasing the slope h is fixed, which means that the number returned is increasing at a fixed rate (we call it the derivative) as the number received increases. When c is more negative, you have "good" months and the number of items returned decreases. When c increases, you have "bad" months and the number of returns increases. Instead, of calling all this just the intercept c, I created a real fancy term called the "work function". This is discussed in more detail in my August 2004 article, Is there a law relating Views and Replies?

Is there a law relating Items Received and Items Returned? I think so. Let me know what you think. Once, we begin to get a handle on this "work function", profits should start soaring! You can start focussing on why the work function keeps fluctuating from one month to the other. Might sound like old wine in a new bottle, but we all fall for NEW BOTTLES and Labels, don't we? But, seriously, I think, there may just be a lot more to all this. I see y = hx + c in too many places, often in the most unexpected places. If you are still reading this ... let me know.

Charmed :) :thanx:

P. S. Einstein was a very smart guy, RIGHT. It was he who first thought of thing called the work function. This is just a nice explanation for the intercept c in the equation K = hf - W which describes photoelectricity. Now, we say y = hx + c = hx - W also implies a work function! I better take a few days off now.Whew! I got all winded just reading Charmed's reply.
What a frenetic pace. Back in the 60's when I was still in college, there was a department devoted to Demographics where EVERBODY was as devoted to statistics as Charmed appears to be.

Those folks were really a hoot at about 1:00 am in the University Tap. While some of us played Cardinal Puff, the statisticians were charting the number of bathroom breaks we took.

Either Steve will find a kindred spirit or he'll end up running model trains full time.:lmao:

Sorry All, but back on the topic of the original post (I'm still looking to improve my top level, mangement review list of metrics), and I'm considering removing Audit Nonconformances from the list.

Don't get me wrong, I would still like to review internals, externals, and observations during management review, but I'm thinking of just using them as a supplement to the review of the other metrics, and not have a separate metric for # of nonconformances.
What would constitute improvement in this metric? If the # of nonconformances decreases month-to-month, does that really mean the system is improving?

Andy, you are absolutely correct! Do not present management with raw data such as # of NCR's. Rather, in my opinion, the NCR's need to be analysed for the development of any trends, then find root cause. It has been my experience that you can usually find a couple of "whoppers" to bring to managements attention. Management can then review to see what resources are needed to get a handle on them or determine if the C/A that has already been taken is sufficient. The NCR's then become backup material.

For example if the analysis of the NCR's on a particlar operation show a trend towards tapered bore sizes you can then perform root cause to determine operator, machine, tooling, etc. Perhaps it a problem with worn out bearings in a lathe in which case you have to ask why your PM program didn't not prevent this.

Also, a VP once told me never to present management with negative information. Don't tell them that the tapered bore sizes are rejected at a 5% rate, but rather tell them that bores sizes are 95% acceptable! That will then lead you into the discussion of what accounts for the 5% and how do we improve to 99.9%, etc.

Anyway that would be my approach.

Does anyoen have a template perhaps in excel that they like to report the metrics? I find mine cumbersome. One maybe that graphs each thing on a page, seperates it and puts it together,