Deming's SoPK (System of Profound Knowledge) Discussion

[Mike S.]

Mike et al,

Once again pulled from the DEN. A paper by Dr. Myron Tribus that addresses some of the "unmeasurables" that depict the health of a system (and why they should be treated as observable, but "unmeasurable").

Scanned it quickly. Interesting article.

Maybe I'm a real dummy? Maybe it is semantics? I dunno, but the writer says do not use "scoring" or "a numerical rating" for suppliers, then suggests a 4 level competence scheme "to assess at what level of competence the managers are operating". So, this is not scoring simply because a number is not used? So what if I assign a score of 1-4 for the 4 levels of competence methodology he advocates -- that ruins its validity?

At some point the rubber must meet the road -- somehow you must take the data you've generated and use it to decide if company A or B or C will be selected as best. If you do that without using a number that's okay with me -- I still see it as measuring the company. Call it "assessing" if that makes you happy, but to me that is still synonomous with measuring.

Please, what am I missing?

 

[Jen Kirley]

Measuring the unmeasurable

I read the attached paper by Dr. Myron Tribus, and thank you for providing that. I will print it out.

It leads me to grapple with the seemingly diametrical issues of "Management by fact (Baldrige element) and "Management by feeling" (a common practice that is sometimes called "Flying by the seat of your pants").

Now I truly do appreciate the antagonism I am implying here, but I feel sure that an attempt to measure progress in these areas is important to know that progress is in fact being made, or else it's a subjective judgement that could be based on a number of factors.

How then, should we measure such "soft" kinds of progress?

I had decided that administering a questionnaire similar to the Baldrige "Are We Making Progress?" to each employee--sticking with a select group or supervisory people won't do--and graphing changes in response "strengths" might be a good idea. I had gone so far as to develop spreadsheets for entering the responses and graphing the results, broken down by shifts and groups, since answers can get statistically laundered by looking at them organization-wide. We can look at group senses that way, and measure degrees of change from one questionnaire cycle to the next.

Does that seem sensible?

This is important to me because I am looking hard at building practical techniques for small businesses outside of manufacturing to use. These businesses would need straightforward ways to know their progress if we were to succeed in making the case that organized quality efforts are worth doing.

Jennifer

 

[Mike S.]

I was in a meeting and couldn't respond sooner.

Sorry, Mike, I didn't mean to ruffle any feathers. I thought (mistakenly I might add) based on your original questions that you were finding fault with Wes's statements, which I thought were more generalizations that anything else. Only now do I get the sense that you were truly digging for understanding. I guess I was assuming a sarcasm that didn't exist. Anyway, I apologize again for my presumptuousness.

Also, I hope the references from Out of The Crisis helped.

Apology accepted. :truce:

Now, about that reference: You said:

"Deming understood three things clearly: 1) Proper use of data and statistics yields excellent opportunities for improvement, 2) Most companies improperly use data, and 3) The biggest sources for improvement cannot be accurately measured, e.g. the impact of customer loyalty, trust, teamwork, collaboration, culture, etc.

Point 3 comes as close as anyone has yet come to finally giving me an example of something important that "cannot be accurately measured". I am not sure whether I believe that they cannot be accurately measured, but I will admit that the measurement would probably not be as easy as something like yield or market share.

 

[Wes Bucey]

The percentage cited got my attention first. Then, knowing Deming wasn't a dumb guy, I wondered exactly what I might be missing -- what exactly is it that cannot be measured? I admittedly don't spend lots of time thinking philosophically about such stuff, so I thought it would be a pretty easy thing to ask for examples of the things Deming felt couldn't be measured, then I could decide if I agreed or not. I didn't expect it would be a big thing.

I'm not here with an agenda to promote or denigrate Deming or anyone else. There are certainly things he says I agree with 100%, others I agree with to varying degrees, and other things I disagree with. JMO. There are very few absolutes on any subject I will agree with, and the closer to an absolute something supposedly is the more likely I will disagree with it. I will look carefully at anything that is supposedly 97% this or that.

OK. This is a nice, fresh start.
I interpret both Mike and Craig to be saying, "Yeah, the certainty of a number (97%) seemed to be at variance with the whole Deming thing and it took us off track."

I personally doubt that Deming intended to be like Yogi Berra in his infamous quote:
99% of the game is half mental. Yogi Berra

Unintentionally, though, beyond the laughter, Yogi made a point similar to Deming's:
"the mental, or psychological aspect, of the game or business is infused in every part of the game or business."

Because businesses are each unique, we rarely have a Control which gets only Placebos ("sugar pills") versus "the real thing." Without a Control, it's really difficult to determine if the new process really is responsible for the improvement or whether the improvement comes about as a result of a placebo effect.

Deming and many others tried to cut through the fog. Some attributed the bulk of improvements to the increased attention the process got as a result of "measurement." That is, the "human element" was the primary impetus. Others said the important factor was "sustainability" (did the process continue at the new level or was there a "relapse"?) Think of the relapse effect like so many yo-yo dieters, who lose weight, then soon rebound to the prior weight or beyond.

Mike asks in a later post:

I still see it as measuring the company. Call it "assessing" if that makes you happy, but to me that is still synonomous with measuring.

Please, what am I missing?

The concept is not against "measuring." We measure many things and make decisions based on the measurement:

• I need to transport two gallons of water - is the bucket big enough?
• I need 2 million widgets a year - is the supplier capable of turning out that many and still meet my Quality expectation?
• I need a rod 6.235 inches long - is it too short, too long or just right?

The biggest concern in a Demingite is whether the measurement is meaningful (as in the examples above) or nonmeaningful:

• John only "handled" 200 calls today instead of his quota of 210.
  (What happened in each call? - not measured. Was customer satisfied? - not measured. Did customer have to call back two or three times? - not measured.)
• Mary only soldered 100 joints today instead of her quota of 150.
  (Was Mary properly trained? Was the quota too high? Was the current on the soldering iron subject to a line voltage drop? Is the solder the proper quality? - Who knows? - the other items weren't considered.)

Deming's primary gripe (and mine) is that too many folks look to the numbers instead of the deeper root cause the numbers may be masking. Too frequently, the psychology of the bosses is to fire the individual instead of improve the system. Deming strove to make the workplace more efficient by directing how the numbers and measurements were ultimately used. In pursuing efficiency his way, the workplace also became more humane.

What's important to measure? What isn't? The guy who comes up with the one answer that fits every situation won't be able to answer us here - he'll be too busy spending the money he'll make.

 

[David Hartman]

Scanned it quickly. Interesting article.

Maybe I'm a real dummy? Maybe it is semantics? I dunno, but the writer says do not use "scoring" or "a numerical rating" for suppliers, then suggests a 4 level competence scheme "to assess at what level of competence the managers are operating". So, this is not scoring simply because a number is not used? So what if I assign a score of 1-4 for the 4 levels of competence methodology he advocates -- that ruins its validity?

At some point the rubber must meet the road -- somehow you must take the data you've generated and use it to decide if company A or B or C will be selected as best. If you do that without using a number that's okay with me -- I still see it as measuring the company. Call it "assessing" if that makes you happy, but to me that is still synonomous with measuring.

Please, what am I missing?

Mike, The 4 level competence scheme deals with "Training Competence" only (a very small portion of the total evaluation Dr. Tribus describes - Things you See; Things you Hear; Attitudes Displayed; Topics of Discussion in meetings; Expectations of the Workforce; Priorities assigned to Customer's Desires; and the Goals of the Enterprise).

Let's walk through an example of one of these elements.

As I am walking through a supplier's facility, I make note of fact that "some" of the equipment has layers of dust on it with an appearance that it has been idle for quite some time.

Later I am at another competing supplier's, and I note "some" of the equipment with layers of dust: As much dust as the previous suppliers? Should I have measured this? As many dusty machines, or more? Could this be an important factor? Should I retune my "Supplier Questionaire" to address how much dust and/or how many machines are acceptable? And if I do, will my judgement be the same as that of the other 6 people within my organization that are visiting our suppliers?

Or, should I treat each of these incidents separately by questioning: Is the equipment broken (poor maintenance)? Has it been replaced by a newer model, and therefore is just taking up space (mis-management of space/resources)? Is it useable, but due to a slowdown in orders there has not been a call for it for sometime (Why the loss of sales)?

And if these are the questions that are really of import, how do I "score" them (especially comparing one supplier to another and possibly using more than one person doing the scoring)?

Sure I could develop a 1 - 5 (or 1 - 10) scale and provide some form of direction to the "assessors" as they rate the suppliers in these areas, but in reality am I not attempting to take a subjective matter and objectively rate it (in which I will never really eliminate the subjective bias)?

Certain aspects of life (and management) are measurable and objective, but many others are subjective (and are like round pegs in square holes when you attempt to measure them objectively) - you can fool yourself into thinking that you have eliminated the subjective bias, but it will always remain and your resultant "scores" will have a systemic variation because of it.

 

[David Hartman]

I read the attached paper by Dr. Myron Tribus, and thank you for providing that. I will print it out.

It leads me to grapple with the seemingly diametrical issues of "Management by fact (Baldrige element) and "Management by feeling" (a common practice that is sometimes called "Flying by the seat of your pants").

Now I truly do appreciate the antagonism I am implying here, but I feel sure that an attempt to measure progress in these areas is important to know that progress is in fact being made, or else it's a subjective judgement that could be based on a number of factors.

How then, should we measure such "soft" kinds of progress?

I had decided that administering a questionnaire similar to the Baldrige "Are We Making Progress?" to each employee--sticking with a select group or supervisory people won't do--and graphing changes in response "strengths" might be a good idea. I had gone so far as to develop spreadsheets for entering the responses and graphing the results, broken down by shifts and groups, since answers can get statistically laundered by looking at them organization-wide. We can look at group senses that way, and measure degrees of change from one questionnaire cycle to the next.

Does that seem sensible?

This is important to me because I am looking hard at building practical techniques for small businesses outside of manufacturing to use. These businesses would need straightforward ways to know their progress if we were to succeed in making the case that organized quality efforts are worth doing.

Jennifer

Jennifer,

Yes, you can use such a questionnaire, but keep in mind such factors as:

Did the respondant have a bad day this time around -Vs- the previous time (e.g. They may have just had an aurgument with their immediate supervisor, or with a co-worker; or maybe they just woke up on the wrong side of the bed today).

These types of questionnaires can become a soapbox for the gripe of the day, or the opposite a forum for kissing up to the boss.

You will never be able to eliminate systemic variation from the process because of its subjective nature.

 

[Jen Kirley]

Jennifer,

Yes, you can use such a questionnaire, but keep in mind such factors as:

Did the respondant have a bad day this time around -Vs- the previous time (e.g. They may have just had an aurgument with their immediate supervisor, or with a co-worker; or maybe they just woke up on the wrong side of the bed today).

These types of questionnaires can become a soapbox for the gripe of the day, or the opposite a forum for kissing up to the boss.

You will never be able to eliminate systemic variation from the process because of its subjective nature.

I agree with the variable nature of personnel responses. Understanding that this is a potentially significant source of unreliability within industries that are heavily human-process oriented, I still question how to appropriately measure progress.

So, such things are subjective but if the entire group--say, a line of 12 phone center operators--provides answers in a significant shift from the responses of the other groups, is it a measurement with value?

 

[David Hartman]

I agree with the variable nature of personnel responses. Understanding that this is a potentially significant source of unreliability within industries that are heavily human-process oriented, I still question how to appropriately measure progress.

So, such things are subjective but if the entire group--say, a line of 12 phone center operators--provides answers in a significant shift from the responses of the other groups, is it a measurement with value?

I believe that given the scenario you presented the questionnaire can be of value (as I previously stated, you can use such a questionnaire), my only caution was to point out that outliers may be personality driven more than system driven. Using the questionnaire to view large population changes could provide you with indicators of controllable variations.

 

[Wes Bucey]

I agree with the variable nature of personnel responses. Understanding that this is a potentially significant source of unreliability within industries that are heavily human-process oriented, I still question how to appropriately measure progress.

So, such things are subjective but if the entire group--say, a line of 12 phone center operators--provides answers in a significant shift from the responses of the other groups, is it a measurement with value?

IMO, it has value, but value for what?
Many egregious abuses of Management by Objective are found throughout call centers.
Typically, folks in a customer service call center are constrained in the extent they may go in understanding and responding to or resolving the customer's concern.

Somehow, the MBO advocates think the answer lies in a tighter script or more "bells and whistles" in the Customer Relationship Software (CRM) they use.

They have True-False, Multiple choice, or value scale questionnaires for the customers and the call center operators to complete, but they pay more attention to the numbers from the preprogrammed questions than to the marginalia or answers scribbled in the space for "your comments."

When an operation is run in an atmosphere of fear, no employee truly believes his answers will be "anonymous." Deming and others have recognized this for a long time.

MBO advocates think they can avoid that stigma by having outside firms conduct the survey - what do the employees really think?

Many questionnaires are poorly constructed, with an aura of "we know we're good, we just need you to tell us how good."

The bottom line may be - what does the entity issuing the questionnaire intend to do with the results? If it is to crow, "9 out of 10 employees love working here!" (as opposed to what? - living under a bridge?) I'd be interested in your reaction.

If it is to develop better training, better working conditions, better reponse to customers, then I might be willing to listen to the proposal.

I guess I'd rather see an atmosphere where essay-type questions replaced multiple choice and number (and soul) crunching.

 

[Mike S.]

I personally doubt that Deming intended to be like Yogi Berra in his infamous quote:
99% of the game is half mental. Yogi Berra

The biggest concern in a Demingite is whether the measurement is meaningful (as in the examples above) or nonmeaningful:

• John only "handled" 200 calls today instead of his quota of 210.
  (What happened in each call? - not measured. Was customer satisfied? - not measured. Did customer have to call back two or three times? - not measured.)
• Mary only soldered 100 joints today instead of her quota of 150.
  (Was Mary properly trained? Was the quota too high? Was the current on the soldering iron subject to a line voltage drop? Is the solder the proper quality? - Who knows? - the other items weren't considered.)

Deming's primary gripe (and mine) is that too many folks look to the numbers instead of the deeper root cause the numbers may be masking. Too frequently, the psychology of the bosses is to fire the individual instead of improve the system. Deming strove to make the workplace more efficient by directing how the numbers and measurements were ultimately used. In pursuing efficiency his way, the workplace also became more humane.

Wes,

First, on the Yogi quote. Actually, Yogi said, "Ninety percent of the game is half mental." Not meaning to nit-pick, but I had heard it often enough to know 99% was wrong and I thought maybe you set me up for that one, so I didn't want to disappoint. :truce:

The rest of your post makes some sense to me -- I agree with most of it. This makes more sense to me than saying you cannot measure most of the stuff that matters. IMO you can, but you can also get caught in the trap of measuring the wrong things, or responding inappropriately.