Annual Customer Satisfaction Survey - Weighted (valence) survey as a change?

Greetings and Salutations,

In a few weeks my company will be conducting their annual customer satisfaction survey. In years past we have employed the use of a rather plain/sterile survey consisting of a few broad questions which has produced vague/crappo data. It's time for change...

I've developed a weighted (valence) survey consisting of the following value sets: "How important is Blah Blah to your company" and "How well are we doing Blah Blah".

I'm curious to see how many other people are using weighted surveys and are willing to share formats and experiences.

Mike
Quality Jedi


SALES......QUALITY

Personally, I've always detested surveys designed to be "easy" for the respondent. I think the weighted survey techniques like
The Likert scale

Principle

The Likert technique presents a set of attitude statements. Subjects are asked to express agreement or disagreement of a five-point scale. Each degree of agreement is given a numerical value from one to five. Thus a total numerical value can be calculated from all the responses.

Presentation

You will almost certainly need to measure attitudes during your practical communication work. The following brief comments are intended as tips, but you should ensure that you also look at the sections listed at the end of this section, as other methods may be more helpful and some of the other sections discuss possible pitfalls you may not have thought of.

The questionnaire can be produced very rapidly on a spreadsheet package:


is one of the most heinous devices created to separate the respondent from any sort of intelligent thought about the subject of the survey. There are always subtle and not so subtle forces working to drive the respondent to "middle of the road" answers, including the sheer mechanics of selecting "3" on a 5 point scale.


Survey folk think that (by being able to manipulate large numbers of responses) their results are VALID.


I prefer one-on-one interviews and open ended questions which require the respondent to think about his answer. Let the survey be tough for both surveyor and surveyed. The desire for "metrics" for reporting often obscures the real impressions the people taking the surveys have about the organization because it tends to force answers. Worse, it gives no clues to the surveyors about details of what to change, if anything.


Wouldn't you rather know from a customer:
"What do you like BEST about our delivery?"
and
"What do you LIKE LEAST about our delivery?"
rather than
"Rate our delivery on a scale from 1 to 5, 5 being best."

QualityJedi said:

I'm curious to see how many other people are using weighted surveys and are willing to share formats and experiences.

Click to expand...

At Fluor, we do give a safety perception survey to employees as part of their annual refresher training. It is sixteen questions, and uses Strongly Disagree, Disagree, Neutral, Agree, and Strongly Agree as responses. I use the statistical methodology at http://www.hanford.gov/safety/vpp/survey.htm to analyze it, and this has worked well for the past 8 years.

Attached is a survey helper file that a team in Dept Of Energy (including myself) developed.

Mike,
Wes (as usual) has some good points. There are differences of opinion as to whether surveys is the right tool to gage customer satisfaction- as you'll find out if you search the forums.

However, if you are set on the survey route, here's one book about conducting and analyzing satisfaction surveys that I found interesting : Handbook of Customer Satisfaction Measurement - Nigel Hill.(ISBN: 0566077663)

Let me elaborate about surveys. I am not against Likert-type surveys entirely, only when the questions "force" folks to hit the middle mark through subtle or unsubtle wording.

I can see a Likert scale working for questions like this series on deliveries:

1. Deliveries ahead of schedule cause a problem in our operation.
   [Likert scale - always to never]
2. We require advance notice of shipment.
   [Likert scale - always to never]
3. Carton labeling must be in form prescribed in purchase order.
   [Likert scale - always to never]
4. Shipping must always be by carrier we specify.
   [Likert scale - always to never]
5. We are satisfied with your shipping to us.
   [Likert scale - always to never]
6. Comment - use this space to elaborate on your thoughts about shipping, especially on anything not covered in the survey questions.
   [Include ample blank space for an answer.]

The series gives a clue to the respondent about shipping factors which he should consider in answering #5 - obviously, the series should take some thought in preparation for each aspect of your relationship with the customer (product pricing, service response, product quality, etc.) It is important to give respondent an opportunity to comment on the question groups as well as the survey as a whole. It gives the respondent a sense of VALUE for his response, implying that his views as an individual will be considered. Consequently, he takes more care in answering the survey.

These are the kinds of things a professional psychometrician considers in crafting a survey or questionnaire or test. Most often, less thought is given to the phrasing or construction of a questionnaire than is given to what beverage to have with lunch. This results in a terrible waste of forests being clearcut to make the paper they are printed on.

re: Customer Survey Tools

We currently use an annual customer satisfaction survey with the 1 to 5 scale which asks our customers to compare us to our competition. We aim it to our most important customers. I'll try to post it later after removing company ID. My feelings regarding its accuracy are rather neutral. (Results indicate that in most instances our customers perceive us as better than our competition.) We're a relatively new company in the American market - we've been producing at this plant for years now and are gaining market share versus our competitors, so we're doing a number of things right which would tend to back up the survey's ratings.

Management also chose to use a similar instrument with a 1 to 5 scale to gauge overall employee satisfaction and outsourced it to a consulting firm to assure anonymity of response. Overall employee response averaged slightly less than 3. Feedback presented by the consultant to management was that was typical for an initial survey, and that it indicated management was doing a good job. Management chose to believe the consultant.

Helllo folks!

I don't use surveys with a scale from 1 to 5. The majority of answers will be in the midlle ("3"). I prefer scales form 1 to 4.
Another problem with surveys tahta comes from Standard it's the possibility you evaluate the data on your survey in terms of time (year after year, if your survey it's annual). I think it's for that, most of us prefer surveys with scales, because it's easier to make comparisions.

(Sorry my bad english, I hope you understand me! )

warrior said:

Sorry my bad english, I hope you understand me!

Click to expand...

Your English is fine. I wish I could speak Portuguese as well.

You might want to have a look at this thread, and Craig Cochran's excellent article on the subject.