The Elsmar Cove Business Standards Discussion Forums More Free Files Forum Discussion Thread Post Attachments Listing Elsmar Cove Discussion Forums Main Page
Welcome to what was The Original Cayman Cove Forums!
This thread is carried over and continued in the Current Elsmar Cove Forums

Search the Elsmar Cove!

Wooden Line
This is a "Frozen" Legacy Forum.
Most links on this page do NOT work.
Discussions since 2001 are HERE

Owl Line
The New Elsmar Cove Forums   The New Elsmar Cove Forums
  Measurement, Test and Calibration
  Measurement Uncertainty in Subjective Tests

Post New Topic  Post A Reply
profile | register | preferences | faq | search

UBBFriend: Email This Page to Someone! next newest topic | next oldest topic
Author Topic:   Measurement Uncertainty in Subjective Tests
Marc Smith
Cheech Wizard

Posts: 4119
From:West Chester, OH, USA

posted 03 July 2000 09:26 PM     Click Here to See the Profile for Marc Smith   Click Here to Email Marc Smith     Edit/Delete Message   Reply w/Quote
Organization: Disorganised Inc.
Newsgroups: misc.industry.quality
Subject: Measurement uncertainty
Date: Sat, 15 Apr 2000 17:06:30 GMT

We regularly measure the effects of certain treatments on textile materials and fabrics.

These effects are evaluated by a panel and the result is a discrete value on a scale, say 0 to 5 (or in another range).

Our auditor requires uncertainty of measurement for all of our tests. I have doubts whether the concept makes sense in this context, but apparantly that is not a discussion that our auditor is prepared to enter in. He wants UM, point.

Any hints or ideas ?




From: Randy Herndon
Newsgroups: misc.industry.quality
Subject: Re: Measurement uncertainty
Organization: WebUseNet Corp
Date: Sun, 16 Apr 2000 08:36:41 -0400

If you are not performing a quantitative measurement then the
measurement uncertainty will be null and void. It sounds as if what
you are doing is more of a survey to say. Without a measureable unit
i.e. ohms, liters, etc you can not do this. I would go around the
auditor and call someone else at the same agency to get assistance.
Remember, the auditors are folks just like us and may have been
instructed that "EVERYTHING" must have an uncertainty when in fact
some tests are simply for mild confidence.


Newsgroups: misc.industry.quality
Subject: Re: Measurement uncertainty
Date: Sun, 16 Apr 2000 15:41:56 GMT
Organization: - Before you buy.

Greetings Johan,

I think the concept of uncertainty does make sense in this context. Why don't you just go ahead and measure the repeatability etc. of the measurement. You will probably find that the rating scale is more variable than a physical measurement would be.

Do this experiment. Select 6-7 colored samples that cover your range of products. Cut each sample in half so that you have two identical samples of seven products. (The exact number of products and samples is not important.)

Select your panel members and others as testers to perform the experiment. Assign code numbers to the samples so that the testers cannot tell which samples are pairs. Let the testers see only one sample at a time. Don't let them see each other do the test, and don't let them collaborate to achieve agreement. Randomize the order of testing.

Perform the complete test under standardized lighting conditions on three different days at least one week apart. Let one day be early morning, next at mid-day, and last late in the day. (Eyeballs vary.)

Analyze the individual test scores (not averages) with analysis of variance. Determine the size of these sources of variability:

variability between identical samples (within day, product, tester)

variability between days (within product, tester)

variability between testers (within day, product, sample)

Report the means and standard deviations that describe each component of variability. Do not round the results to discrete values.

Perform hypothesis tests for each component of variance. This will show which components are probably real. If you need information on hypotheses testing see the message on this forum by Kelly Speiser, Subject: "Statistical Hypothesis Testing". She has a book on hypothesis testing.

When you finish, you will have better knowledge of your test method. This will lead to improvements.

And you will exceed the expectations of your auditor.

If you want to go the next step and make the accept/reject color decisions statistically, see my website at:

The method shown there appplies to rating scale color judgements as well as physical measurements.

Sincerely, Stan Hilliard


From: (Manus)
Newsgroups: misc.industry.quality
Subject: Re: Measurement uncertainty
Date: 16 Apr 2000 23:31:37 GMT
Organization: AOL

Yes, I agree with everyone that this auditor is wacky in asking measurement uncertainty.

The ISO wording (section 4.11.1 para 2) Inspection, measuring, and test equipment shall be used in a manner which ensures that the measurement uncertainty is known and is consistent with the required measurement capability.

I would also be curious if this were a required inspection you are doing versus something that you just test for process control. If it is a process control item and there is other equipment or such other inspections that take place which would verify the textile as meeting the specified requirements... then calibration is not required.

I believe what the auditor is doing mixing an inspection item concept and a workmanship criteria item together. And since the auditor was unwilling to discuss, their ignorance on the subject was obvious.

An auditor under the scope of 10011 is supposed to have documented objective evidence of what they are discussing and how the standard applies as well as what makes up the nonconformance. The nonconformance also has to be of a benefit to your company (it's in the standard folks).

Obviously they bungled their finding and I would challenge it with their appeals process and make them demonstrate what it is they are trying to enforce.

Slan Leat!

Phil McManus

IP: Logged

John C
Forum Contributor

Posts: 134
From:Cork City, Ireland
Registered: Nov 98

posted 08 July 2000 12:59 PM     Click Here to See the Profile for John C   Click Here to Email John C     Edit/Delete Message   Reply w/Quote
You state that you 'measure' effects. I think this gets you off to a bad start since, to measure implies an objective judgement. Yet yours appears to be totally subjective. So I suggest you drop the word measure and replace it with something like 'assign a subjective value' or whatever.
If the auditor still requires you to apply measurement uncertainty then tell him/her that if you had the equipment to evaluate all the variables associated with turning what are generally considered to be subjective assessments, into scientifically valid objective visual and mental tests, then you would use that equipment directly and not via that roundabout route but, you don't have the equipment and, as far as you know, it does't exist.
Be vary careful when suggesting methods to meet unreasonable expectations because, whether or not they are practicable and effective, the auditor will be using them to claim that he has seen a method of achieving this and will be asking for more of it in the future, convinced that some people actually do it.
rgds, John C

IP: Logged

Brian Dowsett
Forum Contributor

Posts: 35
From:Waterford, Ireland
Registered: Nov 1999

posted 13 July 2000 12:39 PM     Click Here to See the Profile for Brian Dowsett   Click Here to Email Brian Dowsett     Edit/Delete Message   Reply w/Quote
I'd say you're already one step ahead of your auditor. Your organisation must have already identified some measurement uncertainty in order to make the decision to use a panel to do the evaluations. Surely the purpose of measuring the uncertainty is so you can either allow for it or improve the measuring system. You have taken the second approach and gone for a group decision to lessen the subjectivity.
I'd definitely agree with John C from Cork, don't placate auditors unless it's absolutely necessary.
If there's no alternative with your man, then do a simple attribute MSA (see the shiny blue QS9000 book)first with single inspectors then with the group, proving that the group has less variation in the measurement.



IP: Logged

All times are Eastern Standard Time (USA)

next newest topic | next oldest topic

Administrative Options: Close Topic | Archive/Move | Delete Topic
Post New Topic  Post A Reply Hop to:

Contact Us | The Elsmar Cove Home Page

Your Input Into These Forums Is Appreciated! Thanks!

Main Site Search
Y'All Come Back Now, Ya Hear?
Powered by FreeBSD!Made With A Mac!Powered by Apache!