Thinking of Measurement as a System

Marc

Hunkered Down for the Duration
Staff member
Admin
#1
I ran across this in the ISL listserve. I am posting it because IMHO it shows a good thought train in a calibration system. QS9000 auditors are more and more asking for details of how a company complies with MSA. Some oldies are being discredited (you do *NOT* have to do R&R on every Control Plan line item) and many new concepts are being asked about (addressing linearity, bias, stability, etc.). They are looking to see logic and understanding throughout the process from choosing M&TE during the design stage (automotive=APQP) through and in production.

This has to do with 'test' equipment, but the concepts within are good thought provokers. Oh well, FYI - I think it is a good read:

-------snippo-------

>From: "SMITH, RON D. (JSC-EM)" jsc.nasa.gov
Subject: Q: Long Term Testing/Smith

> >> I am having to prepare a process for handling the calibration of IMTE that
>> is a part of a long term test...over 5 years. The devices are,
>> specifically, thermocouples, and gas or liquid flow monitoring devices in >> ambient conditions (but the process should apply to any type IMTE). The
>> tests have been going on for 5 years with a few more to go. The testing
>> cannot be torn down for the calibration of the test IMTE since a teardown
>> will effect the integrity of the test.
>> >> Our ISO effort is requiring us to attempt to 1.) cover pre-ISO days and
>> 2.) have a new process to go by for new testing conditions.
>> >> Simply put, the basic process we are looking at starts with:(a.) prior to
>> the start of the test, research IMTE that is expected or advertised to be
>> reliable over the period of the test (b.) Estimate the uncertainty for the
>> test period (If the IMTE spec is /- 1inty for a five year test) (c.)perform at least three pre-test
>> calibrations to evaluate the projected IMTE reliability (this is not the
>> case in one of our long term tests where there is only initial calibration >> data), (d.) at scheduled intervals during the test period, the test IMTE
>> measurement data would be analyzed to establish variability and detection
>> of trends - to be additional uncertainty components (e.) and finally a
>> post-test calibration for the followup evaluation of all data during the
>> testing phase.
>> >> As there is no way to remove these instruments once testing has started
>> and no way to insert a check standard to verify instrument performance,
>> can the data be considered valid from a calibration standpoint while the
>> test is ongoing? Can the instruments be considered as in calibration
>> during the test?
>> >> Comments with regard to the validity of this approach and any suggestions
>> would be greatly appreciated.
>Ron Smith
>Johnson Space Center
>Measurement Standards and Calibration Laboratory
>Houston, Texas

From: nicolet.com (Doug Pfrang)
Subject: Re: Q: Long Term Testing/Smith/Pfrang

Ron,

The fact that there is no way to remove the measurement equipment once testing has started is not especially relevant to your problem. Calibration is always something that is performed at a single moment in time and then is ASSUMED to be valid until the next moment in time when another calibration is performed. ANY piece of IMTE can go out of calibration in the interim. Therefore, it doesn't really matter that there is no way to remove the instruments once testing has started; regardless of whether you can remove it or not (i.e., to check the calibration), the real issue is how inherently STABLE your IMTE is over whatever interim you're using between calibrations.

The answer to this question depends entirely on your SELECTION of IMTE. If you know that you must go X years between calibrations, then you have to SELECT equipment that is going to be stable over that period. Once the equipment is selected and installed, there's not much you can do beyond what you are already doing -- namely, monitoring the output to see if there are any unusual changes in the data. If there are unusual changes in the data, then the question is: are the changes real (i.e., have you detected a real change in what you're testing), or are they due to a malfunction in your measurement equipment.

To answer this question, you presumably, at some point, can take the experiment apart and recalibrate your measurement equipment so see if it is still in calibration. If it is not in calibration, then you generally assume that it drifted out of calibration sometime during your experiment, and your data becomes suspect. You might be able to salvage your data if you can determine when the drift occurred and by how much. You can then go back and correct the data post hoc, compensating for the errors that were introduced by your test equipment.

But even if your measurement equipment is still in calibration at the end of your experiment, and you assume that it stayed in calibration throughout the experiment, you are still making a leap of faith, because you are calibrating the equipment only at isolated moments in time and then ASSUMING the equipment did not change significantly in the interim. This is not necessarily a valid assumption. Therefore, you should still do a statistical evaluation of your data (both during and after the experiment), and of your test equipment (after the experiment), to check for variability and trends, because you can never be absolutely positive that the data you collected are free from malfunctions in your measurement equipment. All you can do is increase the LIKELIHOOD that the data are free from equipment error.

Bottom line: Don't worry that you can't recalibrate the equipment without disrupting the experiment, just continue the experiment and continue examining the data during the experiment. Then, at the end of the experiment, revalidate the test equipment, analyze the data thoroughly to look for ALL possible sources of experimental error (including those caused by faulty test equipment), and draw the best conclusions you can with what you have.
 
Elsmar Forum Sponsor

Marc

Hunkered Down for the Duration
Staff member
Admin
#2
Any current (2006) comments on Thinking of Measurement as a System?
 
Thread starter Similar threads Forum Replies Date
M Informational Critical Thinking and the Process of Evidence-Based Practice Medical Device and FDA Regulations and Standards News 0
E Basic Risk based thinking questions Risk Management Principles and Generic Guidelines 5
Jen Kirley Risk Based Thinking and acts of God/Mother Nature Business Continuity & Resiliency Planning (BCRP) 1
Q Risk Based Thinking - Is a Documented Procedure required? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 15
Wes Bucey What were they thinking? Coffee Break and Water Cooler Discussions 2
R How to Train Staff for Critical Thinking Misc. Quality Assurance and Business Systems Related Topics 3
Chennaiite But who said we are new to Risk based thinking Imported Legacy Blogs 1
L Where next? Thinking outside the QMS box Career and Occupation Discussions 9
Sidney Vianna Are the TC 176 Documents on Risk Based Thinking useful to you? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
A Informational Risk Management (and Risk Based Thinking) in ISO 9001:2015 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 54
R New Draft Guidance Document for FDA's current thinking on De Novo submission Other US Medical Device Regulations 1
M ISO 13485 and Class 1 Medical Device - FDA thinking ISO 13485:2016 - Medical Device Quality Management Systems 14
F Re-Defining Quality Contest - Thinking Outside of the QC Glossary Misc. Quality Assurance and Business Systems Related Topics 7
Wes Bucey Still Thinking about a New Job for New Year? Career and Occupation Discussions 2
J John Seddon's book "Systems thinking in the public sector" Lean in Manufacturing and Service Industries 8
U Logical or Lateral Thinking Coffee Break and Water Cooler Discussions 0
bobdoering Systems Thinking Exercise Imported Legacy Blogs 0
D Time is Money - Lean Thinking - Stories of Lean Improvements Lean in Manufacturing and Service Industries 13
Colin Lateral Thinking Funny Stuff - Jokes and Humour 0
S How can I die cut foam - I'm thinking of a simple hand machine? Manufacturing and Related Processes 15
C Tri-annual in 2008, auditor changing and thinking of changing CB - advice needed. Registrars and Notified Bodies 10
R Production Supervisors..What are they thinking! Philosophy, Gurus, Innovation and Evolution 22
Sidney Vianna Crowdsourcing - Fast, Collaborative thinking After Work and Weekend Discussion Topics 2
M Definition WSBH and WAH - Lean Thinking Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 4
D "Circular Thinking vs Linear Thinking" - Your Thoughts? The Reading Room 13
I Lean thinking, Australia & New Zealand? Lean in Manufacturing and Service Industries 0
Marc Thinking of Vista OS from Microsoft? Will it play HD-DVDs or Blu-ray? After Work and Weekend Discussion Topics 2
Wes Bucey Thinking about a New Job for New Year? Career and Occupation Discussions 116
L Strategy Plan for Systems Thinking Career and Occupation Discussions 6
L Implementing System Thinking Approach Throughout an Organization Misc. Quality Assurance and Business Systems Related Topics 11
M Stories relating to Thinking Outside the Box Coffee Break and Water Cooler Discussions 15
Wes Bucey Really thinking outside the box! World News 5
I Definition Standard Deviation - Working vs. Thinking Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 4
K Do performance appraisals and systems thinking sit comfortably together? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 39
Steve Prevette Leading with Leading Indicators, Systems Thinking, and a "Dashboard" The Reading Room 3
J What Standards are required for Food Packaging? Thinking along hygiene, HACCP etc. Food Safety - ISO 22000, HACCP (21 CFR 120) 6
Marc Museums of the World - Not What You're Thinking Coffee Break and Water Cooler Discussions 1
Wes Bucey Queen Mary 2 - What were they thinking? Fire safety rule breaches Preventive Action and Continuous Improvement 16
D Is the market still good for consultants? Thinking about becoming a consultant Consultants and Consulting 16
S Thinking of those who help. "Who packed your parachute" Coffee Break and Water Cooler Discussions 3
WALLACE Quality and Systems thinking impacting Human Resources (HR) Misc. Quality Assurance and Business Systems Related Topics 13
T Introduce Statistical Thinking before introducing the equations Training - Internal, External, Online and Distance Learning 17
Marc Thinking of buying a Dell computer? After Work and Weekend Discussion Topics 22
Marc Thinking Privacy and Security? Microsoft's Passport Program After Work and Weekend Discussion Topics 0
Marc Thinking of MickeySoft's XP? After Work and Weekend Discussion Topics 0
Marc Registrar Comparisons - We're thinking of switching registrars Registrars and Notified Bodies 6
N Drawing tolerance vs. Measurement device General Measurement Device and Calibration Topics 4
Ron Rompen MSA on automated measurement system - Multiple Step Vision System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
P Thermo-Hygrometer - Measurement of 10DegC @10%RH ISO 17025 related Discussions 4
hogheavenfarm GDT Flatness measurement question Inspection, Prints (Drawings), Testing, Sampling and Related Topics 10

Similar threads

Top Bottom