From: Philip Stein
Subject: Re: Q: Cal. Lab Audit/Miller/Randall/Andrews/Randall/Stein
I have added many of my own comments, mostly contrarian, interspersed in the text.
Phil Stein, the Contrarian Metrologist
>>> Richard wrote (in part)
>>> "The most common N/C that I issue involving calibration has to do with
>>> qualification of calibration subcontractors (ISO 9001/2, element4.6.2).
>>> Unlike suppliers providing products (that can be verified at receiving
>>> inspection) sending out a questionnaire (to be completed by the supplier)
>>> alone won't do it. There should be objective documented evidence that the
>>> calibration subcontractor complies with all of ISO 9002, element 4.11.
A good idea, but NOT required
>>> Acceptable methods include:
>>> - A site audit performed by your organization
>>> - Copies of site audit(s) performed by other companies
>>> - ISO 9002 registration of the calibration subcontractor
>>> - ISO Guide 25 Accreditation of the calibration lab (e.g., A2LA, NVLAP)"
I think that it would be good to have one or more of these pieces of evidence. The most important issue is that none of these are required, and many of them, especially ISO Guide 25 accreditation, may be overkill. It can't hurt, but might be costly. The true weak link comes from nonconformance with 4.11.1, where it states that the measurement capability required must be known and that the measurement uncertainty must be compatible with it. If you know your required uncertainty, it's real easy to decide how hard to scrutinize your cal lab. Clarity begins at home
>>Andrews inquired:
>> Richard - where exactly in 4.11 or 4.6 does the Standard REQUIRE that I
>> exercise the controls that you mention above??
>
>ISO 9001/2, element 4.6.2 states that:
>
>The supplier shall:
>
>a) evaluate and select subcontractors on the basis of their ability to meet
>subcontract requirements including the quality system and any specific
>quality-assurance requirements
>
>Assuming we agree that in the case of a calibration subcontractor the supplier
>"should" be invoking ISO 9001/2 element 4.11 or equivalent (as per element
>4.6.3),
Not a good assumption
>then it follows that the calibration subcontractor must be qualified
>as being able to meet the detailed requirements contained in element 4.11.
Not a requirement. Some portions of 4.11 flow directly from the requirements of the cal customer and must be followed, such as traceability. Others, such as safeguarding of adjustments or control of environment are situation dependent.
>This is a significant differentiating factor from all other subcontractors
>because the supplier is dependent upon the calibration subcontractor's
>compliance with an element of the standard in order to fully comply with the
>standard themselves (the supplier)!
Maybe, maybe not. Often, other requirements such as ingredient or component traceability also flow from the prime contractor to the sub. Again, it's situational.
>> My understanding is that the type and extent of controls exercised is up
>> to the individual company, taking into account various conditions and
>> circumstances. "sending out a questionnaire (to be completed by the
>> supplier)" may be perfectly valid in certain situations.
>
>You are correct in that the vast majority of subcontractors can be qualified
>(at least initially) through the use of a questionnaire (to be completed by
>the supplier). This is acceptable for subcontractors providing a product
>whose quality can be verified at receiving inspection. However, one cannot
>verify that the detailed requirements of element 4.11 were met by performing a
>receiving inspection on calibrated equipment.
This is not strictly true. If a measurement assurance control chart is maintained on the equipment, it should show evidence of correct calibration, etc., but this is a quibble on my part. If you're doing measurement assurance, you certainly are sophisticated enough to have good controls on your cal suppliers.
> For example, receiving
>inspection cannot verify
>that the instrument was calibrated/checked against certified equipment
>having a known valid relationship to international or nationally recognized
>standards.
>(the calibration report may state that the calibration was performed using
>NIST traceable instruments - but how do we really know?
And you can't check every little thing your suppliers claim under other 900x paragraphs either. Sure there are exceptions and improprieties, but why say 4.11 is special?
I've found
>instruments being used in calibration labs that are traceable to NIST through
>calibrations performed "decades" ago. Is that acceptable?
Depends. doesn't sound like a good idea but it's not contrary to the letter
of the definition of traceability, and at the cost of traceable calibrations these days it's not a bad idea to find other ways of maintaining your standards. I'm not advocating being sloppy or overly frugal, but rather keeping costs down by good metrology (as opposed to blindly and religiously following a tight cal schedule).
I've also found
>what is called a "calibration loop" in several calibration labs. This is
>where an instrument is used to calibrate another like instrument that is then
>used to calibrate the first instrument - the instruments are used tocalibrate
>each other. True traceability is lost as the error of each instrument
>accumulates in a never-ending cycle or loop.
If that's all you EVER do, it's risky, but if you check one or both of the instruments in a loop against other things - even the stream of unknowns coming through your regular cal load - you have some considerable insurance against systematic drift.
Even though I've never found an
>instrument being used in a calibration lab that lacked ALL traceability to
>NIST, I have heard stories of instances where this was found by others.)
>{Ref. element 4.11.2b}
Oh I found one, and it was the grand master for the most important measurement in the company. Its manufacturer refused to provide traceability for their calibrations. We replaced it with $75.00 worth of traceable standards and a bit of inspired measurement engineering. It's now in the front lobby in a glass case because it's so pretty.
>- that an acceptable calibration method was followed.
>(The calibration records may list the calibration procedure that was used, but
>is that calibration procedure acceptable to you? Does that procedure check
>ALL of the functions of the instrument - particularly the functions that you
>use? Not every calibration lab uses the manufacturer's calibration procedure.
>Many internally generated calibration procedures are greatly abbreviated from
>the manufacturer's procedure - taking short cuts to speed the calibration
>process. I like to compare the "Equipment Required" section of the
>calibration procedure with the actual listing of equipment used in performing
>the calibration. There are almost always substitutions made because the
>calibration lab does not own the exact equipment listed. At that point I like
>to have the calibration lab prove to me that the equipment used really was
>equivalent. Sometimes it is not.)
>{Ref. element 4.11.2, c}
And sometimes you don't need a full calibration and are silly to pay for one. A metal stem thermometer used for monitoring a tank at 55 degrees was calibrated every 10 degrees from -40 to 250 at considerble cost. It's now only calibrated near the temperature required and placarded that it isn't calibrated outside of that range. Big savings by not slavishly following manufacturers recommendations. Better to use brains.
>- that the calibration subcontractor uses the same acceptance criteria that
>you use?
>(The calibration records may state that the instrument was found to be "in
>calibration" with the manufacturer's specifications, then again it may simply
>state that that the instrument was found to be "in calibration". When nothing
>else is stated then it is assumed that the instrument was found to meet the
>tolerances specified in the "calibration method" used. Some calibration labs
>change the tolerances of instruments in their internally generated
>"abbreviated" calibration procedures so they don't have to make adjustments.
>This may sound like business 101, but the more instruments that pass through a
>calibration lab - the more money they make. I've seen this situation in
>auditing calibration labs more than once.)
>{Ref. element 4.11.2, c}
Practices such as these violate both the letter and the spirit of traceability. Every comparison in the chain must be supplied with a value that includes stated uncertainties. Just saying "in cal" can't be good enough.
>- that the calibration lab will notify the supplier if "their" instrumentation
>(measurement standards) are found to be out of tolerance, causing the
>calibration to be suspect.
>(Virtually all calibration quality system standards {e.g., MIL-STD 45662A,
>ANSI/NCSL Z540-1, ISO 10012-1} require the client to be notified if ANY
>instrumentation used in performing the calibration are later found to be out
>of tolerance. This means that every instrument used in a calibration lab must
>have a listing of every device it was used to calibrate since the last time it
>was calibrated. If ISO 9001/2, element 4.11 is invoked on a calibration lab,
>that means that this requirement must be met in order to satisfy 4.11.2f. How
>do we know that this requirement is being met without a site audit performed
>by someone? As you might well imagine, this requirement is costly for a
>calibration lab to comply with - some do not.)
>{Ref. element 4.11.2, f}
Well this is correct except that if the cal lab is keeping control charts, all it needs to know is that its measurement process was in control when your equipment was calibrated. Then all they have to do is keep the charts - not a complete record of which instruments were used (although many do this too). The charts are a very quick indication of when things go wrong, so only a few recent customers need to be notified when there's a problem.
>- that the calibration was performed under suitable environmental conditions.
>(This requirement is most important for dimensional devices. The majority of
>calibration labs report the environmental conditions {temp/RH} on their
>calibration certificate or report. But is this information accurate? I was
>auditing a calibration lab in New York several years ago and noticed that the
>temperature was ALWAYS at 70 degrees F and the RH was ALWAYS at 35%. Upon
>touring their lab I saw no special considerations made to ensure that the
>environmental conditions stayed so constant. I asked a calibration technician
>how the environmental conditions stayed so stable and his response was that he
>didn't know, he had been trained to ALWAYS record the environmental conditions
>as being 70 degrees F and 35% RH. I asked the calibration lab manager why the
>calibration technicians had been instructed to always record the environmental
>conditions as being 70 degrees F and 35% RH. He responded that they had an
>environmental chart recorder in a building across the street and it almost
>always read 70 degrees F and 35% RH. <Yes - this really did happen>)
>{Ref. element 4.11.2, g}
This is the biggest joke and biggest source of confusion in all of 4.11. You don't have to maintain your environment, or even record it. All you have to do is to calculate the effect that different environmental conditions will have on your calibration and if those effects are negligible, neglect them. If you DO require environmental control, the instruments that measure the environment, such as your thermometer, had better be calibrated as well. This is a very frequent omission.
>After an auditor sees enough of the nonsense that "can" go on in a calibration
>lab, he starts to want "real" verification that the calibration subcontractor
>being used has been legitimately qualified.
This is overreaching by the auditor. It's the supplier that should be vigilant, and the auditor should validate the supplier's vigilance
>On the topic of whether the ISO 9001/2 allows a questionnaire to be used in
>qualifying a calibration subcontractor, ISO 9001/2, element 4.6.2b states (in
>part) that the type and extent of control exercised by the supplier over
>subcontractors shall be dependent upon the type of product (or service) and
>the impact of subcontracted product (or service) on the quality of the final
>product.
>
>Calibration has a significant impact on the final product
True...
>(otherwise you shouldn't be going to the expense of calibrating those instruments).
False... Most places calibrate because 9000 made them do it.
>Therefore the supplier should make a reasonable effort to ensure that the
>calibration subcontractor meets the requirements contained in ISO 9001/2
>element 4.11 or equivalent. Based on the discussion above, a questionnaire
>completed by the calibration lab is not a sufficient means of qualifying that
>calibration lab. The same can also be argued of a testing lab.
>
>Upon finding a company using a questionnaire completed by the calibration lab
>as a means for qualifying that subcontractor, I would issue a noncompliance
>with ISO 9001/2, element 4.6.2 (since I am concerned with requirements of both
>elements 4.6.2a and 4.6.2b being met). This is clearly a judgment call with
>respect to the standard - and is therefore arguable. But hopefully this
>rather lengthy e-mail explains the reasoning for why calibration
>subcontractors should be viewed differently than most other subcontractors
>when undergoing the qualification process.
And in the end, although there are many valid points here, I would disagree that calibration is anything 'special'. I think items such as segregation of nonconforming material are at least as 'special', and pretty soon we reach the conclusion that knowing your suppliers' processes more deeply than a questionnaire can do is a good idea all around.
Philip Stein
Consultants in the Physical Sciences, Measurements, and Quality
National Director, American Society for Quality
Past Chair, ASQ Measurement Quality Division
Also - Check out:
http://www.metrology.org
http://www.hamvention.org