Calibration of Engineering R&D Measurement Equipment or Not

[Pinbike]

I have finally convinced upper management to calibrate all test and measurement equipment. The engineering manager does not want his equipment calibrated due to expense and he says that nothing is used in production. All engineering equipment is R & D only supposedly. He specifically wants to exclude all power supplies (13 of them) because he verifies output with multimeters. I would like to see everything calibrated to avoid problems.

We are in electronic manufacturing so there is alot of meters, scopes, etc. involved in our calibration.

[Rob Nix]

In Reply to Parent Post by Pinbike

I would like to see everything calibrated to avoid problems.

Have you experienced problems due to "non-calibrated" devices is engineering?

I always consider risks vs. benefits when encountering something like this. Is the cost related to subsequent failures greater than the cost of inclusion of the devices into the calibration system? If not, it may not be worth it. You will get better cooperation from them if you can prove the benefits.

Perhaps, as one option, the multimeters (or one master multimeter) could be included without including power supplies. Just some food for thought.

[Jerry Eldred]

I could reasonably argue both sides of this issue. However, I strongly lean toward your side. Matter of fact, I am in an engineering lab at this moment involved (among other things) with calibrating power supplies. Here are some criteria which I use in such circumstances:

1. Test and Measuring Equipment requires calibration no matter what. The simple justification is that if they didn't care about accuracy, they would not even need the test equipment. A multimeter for example is used because they want to check a voltage. If they didn't care about the voltage value, they could use a lightbulb with two test leads attached and determine whether the voltage was present. The very fact that they used a "METER" says they want to know a value. An uncalibrated meter is also an unreliable meter. Therefore, there is a potential safety hazard involved.

2. Uncalibrated Test and Measuring Equipment could sometimes erroneously be used in production. If you cannot prove that the uncalibrated test equipment could NOT possibly be used in production, a potential quality risk is introduced.

3. Engineering/R&D produces results which are directly or indirectly related to developing product specs. Uncalibrated test and measurement equipment used in engineering could cause faulty decisions in design. Then when the product is brought to production those faulty assumptions can cost money, due to inaccurate product specs.

4. Power Supply calibrations. The user does not measure AC ripple, transient response, and full specs of the power supply. The calibration produces improved confidence in the reliability of the power supply. Additionally, to be quite frank, I am pretty cynical about the user externally monitoring the output. They can monitor under a given set of operational conditions. But they very likely don't monitor under a full variety of loads, etc. Some older technology power supplies are not designed with an output accuracy. However, some of the new GPIB controlled precision supplies (such as the HP 6620 series, 6650 series, E3630, etc.) are highly accurate, some to specs of around +/-0.05%. Regardless of what they claim, when users see a high resolution display on a power supply, they expect the output values to be correct. If the supply is not calibrated, it can cause some potentially costly errors in product. My customers use supplies in (fictitious numbers to protect proprietary details) TTL circuitry to design component characteristics. The supplies are set for +5.25 VDC (fictitious). The circuit is loaded and run under various conditions to determine product performance and to provide feedback to the factory. Current to voltage relationships are measured over the GPIB bus, plotted and sent to design teams for feedback. These details are as much a part of determining product specs are factory testing is a part of verifying the product meets specs.

I even have lower tech users who after I described technically what risk potentials are possible, agreed they didn't want to risk it, and agreed to have the power supplies calibrated.

It is understandable that an engineering manager wants to guard his/her budget. However, they must intelligently understand potential risk to product, and be willing to take it.

If you have ISO requirements in that area, that turns the entire topic up a notch (to quote Emeril Lagasse). If you have to send them out and incur a per-item charge to the users department, that also adds some other implications.

Hope this is of some help. I've fought this battle many times. Each time it has been through helping the customer understand the risk that I was able to convince them.

[Jerry Eldred]

NOTE TO ROB: I loved the quote at the end of your reply.
-Jerry Eldred

[Bill Pflanz]

In Reply to Parent Post by Rob Nix

Have you experienced problems due to "non-calibrated" devices is engineering?

I agree with Rob on only calibrating what is really needed. When we did our first ISO registration, maintenance originally said nothing needed calibrated. After convincing them that it did not seem reasonable to do no calibration since even they found a need to keep their instruments calibrated, they came back and said all instrumentation needed calibration.

I finally had them talk to operations about which instruments they relied on for process control and we came up with the final list of critical instruments. If I remember correctly it was only about 10-20% of all the devices. Of course, if any instrument was obviously not metering correctly, the maintenance could and did calibrate. The difference was that the critical instruments were required to be calibrated on a regularly scheduled basis.

Bill Pflanz

[Jerry Eldred]

I suppose I come down somewhere in between the two extremes:
A - Calibrate everything
B - Calibrate only what you "absolutely have to."

I have worked as a consultant in many factories and seen results of B. When B is implemented, cost of calibration seems to outweight quality considerations. I started a lab in a factory where B was implemented. There was seemingly constant question about whether product was good or bad, and first pass yield for circuit boards was 0%. In another factory I have worked with (where communications equipment was manufactrured), they used a "Golden Test Bay" philosophy. Most routine non-critical testing was done on normal test bays, which were periodically compared with the Golden Test Bay. The calibration lab learned (very large factory) that maintenance people would arbitrarily swap test equipment between bays, including the Golden Test Bays.

The point being that inconsistent calibration philosophy regarding any given type of instrument is risk potential. I have worked much of my career in and around Fortune 100 and major brand name companies. I have also worked in and around smaller, cost-conscious companies where B was implemented. Exposure to both philosophies and the consequences has long made me a proponent of implementing close to an A philosophy (as at the top of this posting).

If the instrument is made to measure quantitatively and the user does not want it calibrated, that means he or she doesn't need an instrument.

I must confess though that much of my exposure has been to high quality, precision, aerospace or other compliant products where specs are very important. So my views are biased to that context. There are many companies out there I am sure, where critical specs are minimal enough to invalidate many of these practices. In those cases I would tend to agree (which I already do) that only those items critical to product specs must be calibrated.

The mental test I use in my environment is to ask if I misadjusted the meters on the unit by (for example) 50% or 75%, but the user didn't know I had done so, would those incorrect readings on that unit cause the user to make a bad decision that would impact form, fit or function on product, or would any potential safety issues result?

I must comment also on one other detail in a previous reply. I don't think it is fair to judge whether or not to calibrate based on whether problems have been experienced due to non-calibrated instruments in engineering. That may be the case, but only after an instrument has been uncalibrated for quite a long time. Remember that calibrated accuracies are designed to normally remain in-tolerance for quite a long period after the normal due date. Calibration intervals are established to maintain an acceptable confidence factor that the unit will be in-tolerance. Therefore it may be reasonable to assume some stable instruments could remain in tolerance for years without calibration. I've actually plotted this in one previous lab I started. I began with about 1000 instruments that had not been calibrated for probably 7 or 8 years. My percent in tolerance during first pass was about 62%. That increased annually for about three years until I hit the high 90's (a desirable percentage). This means that it could take many years of no calibrations before everything falls apart from lack of adjustment or maintenance. Therefore, decision not to calibrate, even if it is wrong, could take quite a few years before all the uncalibrated instruments come back to "bite you in the rear."

I would advise caution in using that as a criterion.

[Mike S.]

I don't calibrate everything in the R&D area. I usually decide by asking myself, "If it were my business and my money, would I pay for a cal?" There is more than one way to ensure correct measurements. Calibration doesn't guarantee anything, anyway, since a day after cal. something could go wrong that would cause the instrument to go out of cal. One tool I often use -- whether the item is calibrated or not -- is verification -- measuring a known internal "standard" similar to my unknowns every day prior to measurement of any unknowns. JMO

[Jerry Eldred]

My escape clause in this discussion is that I know I have worked around some rather tight criteria. I'm not supposed to discuss details from within my company. But suffice it to say that our products are sometimes installed in applications that could mean life or death to the user if the product fails (not all the time, but some of it). So a watchword in my context is "Guardband". If a customer (in some of the critical product applications) finds a single defective component, they have been known to shut down very high dollar assembly lines at a cost of hundreds of thousands of dollars per day or hour.

So although my integrity demands that I stick to my viewpoint, I also must allow that not all products are like that. That doesn't change the question or the answer, just the criteria applied to get the answer.

The questions must be asked (with appropriate resolution for your context) as to whether an inaccurate reading or output from the instrument will cause product form, fit or function to deviate from stated specifications. I strongly avoid asking a money-related version of the question. Since my job is quality (calibration is a quality issue), I would be unduly biased if I asked money-related questions.

If I establish that inaccuracy in an instrument yields product deviation from specs, the money question would be how to adequately assure confidence in the accuracy of the instrument to give the user the accuracy they need, at the least cost. The answer may or may not be using a cheaper vendor (you get what you pay for), as cheaper vendors may or may not provide adequate confidence in measurements made by the instrument.

For example, we write custom procedures in some circumstances, with appropriate labelling. A multimeter installed in a system, but used exclusively for DC volts, could have a limited calibration where only DC volts are tested. We then apply what we call a "Red-Line" label (I believe comes from USAF PMEL). The procedure specifies meters hard installed in certain automated testers, and clearly labelled on the cal sticker that only DC volts may be used. There are also some simpler methods for calibrating power supplies that provide users needed confidence while reducing manhours.

I've been longwinded enough for one day.