Calibration Intervals (Frequency) derived from Variables Data

[rdragons]

Kansas has been in Texas all week witnessing testing.

Marc: To elaborate on BradM post. It’s the cost of quality. It’s a bathtub curve.

For calibration intervals to short, manpower and out of service time create additional cost.

For calibration intervals to long, risk of shipping nonconforming product increases. The lower cost level is Customer Satisfaction and Rework. The upper cost level is litigation both contract and liability. Additional cost, just a different flavor.

We want to be in the bottom of the tub. …..did I type that?

Follow up on Wellstone crash final report cleared the VOR, citing pilot error. So I’m not so scared any more.

Alternator………would that be the Horton Emergency Vehicles, John Molinari, Bobby Labonte, or just plain Nissan

http://www.ems.ohio.gov/special/NHTSAnotice04.htm

http://www.njatty.com/articles/auto/jmsm03.html

http://www.joegibbsracing.com/season...15_bl_race.php

http://www-odi.nhtsa.dot.gov/cars/pr...intVersion=YES

Sorry Marc, couldn’t resist. But you made your point. We don’t care if a few barbecue covers are an inch to long. And a pilot is supposed to be able to recognize a broken VOR.

Back to basics……….

Does anyone out there use any form of variables calibration interval analysis?

Hershal where are you? Still want to know about downloading.

[BradM]

In Reply to Parent Post by Marc

I guess what I was getting at is there are situations where such a detailed analysis isn't necessary. In other situations such a level of analysis is not just important, it is critical.

Sorry, Marc. Did not see your response on this one. It must have been during my "where are my notifications going?" phase.

I agree totally with you about what I perceive is the sentiments of your follow-up post, and the appreciation for Hersal (and all your professional covers) who take the time daily to share their expertise here.

I think experience and time teaches the professional about calibration frequency. I think even if you are a beginner, after you have instruments calibrated over a few cycles, one can begin to see whether the intervals are established properly (if you're looking).

I got excited (and am still excited) about this thread when there is some life given to actually looking at your calibration program and managing it. So many times, people get something calibrated, file it, and move on. They never look at their certificate, review the work being done, determine if the proper frequency is in place, if they have the right tool, etc. These forums are jam-packed with confused individuals who legitimately ask two fundamental questions: 1) what is my tolerance, and 2) what should my calibration interval be?

Ok, for #1, most of the time we say Mfg. tolerance. Since I'm talking to my cohorts, I ask:" how many of you have confidence in Mfg. specification??" If it's Fluke, I trust it implicitly. If it's XXXXXXXX, I have no confidence, and establish my own. So then there are several instruments in-between. There is no oversight whatsoever as to how companies ascertain their stated accuracies or recommend intervals. I know, that's a broad brush. But like I said, when I look at Fluke's analysis, you know they know what they're doing. Others?? You have to dig (sometimes I have to call) to find if there is any stated tolerance for the equipment even established.

By following established uncertainty analysis procedures, you can determine this. This is why I appreciate Hershal's torch-carrying on ISO17025 and referring people to legtimate, accredited labs that take pride in their work and do it right.

As for #2, in my experience, this one is a little more tricky. Say four of us bought out a calibration department of a corporation. What would be our intervals? Set aside objective analysis for a second. I bet it would be pretty short, right? We're not being deceptive or unethical. We are a business to make money (short interval means more money$$) and we're erring on the customer's safety side by not letting it go too long and being O.O.T. Sounds like a win-win, yes?

I just think it would be neat to have a tool to run some numbers through should I desire. If I could extend some and shorten others, giving me some $$$ saved to demonstrate to management, that has some promise. But to your point, it would probably not be useful to many others who manage a small group of instruments.

But.... that goes to the database thread. Where Access is the greatest thing in the world to me; for people who really work with databases, they prefer much more robust packages.

[rdragons]

In Reply to Parent Post by BradM

As for #2, in my experience, this one is a little more tricky. Say four of us bought out a calibration department of a corporation. What would be our intervals? Set aside objective analysis for a second. I bet it would be pretty short, right? We're not being deceptive or unethical. We are a business to make money (short interval means more money$$) and we're erring on the customer's safety side by not letting it go too long and being O.O.T. Sounds like a win-win, yes?

Sorry BradM. Short interval more $$ is not win win. Step back and look at the big picture.

Company A voltmeter has a recommended calibration interval of 1 year. Company B voltmeter has a recommended calibration interval of 2 years. Both meters have the same tolerance and can be used for our application. Guess which one I’m going to buy. So while company A makes short interval dollars they are losing market share to company B.

This is where my calibration interval analysis is going. By identifying the current calibration interval design limitations, we can redesign to improve our product towards longer calibration interval and put the competition out of business. Well at least make them uncomfortable.

[Marc]

In Reply to Parent Post by rdragons

By identifying the current calibration interval design limitations, we can redesign to improve our product towards longer calibration interval and put the competition out of business. Well at least make them uncomfortable.

I agree, but then again... Manufacturer claims (even warranty aspects) may not tell the whole story. One would have to take both meters and put them in identical (or very similar) use scenarios and see how well they hold their calibration.

In most situations I would look for what I felt was the best 'built' device or instrument, I would set a 'reasonable' calibration interval and check results each time calibrated. If you get a new meter that is used daily in a very rough environment I wouldn't trust the manufacturers recommendation. I'd look to other multi-meters I already had and start there. Obviously this wouldn't apply in a startup with no history, but those are the exception.

I think for most non-critical (lives do not depend up it) applications which is the case in many companies, measurement equipment calibration cycle time should be looked at in terms of calibration history. My 'rule of thumb': If the instrument keeps coming back in calibration without adjustment, lengthen frequency. If adjustment is necessary but the instrument is within its tolerance, the frequency is probably about right. And, of course, if it comes back having needed adjustment AND was out of tolerance, shorten the cycle. Note that my 'rule of thumb' is general. For example, if a review of the calibration history for the device shows that it was stable until a certain point in time (coming back in calibration without adjustment, or minimal adjustment is necessary but the instrument is within its tolerance), one should be looking at the integrity of the instrument (for example, is it wearing out?).

Now, let's take another scenario I'd like feedback on. A company has 20 digital multi-meters. There are 5 in the calibration laboratory and each is used approximately 5 times a day, 2 are kept by product engineers whose use is not able to be tracked, 13 are used on the line and each is used at 15 minute intervals on each of 2 shifts, and 5 of the 13 are also use on a 3rd shift (same 15 minute interval scenario). Not relying on calibration history, how would one set a calibration cycle for each?

[BradM]

In Reply to Parent Post by rdragons

Sorry BradM. Short interval more $$ is not win win. Step back and look at the big picture.

Company A voltmeter has a recommended calibration interval of 1 year. Company B voltmeter has a recommended calibration interval of 2 years. Both meters have the same tolerance and can be used for our application. Guess which one I’m going to buy. So while company A makes short interval dollars they are losing market share to company B.

This is where my calibration interval analysis is going. By identifying the current calibration interval design limitations, we can redesign to improve our product towards longer calibration interval and put the competition out of business. Well at least make them uncomfortable.

Excellent job. And agreed

Realize my previous post was Devil's Advocacy, and a bit tongue-in-cheek. Observe my statement about setting objective evalution aside. I was implying that most companies out there (IMO) are approaching it this way. I believe many do not have rational approaches to establishing frequency intervals. Too, more of the population responds positively to price and stated accuracy. Few have the knowledge (or the tools, or the desire) to critically analyze the accuracy, calibration methology, etc. As Marc suggested, there are so many variables to the same instrument, it would be difficult to do. Unless... a robust model can be utilized.

[John Nabors - 2009]

I up came with a system several years ago that is competely arbitrary but has served me well. If an instrument has needed adjustment within two calibration intervals I reduce the inverval by 50%. If it passes through 4 intervals without requiring adjustment I increase it by 50% and then review it after four more calibration intervals to see if I can extend it further.

Not very scientific, but it has worked for me.

[BradM]

I think for most non-critical (lives do not depend up it) applications which is the case in many companies, measurement equipment calibration cycle time should be looked at in terms of calibration history. My 'rule of thumb': If the instrument keeps coming back in calibration without adjustment, lengthen frequency. If adjustment is necessary but the instrument is within its tolerance, the frequency is probably about right. And, of course, if it comes back having needed adjustment AND was out of tolerance, shorten the cycle. Note that my 'rule of thumb' is general. For example, if a review of the calibration history for the device shows that it was stable until a certain point in time (coming back in calibration without adjustment, or minimal adjustment is necessary but the instrument is within its tolerance), one should be looking at the integrity of the instrument (for example, is it wearing out?).

I think that is an excellent system, and would serve most fairly well. Basically you have three decision criteria: 1) Failed calibration, 2) passed calibration (but with adjustment), 3) passed calibration (no adjustment). 1 and 3 are fairly consistent; 2 is my interest. Does the vendor always adjust, never adjust unless OOT, adjust at 50%? If I had a little more of a robust system to take the percentage and estimate a frequency, that would be useful.

In Reply to Parent Post by Marc

Now, let's take another scenario I'd like feedback on. A company has 20 digital multi-meters. There are 5 in the calibration laboratory and each is used approximately 5 times a day, 2 are kept by product engineers whose use is not able to be tracked, 13 are used on the line and each is used at 15 minute intervals on each of 2 shifts, and 5 of the 13 are also use on a 3rd shift (same 15 minute interval scenario). Not relying on calibration history, how would one set a calibration cycle for each?

Good one. Why would you not want to at least consider calibration history? Is it not available? I’m saying whether you go off your rule of thumb, or the most sophisticated system available, any realistic forecast should start with historical performance (IMO).

[rdragons]

ALGORITHMIC METHODS
Other methods utilize simple to complex decision algorithms to adjust calibration intervals in response to in-tolerance or out-of-tolerance conditions observed during calibration. Typically, these approaches consist of instructions to lengthen or shorten calibration intervals in response to current or recent observations. Because of their nature, these methods are labeled algorithmic methods. Algorithmic methods have achieved wide acceptance due to their simplicity and low cost of implementation. However, most algorithmic methods suffer from several drawbacks.

The following list is fairly representative:

1. With most algorithmic methods, interval changes are in response to small numbers (usually one or two) of observed in-tolerance or out-of-tolerance conditions. It can be easily shown that any given in-tolerance or out-of tolerance condition is a random occurrence. Adjusting an interval in response to small numbers of calibration results is, accordingly, equivalent to attempting to control a process by adjusting to random fluctuations. Such practices are inherently futile.

2. Algorithmic methods make no attempt to model underlying uncertainty growth mechanisms. Consequently, if an interval change is required, the appropriate magnitude of the change cannot be readily determined.

3. Algorithmic methods cannot be readily tailored to prescribed reliability targets that are commensurate with quality objectives. The level of reliability attainable with a given algorithmic method can be discovered only by trial and error or by simulation.

4. If an interval is attained that is consistent with a desired level of reliability, the results of the next calibration or next few calibrations will likely cause a change away from the correct interval. To see that this is so, consider cases where reliability targets are high, e.g., 90%. For a 90% target, if the interval is correct for an item, there is a 0.9 probability that it will be observed in-tolerance at any given calibration. Likewise, there is a 0.81 probability that it will be observed in-tolerance at two successive calibrations. With most algorithmic methods, such observations will cause an adjustment away from the item’s current interval. Thus, algorithmic methods tend to cause a change away from a correct interval in response to events that are highly probable if the interval is correct.

5. With algorithmic methods, although a correct interval cannot be maintained, a time-averaged steady-state measurement reliability can be achieved. The typical time required ranges from fifteen to sixty years.

6. With algorithmic methods, interval changes are ordinarily computed manually by calibrating technicians, rather than established via automated methods. Accordingly, operating costs can be high.
Quote from Dr. Castrup

In metrology world it is the user of the product that has responsibility to establish calibration interval based on their usage. Which is in conflict with the fact the manufacture of the product has the largest database of information available to establish calibration intervals.

I think the dilemma is “instrument” vs. “instruments”. If I have one instrument and want to adjust cal interval the only method that will work is an algorithmic method. “John Nabors” has a good one, but once the average is found the cycle of 4 good cals and two OOT cals is a Reliability of 66.66%. Which means it is returned out of cal 33.33% of the time. There just isn’t enough data to work with.

I have a large group of instruments and if you refer back to plot03 last graph you will note the average is 125 weeks the dogs are 69 weeks and the gems are 180 weeks. I don’t currently know if this is random with increasing uncertainty over time or if there really are gem instruments that will go for 180 weeks. If it’s random “John Nabors” algorithmic will be constantly searching a 69 to 180 week window. If there are really gems in the group and “Jim Nabors” owns it algorithmic may save some money.