Calibration Vendor (Supplier) Performance and Expectations

[psquared]

Holiday greetings:

In response to calibration OIs during ISO 9001 audits we have been taking a closer look at our calibration practices both in-house and vended.

I have a few issues I could use a sanity check on. I do have a copy of ISO 17025- 2005 but my questions remain.

One vendor has the habit of listing on their calibration cert, in the field labeled "accuracy" the phrase "per manufacturers manual." This is done although they may not have the Mfgs. manual. In the same cert and data sheet they show no tolerance, adjustment limits or any objective value by which they determine if the difference between applied load and the measured value are within allowable limits.

1)How can this be justified- shouldn't the test criteria be documented?

2)Also isn't claiming a reference to the manufacturers specs, when they don't have it poor practice at best and in reality, falsifying information.

3)When 2 analogue force gauges would not read full scale in one direction they recorded NA for the values, did not identify the gauges as damaged, noted "see data" on the cert, but put white calibration stickers on them with "certification" dates,and due dates filled out.

Last question do these practices sound shoddy for an ISO 17025 accredited calibration house. We use other houses that demonstrate much better work practices"

Your comments will be greatly appreciated.

psquared

 

[BradM]

Holiday greetings

And the same to you!!

One vendor has the habit of listing on their calibration cert, in the field labeled "accuracy" the phrase "per manufacturers manual." This is done although they may not have the Mfgs. manual. In the same cert and data sheet they show no tolerance, adjustment limits or any objective value by which they determine if the difference between applied load and the measured value are within allowable limits.

1)How can this be justified- shouldn't the test criteria be documented?

I am intrigued. How are you sure they don't have a copy of the manual? Too, what are they referring to as the calibration procedure? Maybe they have their own internal procedures, or possibly a GIDEP procedure?

2)Also isn't claiming a reference to the manufacturers specs, when they don't have it poor practice at best and in reality, falsifying information.

Well... kind of with #1, how do you know they don't have that information? Possibly they have templates and such set up with that information on it?

3)When 2 analogue force gauges would not read full scale in one direction they recorded NA for the values, did not identify the gauges as damaged, noted "see data" on the cert, but put white calibration stickers on them with "certification" dates,and due dates filled out.

Did they note the failure on the paperwork, on a work order, or contact you (or someone at your organization) about the instruments?

Last question do these practices sound shoddy for an ISO 17025 accredited calibration house. We use other houses that demonstrate much better work practices

Well... on the surface, it does sound highly suspicious. Some thoughts:

1. Issue a report to them, in writing, of what happened, what you observed, your complaints, etc. I would make sure someone in charge (upper management) is given the documentation. Give them a chance to rectify the problem; or clarify is something may be misunderstood.

2. Drop them and use someone else. Yes, I know that is some work. But there is nothing more unsettling than not having any confidence in a calibration vendor. Some people just want a sticker. I want a calibration service that gives me confidence. Confidence in their abilities, and confidence that I can trust what they put on a report. After all, I paid for that.

 

[George Weiss]

Greeting Psquared,

BradM was on target, so I can only say "in agreement".
I add:
Suggest changing another company's culture should not be your plan.
The problems are worth discussing with the vendor, but seemed to show many holes in performance.
1. Bad Cert
2. Bad report
3. Bad calibration practices
4. Bad CERT and Report review
5. Bad sign-off by the QA person
This can not be blamed on just the calibration tech. So asking for a re-calibration would likely find similar performances.
When it smells this fishy, I would say the cheap priced calibration was too cheap.
You did mention being ISO9001, and it is assumed you are asking about an ISO17025 calibration. your ORG is over achieving !
You did not mention if there was any uncertainty reported.

 

[Jeff Frost]

In my humble option the use of ISO 17025 is inappropriate for most organizations calibration process. I usually developed the calibration process based the requirements of ISO 9001, ISO 10012 (referenced in Cl 7.6) and ANSI/NCLS Z540.3 in combination.

A system based on these three specification/standards meets most customer and government requirements. ISO 9001 Cl. 7.6 serves as the general frame of the process. ISO 10012 contains good guidance on management of measurement process and calibration of equipment and fills in the frame of 9001. ANSI/NCLS Z540.3 plug the remaining gaps for organization performing in-house and contracted outside calibration services. Also because Z540.3 replaces Z540.1 which replaced MIL-STD-45662 you have a compliant system when 45662 is a specified requirement of customer's purchase order.

Next because this trio is geared towards most companies not offering 2nd party calibration service (I.e. ISO 17025) you have a system that works. Also you will have an understanding of what must be specified in the PO to your calibration service provider. My guess is that your current supplier is meeting your PO requirement because they do not detailed what must be reported on the certification or report.

 

[Helmut Jilling]

...2. Drop them and use someone else. Yes, I know that is some work. But there is nothing more unsettling than not having any confidence in a calibration vendor. Some people just want a sticker. I want a calibration service that gives me confidence. Confidence in their abilities, and confidence that I can trust what they put on a report. After all, I paid for that.

I agree. It is like buying a used car that shows a lot of abuse. It might run fine, but there are too many things that can go wrong. They don't sound like they are trying to do a real good job for you, the customer. There are other good suppliers out there. You have to be careful. Most calibration companies are good, but some are like really cheesy used car salesmen.

 

[psquared]

Mark, George Jeff,

Excellent points and observations.

I’ll respond to all 3 if you don’t Mind.

We are using ISO 9001 to as a catalyst for continual improvement, and to help us excel at enhancing customer and stakeholder value. (the truth, not the party line) So 9001 compliance is only the starting point of our effort to analyze and improve our calibration practices. Since, perfection can be the enemy of good, I’m trying not to split hairs over these issues, but making sure I’m not missing something. Strategically over achieving makes for some loyal customers.

The vendor in question stated they had no manual or manufacturer specs. My contention is that they should not have stated that they did. They have since done this on other of the same type gauge.

They did not provide an uncertainly value. but stated "see scope" and cited the boiler plate 95%confidence level at the k-2 coverage factor..

They did not document the gage failures – and they certified the gages with the sticker, and noted “data only” on the data page of the cert.

When informal e-mail and discussion with them failed to achieve satisfactory answers, I issued a Supplier Corrective Action Request. I am reviewing other suppliers as possible sources for this work also.

To Jeff’s point about them meeting PO requirements- point well taken, our PO is not very detail, but other 17025 accredited suppliers are doing an outstanding job doing calibrations from simple POs and are providing the full range of data, conclusions, methodology details etc... that one could hope to receive from the calibration process.
If I have to specify all of this in my PO I may be using the wrong calibration service, as some of you have indicated. Time will tell as they get back to me


Forgive this editorial statement but. It is an unfortunate reality that formal accreditations sometimes raise the performance expectation-bar more than the actual performance. Accreditation bodies find evidence of compliance, but they do not guarantee complete uniform compliance or guarantee excellence. Thank God for regular audits.

Thank you,

Paul

 

[BradM]

Mark, George Jeff,

Excellent points and observations.

I’ll respond to all 3 if you don’t Mind.

We are using ISO 9001 to as a catalyst for continual improvement, and to help us excel at enhancing customer and stakeholder value. (the truth, not the party line) So 9001 compliance is only the starting point of our effort to analyze and improve our calibration practices. Since, perfection can be the enemy of good, I’m trying not to split hairs over these issues, but making sure I’m not missing something. Strategically over achieving makes for some loyal customers.

The vendor in question stated they had no manual or manufacturer specs. My contention is that they should not have stated that they did. They have since done this on other of the same type gauge.

They did not provide an uncertainly value. but stated "see scope" and cited the boiler plate 95%confidence level at the k-2 coverage factor..

They did not document the gage failures – and they certified the gages with the sticker, and noted “data only” on the data page of the cert.

When informal e-mail and discussion with them failed to achieve satisfactory answers, I issued a Supplier Corrective Action Request. I am reviewing other suppliers as possible sources for this work also.

To Jeff’s point about them meeting PO requirements- point well taken, our PO is not very detail, but other 17025 accredited suppliers are doing an outstanding job doing calibrations from simple POs and are providing the full range of data, conclusions, methodology details etc... that one could hope to receive from the calibration process.
If I have to specify all of this in my PO I may be using the wrong calibration service, as some of you have indicated. Time will tell as they get back to me


Forgive this editorial statement but. It is an unfortunate reality that formal accreditations sometimes raise the performance expectation-bar more than the actual performance. Accreditation bodies find evidence of compliance, but they do not guarantee complete uniform compliance or guarantee excellence. Thank God for regular audits.

Thank you,

Paul

Paul, thank you for the update.

I think you need to go with your gut and stop using their services. Set aside 17025, these folks sound like they have some questionable practices. If I knew that there was a problem with a device sent to a vendor and they did not directly address it, that's a deal breaker with me.

One should not be mandated by a Accreditation body to do good work.

 

[dv8shane]

I agree with Brad drop them and I would add start investigating and possibly document what impacts their poor workmanship may have had on your company. You do not have a number for the one direction of F.S.D. on the gauges you talk about so what are you going to do now about units you have done using this device? If you tested items at this point you will have to recall them and eat the costs as well as replacing the device and calibrating it at another supplier.

I suggest you contact their accrediting board and submit your documents if they are unresponsive to your CAR

Accredited does not always mean quality although many see it as implied.

 

[Jeff Frost]

Mark, George Jeff,

To Jeff’s point about them meeting PO requirements- point well taken, our PO is not very detail, but other 17025 accredited suppliers are doing an outstanding job doing calibrations from simple POs and are providing the full range of data, conclusions, methodology details etc... that one could hope to receive from the calibration process.
If I have to specify all of this in my PO I may be using the wrong calibration service, as some of you have indicated. Time will tell as they get back to me

Looks like you have the assumption that all calibration suppliers should give you details certifications even if it is not contractually required, no matter what the additional cost may be incurred by supply for this level of certification. So your first course of action, as your indicate, is to correct your internal process for stating contractual requirements to the supplier because it looks like this supplier is meeting your PO requirements.

Next, you should open an internal corrective action to assure that, as you indicate, the questionable certification received by this supplier has not impacted product delivered to your customer. If you sufficient evidence of malfeasance then contact accreditation body and request an investigation.

Good luck