"It depends on the Auditor" Is excessive variation jeopardizing system certification?

[Dan M]

It depends on the auditor. Our first auditor 6 years ago said we had to do gage studies on the gages listed within our lab
Scope, but did not care about the gages listed in the production control plan.

Our new auditor this year wrote a minor nonconformance because MSA studies were Not completed for gages listed on the control plan

 

[Sidney Vianna]

Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

I copied Dan M's post from another thread to start this discussion. While we all know that some variation will always exist among auditors of management systems, do we have too much variation? Is the range of interpretations compromising the intent of standardization? Are the conformity assessment sector not doing enough to reduce variation among professionals?

Dan M's initial sentence "It depends on the auditor" is quite a serious indictment of the auditing organizations, isn't it? To operate under a process whereby your system is good or bad depending on the external auditor du jour is not healthy.

I had approached some people in the ISO CASCO group with some suggestions on how to attempt to minimize variation in the management system assessment sector. Got zero response. I wonder if they care, sometimes. Despite the fact that management systems will always vary due to context (internal and external), I personally believe that there is too much variation and we should be working on reducing it.

I also added a poll.

 

[Ninja]

Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

The poll, unavoidably, is pretty subjective. Interesting, though to see what folks think.

I, for one, expect variation between auditors and would not be surprised even a little if two auditors had opposing views on the same thing...we see it here on the forum all the time.

I voted "manageable", because auditors are humans...and cost/benefit analysis is used to decide whether to fight a finding, or simply comply with a human's interpretation of something.

If it were an option in the poll, I would have voted "Variation is large enough to undermine the value of accreditation". Not remove the value, mind you...but to undermine it...

 

[Kronos147]

Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

The MR has to be proactive. Make sure the auditor cites the non-conformance against a requirement, read the requirement yourself, then see if there is anything the auditor missed.

If it still seems subjective after discussions, and there is no consent from the MR, yet the assessor insists it will be an official non-conformance, end the audit, or appeal the finding.

Finally, if an independent person cannot verify the written non-conformance, there may be another reason for an appeal.

 

[Jen Kirley]

Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

The MR has to be proactive. Make sure the auditor cites the non-conformance against a requirement, read the requirement yourself, then see if there is anything the auditor missed.

If it still seems subjective after discussions, and there is no consent from the MR, yet the assessor insists it will be an official non-conformance, end the audit, or appeal the finding.

Finally, if an independent person cannot verify the written non-conformance, there may be another reason for an appeal.

Spot on. There is variation among auditors, and try as they might the CBs will probably never fully "calibrate" us. The clients can do their part by fearlessly disputing nonconformances which do not list a clear "shall" or relevant interested party requirement. A good CB will include the overturned action requests in their regular training. It benefits us and the clients to clarify and re-affirm our boundaries.

 

[AndyN]

Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

Spot on. There is variation among auditors, and try as they might the CBs will probably never fully "calibrate" us.

"Try"? I see little evidence of "try" and, I'm aware of auditors who don't believe they need "calibration"...

 

[Howard Atkins]

Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

We are not talking about variation rather incorrect information.
The lab issue above is simply an error on the part of the initial auditor which the company should have questioned as to where it states this in the standard.

The variation is on some cases on the accreditation body, the IATF consistently accuses us of soft grading and not finding more Major Nonconformacies, monitoring each CB to see the number given and the ration of minor to major.
One big factor that involves variation is the auditee, if one sees that he is intent on improving then the carrot my be more beneficial than the stick.

 

[ScottK]

Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

I fearlessly dispute things when necessary.
Still have the rare hour long argument about calibrating tape measures for a +- 1" reference specification.

So - I really don't think it's possible to have 100% calibration but I also don't see the variation as excessive for NBs and CBs, in general. Once in a while there is an outlier.

Although there was one instance where I switched registrars... the last auditor from the prior registrar had worked for the new and when the new registrar came in for the transfer audit and reviewed last findings, the auditor commented something like "yeah, I worked with that auditor. These couple of findings are BS and typical of that person work".

The absolute worst experience I have had with auditors have been with SQA Services coming in to represent customers. The variation is wild as each of the auditors are contractors. I dread getting calls from them saying "we're auditing on behalf of ________".

 

[AndyN]

Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

In my world, I hear at least once a month - from various sources - of clients who have had even basics missed from their audits for years! We aren't talking about calibrating auditors in terms of a degree of precision or accuracy, here. We're talking basics such as;

Zero MSAs ever done
Zero Engineering staff competencies ever performed (safety related too!)
Zero Inspection records

We forget that 3rd Party Certification has been with us since the start of the 1990s. These gross omissions from certifications shouldn't be happening!

 

[Golfman25]

Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

As I see the fundamental problem is there is no oversight body "ruling" on what is a good interpretation and what isn't. Sure, you can appeal but what guides the appeal? Most of this is subjective -- does what I am doing comply with the standard? So if you do something out of the norm, then you're more likely to have a "finding." It takes work to figure out if something not frequently seen is compliant. Just look at the non-required process map.

So what you end up with is disjointed interpretations between auditors and CBs. Just look at the interpretations here. If there was a database of appeal results maybe it could be used as lessons learned. Without it, we are all just guessing.