SBS - The best value in QMS software

"It depends on the Auditor" Is excessive variation jeopardizing system certification?

Do we have excessive variation of interpretation among management system auditors?

  • No, we don't see variation between different auditors

    Votes: 0 0.0%
  • We see some variation, but it is manageable

    Votes: 18 52.9%
  • We see variation and it hurts our system

    Votes: 7 20.6%
  • We see excessive variation between different auditors

    Votes: 9 26.5%

  • Total voters
    34
  • Poll closed .

Dan M

Involved In Discussions
#1
It depends on the auditor. Our first auditor 6 years ago said we had to do gage studies on the gages listed within our lab
Scope, but did not care about the gages listed in the production control plan.

Our new auditor this year wrote a minor nonconformance because MSA studies were Not completed for gages listed on the control plan
 
Elsmar Forum Sponsor

Sidney Vianna

Post Responsibly
Staff member
Admin
#2
Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

I copied Dan M's post from another thread to start this discussion. While we all know that some variation will always exist among auditors of management systems, do we have too much variation? Is the range of interpretations compromising the intent of standardization? Are the conformity assessment sector not doing enough to reduce variation among professionals?

Dan M's initial sentence "It depends on the auditor" is quite a serious indictment of the auditing organizations, isn't it? To operate under a process whereby your system is good or bad depending on the external auditor du jour is not healthy.

I had approached some people in the ISO CASCO group with some suggestions on how to attempt to minimize variation in the management system assessment sector. Got zero response. I wonder if they care, sometimes. Despite the fact that management systems will always vary due to context (internal and external), I personally believe that there is too much variation and we should be working on reducing it.

I also added a poll.
 
Last edited:

Ninja

Looking for Reality
Staff member
Super Moderator
#3
Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

The poll, unavoidably, is pretty subjective. Interesting, though to see what folks think.

I, for one, expect variation between auditors and would not be surprised even a little if two auditors had opposing views on the same thing...we see it here on the forum all the time.

I voted "manageable", because auditors are humans...and cost/benefit analysis is used to decide whether to fight a finding, or simply comply with a human's interpretation of something.

If it were an option in the poll, I would have voted "Variation is large enough to undermine the value of accreditation". Not remove the value, mind you...but to undermine it...
 

Kronos147

Trusted Information Resource
#4
Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

The MR has to be proactive. Make sure the auditor cites the non-conformance against a requirement, read the requirement yourself, then see if there is anything the auditor missed.

If it still seems subjective after discussions, and there is no consent from the MR, yet the assessor insists it will be an official non-conformance, end the audit, or appeal the finding.

Finally, if an independent person cannot verify the written non-conformance, there may be another reason for an appeal.
 

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#5
Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

The MR has to be proactive. Make sure the auditor cites the non-conformance against a requirement, read the requirement yourself, then see if there is anything the auditor missed.

If it still seems subjective after discussions, and there is no consent from the MR, yet the assessor insists it will be an official non-conformance, end the audit, or appeal the finding.

Finally, if an independent person cannot verify the written non-conformance, there may be another reason for an appeal.
Spot on. There is variation among auditors, and try as they might the CBs will probably never fully "calibrate" us. The clients can do their part by fearlessly disputing nonconformances which do not list a clear "shall" or relevant interested party requirement. A good CB will include the overturned action requests in their regular training. It benefits us and the clients to clarify and re-affirm our boundaries.
:2cents:
 
#6
Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

Spot on. There is variation among auditors, and try as they might the CBs will probably never fully "calibrate" us.
"Try"? I see little evidence of "try" and, I'm aware of auditors who don't believe they need "calibration"...
 

Howard Atkins

Forum Administrator
Staff member
Admin
#7
Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

We are not talking about variation rather incorrect information.
The lab issue above is simply an error on the part of the initial auditor which the company should have questioned as to where it states this in the standard.

The variation is on some cases on the accreditation body, the IATF consistently accuses us of soft grading and not finding more Major Nonconformacies, monitoring each CB to see the number given and the ration of minor to major.
One big factor that involves variation is the auditee, if one sees that he is intent on improving then the carrot my be more beneficial than the stick.
 

ScottK

Not out of the crisis
Staff member
Super Moderator
#8
Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

I fearlessly dispute things when necessary.
Still have the rare hour long argument about calibrating tape measures for a +- 1" reference specification.

So - I really don't think it's possible to have 100% calibration but I also don't see the variation as excessive for NBs and CBs, in general. Once in a while there is an outlier.

Although there was one instance where I switched registrars... the last auditor from the prior registrar had worked for the new and when the new registrar came in for the transfer audit and reviewed last findings, the auditor commented something like "yeah, I worked with that auditor. These couple of findings are BS and typical of that person work".

The absolute worst experience I have had with auditors have been with SQA Services coming in to represent customers. The variation is wild as each of the auditors are contractors. I dread getting calls from them saying "we're auditing on behalf of ________".
 
#9
Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

In my world, I hear at least once a month - from various sources - of clients who have had even basics missed from their audits for years! We aren't talking about calibrating auditors in terms of a degree of precision or accuracy, here. We're talking basics such as;

Zero MSAs ever done
Zero Engineering staff competencies ever performed (safety related too!)
Zero Inspection records

We forget that 3rd Party Certification has been with us since the start of the 1990s. These gross omissions from certifications shouldn't be happening!
 

Golfman25

Trusted Information Resource
#10
Re: "It depends on the Auditor" Is excessive variation jeopardizing system certificat

As I see the fundamental problem is there is no oversight body "ruling" on what is a good interpretation and what isn't. Sure, you can appeal but what guides the appeal? Most of this is subjective -- does what I am doing comply with the standard? So if you do something out of the norm, then you're more likely to have a "finding." It takes work to figure out if something not frequently seen is compliant. Just look at the non-required process map.

So what you end up with is disjointed interpretations between auditors and CBs. Just look at the interpretations here. If there was a database of appeal results maybe it could be used as lessons learned. Without it, we are all just guessing.
 
Thread starter Similar threads Forum Replies Date
T System Corrective Actions when Conforming Product depends heavily on Inspection Nonconformance and Corrective Action 3
bobdoering A Statistician?s Favorite Answer: ?It Depends,? Part 2 Statistical Analysis Tools, Techniques and SPC 2
bobdoering A Statistician?s Favorite Answer: ?It depends,? Part 1 Statistical Analysis Tools, Techniques and SPC 1
O MSA Guidelines Appendix C - Value constant K3, which depends upon the number of parts Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
J IATF 16949 Internal Audit question - Auditor's responsibility Internal Auditing 6
W Redacting Info Before Giving to Auditor ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
T Quality auditor legal right to see Board meeting minutes ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
V Certified Auditor - Need of additional certification specific to industry ( GMPs) ASQ vs ECA vs others Professional Certifications and Degrees 1
V Internal Auditor Competency KPI IATF 16949 - Automotive Quality Systems Standard 14
R American Petroleum Institute - Becoming an API Auditor Professional Certifications and Degrees 2
B Lowest cost way to pass Lead Auditor exam ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
B Internal Auditor Competency - Product Auditors Internal Auditing 9
U Internal Auditor not trained but done Audit for some process Nonconformance and Corrective Action 5
Z Auditor Findings ISO 14001:2015 vs. 45001:2015 ISO 14001:2015 Specific Discussions 6
B IATF16949 audit requirement - Auditor request UCL and LCL must be show Xbar-R, IATF 16949 - Automotive Quality Systems Standard 7
A Becoming an ISO27001 3rd Party Auditor Career and Occupation Discussions 4
L ASQ's Biomedical Auditor Course Test ASQ - American Society for Quality 1
M Tips on preparing for IATF 16949 Internal Lead Auditor exam Manufacturing and Related Processes 1
G Same parts but new customer - What will the auditor ask me? IATF 16949 - Automotive Quality Systems Standard 2
Gun46 ISO 9001 : 2015 Lead Auditor Exam General Auditing Discussions 16
K %GRR was between 10-30% so we have to have a "backup plan" per auditor IATF 16949 - Automotive Quality Systems Standard 15
S ISO 13485 Lead Auditor - Debate between our Quality Team and Regulatory Auditor - Internal Auditor Training ISO 13485:2016 - Medical Device Quality Management Systems 17
R ISO 45001 Lead Auditor Exam paper Training - Internal, External, Online and Distance Learning 1
B Internal and external auditor competency to CSR's IATF 16949 - Automotive Quality Systems Standard 20
A Our auditor told if we didn't have a patent we would have to do a validation or verification ISO 13485:2016 - Medical Device Quality Management Systems 6
W Certification for IATF Lead Auditor will expire in 2020 IATF 16949 - Automotive Quality Systems Standard 2
D Impartiality of Internal Auditor ISO 9001/13485 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 18
Ed Panek Auditor driving us nuts - ESD requirements ISO 13485:2016 - Medical Device Quality Management Systems 23
A OHSAS 18001 external auditor finding personal interpretation? Occupational Health & Safety Management Standards 5
S IRCA Lead Auditor training and Exam tips Training - Internal, External, Online and Distance Learning 5
L ASQ CBA biomedical auditor - CBA primer material is enough to study? ISO 13485:2016 - Medical Device Quality Management Systems 6
B VDA 6.3 Qualification as Process Auditor training course and exam VDA Standards - Germany's Automotive Standards 0
F ISO 21001 Educational Organizations Management - How to become an auditor Other ISO and International Standards and European Regulations 1
J Getting training either from ASQ or from SAI Global - ISO 9001 Lead Auditor training Training - Internal, External, Online and Distance Learning 1
P ASQ Certified Biomedical Auditor (CBA) Certification Preparation 2019 ASQ - American Society for Quality 3
M Medical Device Design Control Auditor Recommentations General Auditing Discussions 19
G Third party auditor mentions no grace period for calibration Calibration Frequency (Interval) 22
D Where (in US) can I get the VDA Auditor Edition book? VDA Standards - Germany's Automotive Standards 3
S AIAG CQI Auditor Qualification and 3rd Party Certification Requirements General Auditing Discussions 2
M IATF 16949 7.2.3 Internal Auditor Competency - Trainer's competency Internal Auditing 7
C Recommendations for UK-based ISO 13485 internal auditor training ISO 13485:2016 - Medical Device Quality Management Systems 1
Sidney Vianna AS9100 News July 2019 AAQG/RMC CB Auditor Workshop - Presentation Materials AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 0
D Scope of Facility - Our auditor asked us last week for our "Scope of the Facility" AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 12
A ISO 9001 lead auditor as Full time career India Career and Occupation Discussions 2
J Manufacturing Process Auditor Requirements - IATF 16949 IATF 16949 - Automotive Quality Systems Standard 9
GreatNate ISO 9001:2015 Lead Auditor Course? (who to take with) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 15
A External Auditor issue with Internal Audits Internal Auditing 7
Q Internal Auditor competence for ISO 14001 ISO 14001:2015 Specific Discussions 11
S IATF 16949: Is "Certified" Internal Auditor mandatory? IATF 16949 - Automotive Quality Systems Standard 9
S Internal Auditing for API Spec Q1 - auditor qualification requirements Oil and Gas Industry Standards and Regulations 6

Similar threads

Top Bottom