Auditor Opinion and consistency - Each auditor has different 'opinions'

[Carl Keller]

Caster,

I have certainly had some good experiences also.

Amazingly, some of them have come from Registrars that were not considered to be the "cream of the crop". In fact, my worst experience was with one of the most highly regarded (and expensive) Registrars.

There has always been a lack of consistency between registrars and a credibility problem with the RAB.

A thread concerning good experiences would no doubt have many positive contributions.

carl-

 

[Sidney Vianna]

To make an analogy, we have a incredibly well thought out and progressive Constitution in the US. That document has served us well for over 200 years. But every day the Legislative Power keeps on plugging and so does the Judicial system. Judges interpret the constitutionality of an issue on a daily basis. It is part of the "game". Since the ISO 9001 Standard was deliberately written to apply to as many as possible business environments as possible, BECAUSE of that, it does require common sense application and interpretation. If it were not for that, the TC 176 would have not developed an official for a for deliberating and posting “official/sanctioned” ISO 9001:2000 interpretations.

In my opinion, what 3rd party auditors MUST NOT do is to limit the range of acceptable interpretations to their "comfort zone". Typically, there are many different ways to comply with any requirement of a management system standard. 3rd party auditors should verify that the organization being audited has found a meaningful, sound, way to comply with the requirement for their business system. Unfortunately, poorly qualified auditors allow their baggage to interfere with the concept of a fair assessment and try to “impose” their way to system being audited. That is WRONG.

And just to exemplify how requirements NEED to be interpreted, I will borrow a requirement from AS9100, in relation to the control of scrap material:

Product dispositioned for scrap shall be conspicuously and permanently marked, or positively controlled, until physically rendered unusable.

What constitutes a conspicuous and permanent way of identifying scrap material? A red tag? Some would say that a tag is not permanent. Spray the product with red ink? Some would argue that ink is not permanent. I have seen a number of very experienced aerospace auditors arguing over ways of complying with this requirement. So the point is: there will be different interpretations over any requirement, but as long as the organization implementing the system has defined a solid way of demonstrating that they have addressed the requirement, they should not be concerned with auditor’s “opinions”.

One more point. In my opinion, if an organization disagrees with an interpretation from a 3rd party auditor, the complaint process should NOT be the first line of action. Accredited registrars must have an APPEAL process which by, registrants can argue against what they believe is a bad judgment call by the auditor. And there is an escalation process for the appeal process. Still in my opinion, a complaint should be used for systemic problems, not isolated disagreements.

And, even though I don’t have hard numbers/data, my estimate is that, for the registrar that I work for, we have some kind of appeal about an auditor’s interpretation in about 2% of the audits we perform. And from the appeals process, my estimate is that 50% of the time the nonconformities are overruled. We are continuously trying to “calibrate” our auditors, but until human cloning is commercially viable, we will have to live (and manage) with the fact that auditors do have differing views, sometimes. It is inherent to the process.

 

[RosieA]

One way that registrars are able to reduce their costs is to freelance their audits, which is different than using subcontracted auditors.

Sidney, what is the difference between freelance and subcontracted audits?

 

[Sidney Vianna]

Subcontracted auditors have an ongoing relationship with a Registrar. They perform audits for a registrar frequently, so they can really understand the registrar’s protocols, procedures, processes, people, etc…. Just like in any other profession, you need hands on experience to mature as an auditor.

Freelancing auditors, on the other hand, are those that have no real continual relationship with the registrars. They just satisfy certain minimum requirements, such, as an example having a RABQSA Lead Assessor card. But because they do not work consistently and continually with a registrar, they do not really “represent” the registrar. They represent themselves. There are some people out there moonlighting as auditors.

If 3rd party auditing can be challenging for people that do this, day in day out, for people that work as 3rd party auditor as a “side job” it can be a daunting task. And typically, who pays the price for that lack of experience, is the registrant. Both literally and figuratively.

 

[RosieA]

Caster,
I, too, have had some great experiences. I've worked with 4 Registrars and the one I never had any issues with was BSI. They were wonderful to work with. I was Director of Quality for a small medical devices company, charged with taking us through ISO9001 and the EU MDD. I was new to the industry, and the MDD new also. There was excellent direction, language interpretation, help with finding good web sites, and I never disagreed with a finding. They didn't tell me what to do, but gave me great help in finding resources to improve my understanding.

When we were going through the process of building our first sterile product, the BSI Microbiologist, who did a preassessment of our contract sterilizer with me, was outstanding. I learned more from her in two days than I would from a week long seminar. It remains one of the best cooperative business relationships I've ever had.

So even though I've had a few bad experiences with other Registrars, my BSI relationship serves as a benchmark for all others.

 

[Cari Spears]

One more point. In my opinion, if an organization disagrees with an interpretation from a 3rd party auditor, the complaint process should NOT be the first line of action. Accredited registrars must have an APPEAL process which by, registrants can argue against what they believe is a bad judgment call by the auditor. And there is an escalation process for the appeal process. Still in my opinion, a complaint should be used for systemic problems, not isolated disagreements.

Great post. The "Do not concur" option on the finding was the APPEAL process that I used to disagree with the auditor I mentioned earlier. She probably reviewed the findings and my responses with the technical person at the registrar. Maybe she learned something. I didn't have a problem with any of the other auditors from that registrar either, so no need to complain.

 

[vanputten]

Hello Carl:

It probably takes just as long to post an official compalint to the IAF as the time it took you to post these comments at The Cove.

Also, if we want the conformity assessment industry to improve, on way would be to have a central database of complaints, issues, concerns, so the data can be trended, addressed, etc. Yes, throwing the auditor out on their ear will eventually eliminate bad auditors via business pressures. Also, I beleive the IAF looks over the ANAB (RAB). If the RAB has created a sorry state of affiars, how does kicking the auditor to the curb address the issues created by the RAB? In my opinion, notifying the IAF is more of a systems approach. The IAF can look at the customer perception of the entire industry if we notify them.

Regards, Dirk

 

[Carl Keller]

Dirk,

A "...central database of complaints, issues, concerns,"???

You can't be serious, we can't even get a central database that tells us the number of REGISTRATIONS!

ISO takes no responsibility for implementation of the standard or overseeing the Registration process and keeping the Registrars in line, so we have the RAB, who apparently has such little credibility that we needed the ANAB, now we need the "IAF police" to keep the RAB in line so what's next? Who is going to keep the IAF in line?

Kicking the auditor to the curb doesn't address any of the RAB issues, it isn't meant to. It isn't my job to fix the RAB, it is theirs, and until they do, people will continue to look at them as a complete joke.

Kicking the auditor to the curb DOES send a message to that particular registrar that you are not going to stand for incompetence and/or bullying and allows you to run your business the way you see fit while still within the guidelines of the standard.

The whole thing is a scam and all of these groups have been formed to make profit.

Really, it is as simple as asking for firm evidence of what part of the standard your system does not adhere to. If the registrar can show you, you have to do it, like it or not, if they can't, you don't have to and they need to back off.

Getting back to the original topic, two registrar auditors should NOT be telling you opposing viewpoints on the same topic.

Carl-

 

[Hershal]

"One more point. In my opinion, if an organization disagrees with an interpretation from a 3rd party auditor, the complaint process should NOT be the first line of action"

I believe I was the first in this thread to mention using the complaint mechanism, and I stand by that.....

But to be fair I did state that it should be used AFTER trying to get an auditor to remove/reverse a disputed finding.

Hershal

 

[Paul Simpson]

Off the beaten track

While you make some very valid points, they are a little off track to the original post.

I don't agree. While it may seem that way...

Personally I'm getting tired of having a new person with their own opinions telling us that we should change A to B when the previous auditor said we should change B to A.

Auditors rarely say "change this" they are required to record a non compliance against an element of the standard or a requirement of the company's documented system that something isn't being done.

Ken has auditor A telling him he needs to do it one way, then Auditor B coming in and telling him something 180 degrees off.

The point of my post was (and is) that whether you judge something to be non compliant or not depends on what experience you have leading up to the audit. If you have two auditors (separately) and one company discussing a non compliance you have two communication processes that may or may not work very well. For example. If, as a company you have misunderstood the non compliance (not saying this is the case with Hoosierken - the non compliance may not have been communicated effectively) and take "corrective action" that means you are non compliant - as an auditor I cannot ignore that - I have to raise it as a non compliance - and I should also take it up internally with my colleague.

The bottom line is, if the registrar auditor sticks to what the standard actually states rather than interpretation and implication, there will not be any confusion or need to explain an "understanding of a requirement".

Each management system is an interpretation of the standard. By asking a 3rd party in to assess it you are inviting an interpretation of how the company's interpretation meets the original requirements (and customer requirements, statutory obligations, ....). If there is a mismatch as an auditor I then have to go back to the documented system or the standard and, as clearly as I can, state where the mismatch is. You are right that this should not be an opinion but unfortunately there are lots of grey areas in audit. Fact. My preference is to raise any contentious areas verbally - not as non compliances - and follow up with the company next time.

After all, it isn't really rocket science, it is 14 pages of requirements. If an auditor can't point at one of those 14 pages and say "It says right here you have to do it" then you don't have to do it and signing anything that says you will do it is a dangerous practice in my mind.

It is only 14 pages of requirements - in the same way as the 10 commandments are "only" 10 sentences. There is a vast difference between being a simple document and one that is "easy" if it wasn't so the cove wouldn't exist to discuss all the points in the threads.