Auditor and Registrar Competence

[Marc]

Subject: Re: ISO Conflict of Interest /../Vianna/Borges/Naish
Date: Tue, 13 Apr 1999 12:36:46 -0600
From: ISO Standards Discussion <JENNEJOHNN@UWSTOUT.EDU>

From: PNaish@aol.com
Subject: Re: ISO Conflict of Interest /../Vianna/Borges/Naish

Ellie's information about BSI and their qualification of auditors and customer feedback is good. But it is not limited to BSI. As a registrar the things Ellie mentioned are the things the registrars are suppose to be doing to keep their cetification through the certification bodies.

I have used TUV Rheinland of North America for a number of audits. I have done some qualifying of registrars and have asked some of the TUV auditors as well as auditors from NSAI and UL about their training. TUV Rheinland has the same focus and does regular training of auditors both inside the company and by sending auditors to training to outside sources to stay in step with other registrars and the auditing community. NSAI indicated their training was primarily in Ireland. I only talked to one UL auditor but was told they also do in house training.

I have only had the opportunity to do a good auditor comparison with TUV where I have been exposed to 5 or 6 auditors. I have however, talked to others who used them and found they had been exposed to auditors other than the ones I had experienced. There was a range of professionalism and value for the dollar. There were a couple that gave you ten dollars worth of value for 2 dollars spent. Then you got five for two and some where you got two for two. But they have most consistently provided good service for what you paid.

I have seen the same thing within every group of individuals I have been in. When I was at Intel Corp. each group had people who put out 150% and those that you got 75% from. In school you have A students who work until midnight studying and those that do an hour so they can skate by with C and D.

In every case we have to let the registrar know. The unforunate side of this is that if it stops at the registrar what we can end up with is the person moves from one registrar to another and we continue to be victims of poor skills.

Like the earlier message, may be we should be communicating the information to the certification body who is listed on the certificate as well. They can be more aware as they do their audits of registrars and watch for internal corrective actions being taken. The registrar is required to address all corrective actions addressed. However, if they see no complaints because the registrar disposes of them the only route is the direct route to the certification body.

I know from talking to a registrar when I had a problem with the response from another registrar that any legitimate complaint sent by you to the certification body has to be addressed by the registrar and is evaluated as a part of the audit of the registrar.

The earlier comment about the company getting what the auditor may have thought they wanted is not out of line with a similar experience I heard of from another company. When the company received a preaudit from one registrar they did not do well. The company thought the registrar was unduly harsh since a sister company had passed their audit with a different resgistrar with the same processes.

So they called the registrar in that did their sister company and had a second pre audit done. while they had made some changes a number of the items considered non confroming remained unchanged. When the second registrar was asked about the differences between the registrars, the auditor said they used to be toughter and more strict in their audits but because the clients complained too much and either took the business elsewhere or threatened to they were much easier on them now.

If this company gets poor audits knowing the first registrar audited strictly to the standard and the second doesn't they are sending an indirect message that they don't want a thorough audit. It is possible this is a similar situation.

Phyllis

[barb butrym]

All audits are different and have many variables, some justified and some not. personality of the auditor comes into play, as it should..... One auditor can say the same as another, but presented differently, get quite a different response. Registrars should, if they don't already (which evidence points most don't), take that presentation of information (ie minor nonconformances and opportunities)and be consistent in how thier auditors approach it. URS of the UK does just that here in the states. Their North east operation, of which I am a part, takes great pride in their client friendly approach. The audits are thorough and professional. The standards, both registrar requirements and ISO/QS xxxx, are strictly adhered to, but no drilling or invasive technicques are allowed. If an issue arises during the audit the client is immediately made aware, absolutely no surprises, and is allowed to correct the situation if he can, and provide evidence of that...no 'gottcha' allowed, more of a 'we have a problem'. If the client presents a case and proves it works, and is not in violation of the standard, then that response is noted, and verified at the next audit, if it works for the company, and meets the intent of the standard then it is allowed. Opportunities for improvement are not show stoppers. We always take into consideration if something is 'value added' or BS, and the effect on the end product or service and the customer. So much is 'auditor dependant' that we try to cut that variation by having each of our auditors go through the same QSLA course (regardless of their previous training) with our trainer of choice, and a internship with our seasoned auditors, prior to sending them out on thier own. They are closely scruitinized to ensure they represent URS, North-east US, they way we want to be portrayed.

I hesitated to respond here as i didn't want to appear as if I were selling the registrar. But as a consultant, I have seen and experienced the pains mentioned above all too often, and when URS asked me to work with them here, I agreed ,only if these terms were met, URS agreed immediately, and have supported all of us here in this endeavor. Their US presence has nearly doubled and is still growing since we started the North-east operation, In addition to consistency in auditors, it has eliminated the cost of international flights. It has been a great incentive for the clients, and the word of mouth "sales" are over the moon. Our customer feedback has been awesome.

OK now off the soap box....Anyone pursuing registration needs to really get to know the registrar and its 'policies/personalities'...talk to the auditors. You need to be comfortable disagreeing with him/her. What do they know about your type of business/clients....will they enforce QS like requirements on a putty knife manufacturer? Do your homework, money is not the endall reason to base your choice on.

It's been too long since someone posted to this, so here's two cents worth.
Barb, you last posted on my father's birthday, so it got my attention.
Barb's point of interviewing cannot be overstressed. Always start out with a long list of potential registrars. Decide what accreditations you want or need. Decide if you want to tolerate large travel expenses. Determine if you must have a "big name" or if you can live with a smaller, but perhaps even more competent name. Deterime the level of registrar expertise in the standard to which you want to register. Does the registrar/auditor(s) have experience in your industry?
After you have narrowed your list to three-to-five desired registrars, request actual auditor visits or at least telephone calls for interview purposes. Determine from that the pervieved competence and perhaps just as important, the "chemistry". Be comfortable with the registrar's appeal process. Is it adequate to satisfy you?
Once you've made your selection and signed the contract, remember your right to request alternate auditors if an experience is not good. And, after all that, remember that in three years you have an opportunity to start over.

[Marc]

I agree and I have had posted (for quite a while now) two things to help:

http://Elsmar.com/pdf_files/ - Look for Registrar_B.pdf and Registrars.pdf

[Randy]

We just got our certificate for ISO 9002. Quite honestly I don't think the auditors we had actually knew the who, what, where, when, how, or why of our operation.

They apparently have been in the business for a while and work for a major Registrar. They definitely knew "their business" and I found them listed on the RAB as certified Lead Auditors. They did identify a couple of "major and minor -D's".

But knowing a little about ISO myself, and knowing our operation since it's beginning, there's no way that I could objectively say "this system conforms to the standard". The system we have is broke, not beyond repair , but still broke. My feelings are echoed by others in the know here too.

[Laura M]

Out of curiosity...when you say a "couple of majors and minors" I would guess that it is a couple of minors, no more that one major.

Along the "our system is broke" line,
I always found it interesting that a major can be identified if the non-conformance resulted in "probable shipment of non-conforming product"
If that's the case, then a supplier with returned Parts per million over zero has a quality system which resulted in not a "Probable shipment", but a 100% chance of shipping N/C material! So obviously, there is discretion here.
Also, if you just got certified, don't dispair. I remember thinking that, with the N/C's identified there was so much more work to do. It's easy from within to know the problems. Hopefully what you have in place can spring board additional improvement. Your registrar should be looking for improvement at each visit.

Replying to Marc's first post...
TUV Management Service has training once an year in Danvers, MA. All the auditors travel out to participate in cross-training which includes outside training, interpretations, audit techniques and critiques, developments in the standards. This is typically a week long affair. This requirement is imposed by our mothership in Germany.
Within the Danvers, MA organization, the goal is also that on an annual basis, 3% of the time is delegated to training.
When I started with TMS about 3.5 years ago I also had to participate (PASS!!!) the TUV Cert Lead Auditor week long course. This was additional training since when I came to TMS, I already had certification as Lead Auditor RAB and IRCA. Let's not forget QS-9000 certification as well.
I will say this - always check and double-check your auditor's and your registrar's credentials.


------------------
Pete Maizitis
Lead Auditor
TUV Management Service

All registrars are required to have a continuing education/training program for auditors. On site evaluation of their performance is conducted on a periodic basis and their audit reports should be reviewed constantly to see if the audits are meeting the "policy" of the registrar. I've heard a lot of talk of inconsistancy between auditors, with one identifying a "major" and another identifying a "minor" while someone else misses it altogether. Properly trained auditors with a consistant management policy will have very little variation from auditor to auditor or audit to audit.
As far as identification of "major" vs. "minor", look at ISO 10011. An auditor is responsible to report all observations of departure from requirements. All observations that indicate a breakdown in a system requirement are to be reported as nonconformities.
The registrar should have a clear policy on how the results of audit should be reported. Our position is that isolated occurrences are "observations" and multiple observations would constitute a "nonconformity". We are getting away from major, minor, and concern. The objective is primarily to find out if the system is working or broken.
Registrars are supposed to stay away from consulting.