Audit Man-days - And Other Charlie Thoughts

[Marc]

From: Charley Scalies <[email protected]>
Subject: Q: Registration Costs /Scalies

I am far too old to embark on crusades and too smart (I think) to tilt at windmills, but I am still cranky enough to make noise when things don't pass the smell test. I raised this issue once before, and would like to have it addressed again, hopefully in more depth, than before.

Registrars, at least those accredited by RAB, have increased the number of auditor person days required to conduct registration audits. At least that is the case with smaller organizations (100 to 200 employees.) When asked, they all cite some accreditor guidelines as being the cause. They claim, "We really have no way to stray from these guidelines except in unusual circumstances."

In the past, 4 person days seemed to be the average. Now it's 6!

Would someone enlighten me as to where these guidelines originated? Are they the emanations of the RAB, alone, or is it wider in scope? What is the pain - to the registrar - if they are not followed? What was the justification? (Not the reason, because I think I can guess that.) Would anyone care to attempt convincing us that auditors really need 6 person days to assess the quality system of a 100-200 person firm? (IMHO if one does, he/she is grossly incompetent - and I mean that in the nicest possible way.)

In a discussion concerning multiple accreditations, Malcolm Bell recently said

> The essence of an accreditation body is that it should be independent,
> non-commercial and non-competitive so that its accreditation decisions will be
> free from commercial influence.

"should be" is the operative word

> National Accreditation Authorities were established to monitor and control
> the work of commercial Certification Bodies (Registrars).

But

> Certification Bodies are, in general, in business to make a profit and they
> might, in theory, cut corners or lower standards in order to acquire a bigger
> market share.

Might they not also collude to with the accreditor to drive registrant costs up?

> Even in my own small "neck of the woods" (New Zealand) it is well
> known that prospective clients often search out the less demanding and/or
> cheaper Certification Bodies if they are just after a fast-track route to an
> ISO 9000 certificate on the wall. And there will always be somebody ready to
> oblige.

Is it any wonder that more and more registrants see that registration really adds no value to product quality? So why shouldn't they buy solely on price? I remember seeing a sign on a Purchasing Agent's wall. It said something like, "The bad taste of poor quality lingers long after the sweetness of low price is gone." But in this case, what quality?

> The National Accreditation Authorities of the world have been moving towards
> an international mutual recognition ageement at glacial speed. To do otherwise
> would be akin to turkeys voting in favour of Christmas.

Are we the turkeys? If so, perhaps we should gobble louder. If we are the customer, perhaps we need to start acting like it.

Cranky ole me.
--
Charles J. Scalies/2000+

 

[Marc]

From: Brian Charles Kohn <[email protected]>
Subject: Re: Q: Registration Costs /Scalies/Kohn

I was a manager for an RAB- and RvA-accredited registrar, before I went back into industry. The rules were very clear. ISO 9001 registration assessments for organizations between 100 and 250 people were always required to be 7 days. Here is the table handcopied right from the accreditation body:

Employees Min Days
0 - 4 2
5 - 9 2.5
10 - 15 3
16 - 19 3.5
20 - 29 3 + 1 for doc review
30 - 59 5 + 1
60 - 99 6 + 1
100 - 249 7 + 1
250 - 499 9 + 1
500 - 999 10 + 2
1000 - 1999 13 + 2
2000 - 3999 15 + 3
4000 - 7999 18 + 3
8000+ 20 + 4

I don't know why this info isn't more widely published. BTW: ISO 9002 would be a little less; the numbers vary from range to range and there isn't a formula.

> Would someone enlighten me as to where these guidelines originated?
> Are they the emanations of the RAB, alone, or is it wider in scope?

Actually, I believe the numbers we used were from the RvA, which preceded the RAB by many years. The RAB clearly defers to the older organization with respect to this issue.

> What is the pain - to the registrar - if they are not followed?

Without justification, the accreditation body can invalidate the registration, and write-up the registrar for a nonconformance. The appropriate remedy, IMHO, would be forcing the registrar to re-do the assessment gratis, for the differential amount of time, but some assessors might take a harder stance. Beyond remedial action, the registrar would have to take timely and effective corrective action to prevent the problem from recuring, or face loss of accreditation.

> What was the justification?

The justification for the table, I assume, is decades of experience conducting effective assessments.

> Would anyone care to attempt convincing us that auditors really need
> 6 person days to assess the quality system of a 100-200 person firm?

Having actually done the job for five years, and darned well, if I say so myself, I feel that the number of days is pretty-much inadequate for the less experienced auditors, and pretty-much a little over-the-top for the most experienced auditors. In other words, on the average, it is dead-on-target.

> (IMHO if one does, he/she is grossly incompetent - and I mean that in the
> nicest possible way.)

You're entitled to your opinion, of course, however, I can only assume that you just don't understand what is involved. Moreover, it amazes me how there are folks in your camp AND folks in the camp that assert that the registration assessment by the registrar is enough to ascertain whether an organization is compliant (and therefore internal audits are not for that purpose.) That so totally amuses me that I can't control myself. Seven days for an organization of 200 people just barely lets an assessor scratch the surface.

> Certification Bodies are, in general, in business to make a profit and they
> might, in theory, cut corners or lower standards in order to acquire a bigger
> market share. Might they not also collude to with the accreditor to drive
> registrant costs up?

>From my own experience as a purchaser of registration services, the reality of the business is that registrars have violated the requirements I quoted above, on many more occasions than any care to admit, in the interest of boosting market share but undercutting competitors who quote per regulations.

> Is it any wonder that more and more registrants see that registration really
> adds no value to product quality? So why shouldn't they buy solely
> on price?

I see no problem with registrants using price as a discriminator. However, that is precisely the reason why:
- accreditation bodies have to set up standards for auditor days and auditor qualifications,
- accreditation bodies have to be meticulous in assuring that registrars are applying the appropriate level of rigor on their clients, and - *purchasers* (i.e., the beneficiaries of certification) must be vigilent in their examination of certificates; i.e., ensuring that the registration is accredited by an accreditation body that applies the rules I've been discussing vigorously.

Brian Charles Kohn [email protected]

 

[Marc]

From: Pat Dey <[email protected]>
Subject: RE: Q: Registration Costs /Scalies/Kohn/Dey

> From: Brian Charles Kohn <[email protected]>
>
> I was a manager for an RAB- and RvA-accredited registrar, before I
> went back into industry. The rules were very clear. ISO 9001
> registration assessments for organizations between 100 and 250 people
> were always required to be 7 days. Here is the table handcopied right
> from the accreditation body:
>
> Employees Min Days
> 0 - 4 2
> 5 - 9 2.5
> 10 - 15 3
> 16 - 19 3.5
> 20 - 29 3 + 1 for doc review
> 30 - 59 5 + 1
> 60 - 99 6 + 1
> 100 - 249 7 + 1
> 250 - 499 9 + 1
> 500 - 999 10 + 2
> 1000 - 1999 13 + 2
> 2000 - 3999 15 + 3
> 4000 - 7999 18 + 3
> 8000+ 20 + 4


The table is non-linear. Why?

There's no reference to number of documents, processes etc to audit. Why?

Isn't it easier and quicker to audit a company which has few processes
followed by lots of people, than one with lots of processes each followed by
a few?

> I don't know why this info isn't more widely published.
> BTW: ISO 9002 would be a little less; the numbers vary from range to
> range and there isn't a formula.
>
> > Would someone enlighten me as to where these guidelines originated?
> > Are they the emanations of the RAB, alone, or is it wider in scope?
>
[..]

> > What was the justification?
>
> The justification for the table, I assume, is decades of experience
> conducting effective assessments.

How was "effective" defined?

> > Would anyone care to attempt convincing us that auditors really need
> > 6 person days to assess the quality system of a 100-200 person firm?
>
> Having actually done the job for five years, and darned well, if I say
> so myself, I feel that the number of days is pretty-much inadequate
> for the less experienced auditors, and pretty-much a little
> over-the-top for the most experienced auditors. In other words, on
> the average, it is dead-on-target.

Assuming the distribution of experience is normal - is it?

> > (IMHO if one does, he/she is grossly incompetent - and I mean that in the
> > nicest possible way.)
>
> You're entitled to your opinion, of course, however, I can only assume
> that you just don't understand what is involved. Moreover, it amazes
> me how there are folks in your camp AND folks in the camp that assert
> that the registration assessment by the registrar is enough to
> ascertain whether an organization is compliant (and therefore internal
> audits are not for that purpose.) That so totally amuses me that I
> can't control myself. Seven days for an organization of 200 people
> just barely lets an assessor scratch the surface.

Right on!! Hence my question about the definition of "effective".

[..]

> I see no problem with registrants using price as a discriminator.
> However, that is precisely the reason why:
> - accreditation bodies have to set up standards for auditor days and
> auditor qualifications,

Everyone in industry has to compete to improve productivity whilst
maintaining quality. Indeed, that's often what "quality" is about. Why
should registrars be immune?

> - accreditation bodies have to be meticulous in assuring that
> registrars are applying the appropriate level of rigor on their
> clients, and
> - *purchasers* (i.e., the beneficiaries of certification) must be
> vigilent in their examination of certificates; i.e., ensuring that the
> registration is accredited by an accreditation body that applies the
> rules I've been discussing vigorously.
>
> Brian Charles Kohn [email protected]

Pat
--
Patrick L Dey
[email protected]

 

[Marc]

From: Charley Scalies <[email protected]>
Subject: Re: Q: Registration Costs /Scalies/Kohn/Scalies

> From: Brian Charles Kohn <[email protected]>
>
> You're entitled to your opinion, of course, however, I can only assume
> that you just don't understand what is involved. Moreover, it amazes
> me how there are folks in your camp AND folks in the camp that assert
> that the registration assessment by the registrar is enough to
> ascertain whether an organization is compliant (and therefore internal
> audits are not for that purpose.) That so totally amuses me that I
> can't control myself. Seven days for an organization of 200 people
> just barely lets an assessor scratch the surface.

Thanks for your input. As always, I respect your opinion. I just don't agree with some of it. To begin with, I didn't know I was in any particular camp. If I am, it certainly is not with those who believe that a registrar's audit determines compliance with standards. It verifies the existence of a system that is in compliance with standards. The registrant must demonstrate compliance and effectiveness on its own: and the registrar verifies that such was accomplished. It is for that very reason I hold that registration adds no value to a quality system.

How about seven days for a 100 person firm? That's more than 1/2 hour per employee. That's an awful lot of scratching by my measure!

Are you telling me that you cannot verify the existence of a system in compliance with ISO900x in less than seven man days? 100 - 200 persons companies simply do not have that many different processes to be audited, certainly at Level 2. Are you auditing Level 3 procedures, too? How much objective evidence do you need to sample in order to verify compliance? Is it your paradigm we are seeing here? Long ago, I learned that if you want to ensure that a design project never hits the factory floor, you should throw more engineers at it. Or if you want to create a traffic jam, put a cop in place of a traffic signal.

Charley

 

[Marc]

From: Charley Scalies <[email protected]>
Subject: Re: Registration Costs /Scalies/Parker/Scalies

> From: Bruce Parker <[email protected]>
> Subject: RE: Registration Costs /Scalies/Parker
>
> I understand that all the Registrars are now bound by a document called
> ISO/IEC Guide 62 - General requirements for bodies operating assessment and
> certification/registration of quality systems, published in 1996. This
> Guide shows the minimum on-site and total days for each of: Initial
> Assessment, Annual Surveillance, and Re-assessment.
> For 100-250 persons the values are: 6 (on-site) and 8 (total) auditor days
> for initial assessment, 2 & 2.5 for annual surveillance, and 3 & 5 for
> re-assessment.
> My organization's initial assessment is in March and we just expanded over
> the boundary from 20-29 persons (2.5 & 4 days) to 30-59 persons (4.5 & 6
> days) so I am eagerly awaiting the outcome and how long it takes!

What was the registrars' quote? How many man days? Does it have the right to increase the number of days because you went over the imaginary line? The audit will "take" as long as they have.

I have been recommending to my customers that they inform quoting registrars that, if they want a shot at the business, they are well advised not to gild the lilly. There's enough competition out there among them that even the "good" ones will listen to reason. After all, these "guidelines" are their rules: not yours.

You are the customer. Act like it. Just say no.

Charley

 

[barb butrym]

The registrar leaves a little lee way to the auditor, in that at Doc review....if one finds there is just a simple process performed by numerous people on 3 shifts, then the day adjustment can be justified, agreed on and changed. The justification must be clearly documented. How to do this, and individual registrar policy varies from one to another. Some won't unless pushed by the client. Some readily make the adjustment, some don't want to bother with the extra paperwork. The requirement is clear. In the past registrars cut days to get a financial edge, now that so many have been cited, they are getting religious about following the rules.

Another rule that is coming into play is the trainee auditor getting audit time. Used to be some registrars would use trainees, call then auditors, charge for their days work, and not pay them. No longer allowed. You cannot fill a required day with a trainees time. Either they meet your auditor requirements or they cannot be counted.

 

[barb butrym]

I know of one registrar that would have 1 lead auditor, and 5 trainees, charge for 6 man days, and pay only the lead auditor. AND do 5 of these in a week. His clients came from his consulting company and from his dirt cheap rates. He is now under investigation. He has "sold" the consulting group, but still has close ties if you know what I mean.

 

[Marc]

Also see this http://elsmar.com/ubb/Forum9/HTML/000014.html

 

[Walt]

I find discussions about governing body requirements for audit duration very interesting. I've been a Lead Auditor in the US Nuclear Industry for 20 years and a
QMS-LA through the RAB for over six years. I primarily perform external audits of suppliers in the Nuclear field. Our program does not put a requirement on how many days an audit must encompass. The requirement is that the audit must cover the companies quality program and the commodities/services supplied. The audits are in great depth and detail and are basically free to the supplier being audited. The supplier generally provides lunch to the audit team and occasionally takes the audit team out to dinner one night. It must be said that the intent of our audits are not to make money, increase market share or to out do the competition. The sole focus and purpose of the Nuclear Audit program is to provide reasonable assurance that the supplier can consistently provide a product that meets the technical and quality requirements of the purchase order/contract. The governing bodies placing audit duration requirements, team size, etc. results in financial issues occurring and eventually chips away at credibility. However in a free enterprise system there are limitations on how these audits can be performed and the users can be assured of the quality and credibility of the audit results. The intent of ISO audits is not for marketing but for assurance of consistent quality to customers. The benefits to marketing are supposed to be an incidental result not the purpose. I don't know the answer but I do understand the issue. This is just another perspective.

 

[Marc]

The audit days requirement is a 'control' feature, but it's silly. At a recent audit requiring 6 man days, by the time they hit the end of the second day (2 fellas so 4 man days) they were essentially 'there'. They played around the next morning for a bit over 2 hours and that was that. Off they went. Want to bet the client paid for the whole day?

Nuclear IS a lot different than ISO (not to mention QS) and the lack of a profit motive is a significant aspect. I agree with your point.