Average number of Nonconformances during a surveillance assessment

[Jim Wynne]

I have my big girl underwear on...

[old joke]
Me too, ever since my wife found them in the glove compartment. [/old joke]

 

[Helmut Jilling]

I voted "over 8" because for the last 2-3 years (heck, maybe longer than that - I've only been here just less than 2 yrs), we've had double-digit minor NCs and several majors. Not an ideal situation by any stretch of the imagination, I know. However, we have worked very hard to improve the QMS and we remain hopeful that this year's audit will be the best we've had in several years.

BTW, we always have the same lead auditor - not necessarily a good thing all the time....

Do you find they are the same types of issues each time, or are they always different? Can you give us some idea as to what kinds of issues?

 

[Helmut Jilling]

Obviously, the number of NC's does not tell the whole story. But there is anecdotal evidence that contracted auditors (vs. full time employee auditors) write fewer NC's, compared to the fully employed counterparts. The primary reason is the fact that auditors realize that, if they upset the paying customers, they might not be welcomed back. Which means loss of revenue for the contracted auditors. Back in the early and mid 90's, some US based CB's used to boast that they only employed full time auditors. There is a reason for that. Due to the competitive pressure of the certification market, I believe that most large CB's have been forced to adequate their workforce.
Don't get me wrong. I am not saying that all contracted auditors are bad and all full time auditors are good. Not at all. I personally know several contract auditors that are extremely good at what they do. But being black listed from a client has obvious different consequence$ for the two categories. After all, contrary to what many believe, the pocket is the most sensitive part of the human anatomy.

I am a contract auditor and frequently audit with a pool of about 8-10 contract auditors. I rarely recall any conversations along these lines. We don't over do or pull nc's for financial reasons. Shame on any auditors who do. Our group is always discussing what is right for the client and the standard. I have been released by clients who did not approve of findings, and knew they were going to do it, but it is their loss.

 

[GStough]

Do you find they are the same types of issues each time, or are they always different? Can you give us some idea as to what kinds of issues?

In comparing the last 2 audits and the findings, I'd have to say that what they find is usually similar. For example, when training records are reviewed, it has been our rotten luck that the files they request just happen to have a little something wrong or something that they have a concern about, such as missing signatures on the training documentation or a couple of records were missing ISO 9001 training documentation.

Another area where we have been hit hard is regulatory (EC) and CE marking. The individual findings may be a little different, but still within the same area (reg/CE/EC). There may be other examples, but these are the first to come to mind.

I think that one of the disadvantages of retaining the same auditor for an extended period of time (we've had the same one since initial registration about 6 or 7 years ago) is that he/she can be so focused on past deficiencies that he/she often has trouble seeing all the improvements that have been made. He/she has become so accustomed to finding NCs that it becomes the expectation. I am in no way implying that all lead auditors do this, but it is certainly a reality in at least our case.

Feel free to PM me if you'd like to continue the discussion.

 

[Helmut Jilling]

In comparing the last 2 audits and the findings, I'd have to say that what they find is usually similar. For example, when training records are reviewed, it has been our rotten luck that the files they request just happen to have a little something wrong or something that they have a concern about, such as missing signatures on the training documentation or a couple of records were missing ISO 9001 training documentation.

Another area where we have been hit hard is regulatory (EC) and CE marking. The individual findings may be a little different, but still within the same area (reg/CE/EC). There may be other examples, but these are the first to come to mind.

I think that one of the disadvantages of retaining the same auditor for an extended period of time (we've had the same one since initial registration about 6 or 7 years ago) is that he/she can be so focused on past deficiencies that he/she often has trouble seeing all the improvements that have been made. He/she has become so accustomed to finding NCs that it becomes the expectation. I am in no way implying that all lead auditors do this, but it is certainly a reality in at least our case.

Feel free to PM me if you'd like to continue the discussion.

Little repeat things are often caused by carelessness. For example, is the HR clerical person reviewing the records before filing them? Is the form clear as to what needs to be signed and where?

If the ones they request contain these errors, it is likely others do also. Do an internal review of the files, and clean them up.

I have some clients where I have been assigned for some time, and some are newer. I think that part is up the the individual auditor.

 

[mlthompson]

I guess I have found that audit results are somewhat based on the individual auditor. I cut my QA teeth in the aerospace industry before the days of NADCAP and AS9100 so each customer had their own audit team. We had an average of 40-80 audits a years. The scruntiny of the various auditors was quite vast. I dealt with auditors on one side of the scale who asked all the right questions and took verbatim what you said without checking even one document as evidence. On the other other side of the scale, the auditors left you crying in their wake. I even had one auditor come in, did a plant tour, and at the end of the tour had a list of audit nonconformances. He didn't say a thing during the audit, but strangely enough the findings were legit. I am glad that accreditation bodies attempt to level out the playing field. Addtionally I feel that there was some audit politics involved where the auditors had to find nonconformances in order to justify their jobs.

Sometimes I imagine that the assignment of an ISO auditor is a crapshoot. If your lucky you'll get one the is fair and reasonable. If your not so lucky, you'll get one that scruntizes beyond the point of being value added to the client and their customer base.

In regards to contract vs staff auditors I only have a couple of ISO auditors to compare. My previous auditor was a contract auditor. He worked with us more and was more forgiving on the assignment of audit nonconformances. Within five years of certification, Corporate Mgmt rated the ISO system has the highest rated competency in the Manufacturing Group. My current ISO auditor, who is a staff auditor, is a bit more scruntinzing. It appears that this person scruntizes, in my opinion, beyond being value added. Sometimes I wonder if this person is in fear of being found at fault during a witness audit.

 

[Sidney Vianna]

I guess I have found that audit results are somewhat based on the individual auditor.

You are correct. There is quite a wide variation in auditor's performance. The same system being assessed by different auditors can be rated from marginal to best-in-class. The providers of assessment services should be doing a much better job to "calibrate" their assessors. Until we are able to "clone" Dolly-the-Auditor, there will be auditor variation to be dealt with.

Sometimes I imagine that the assignment of an ISO auditor is a crapshoot. If your lucky you'll get one the is fair and reasonable. If your not so lucky, you'll get one that scruntizes beyond the point of being value added to the client and their customer base.

Well, remember that you can request to interview potential auditors that would be assigned to your account, you can ask for references. It does not to have to be a crapshoot. If you do due diligence, you can minimize the chances of an auditor incompatibility being found during the course of the audit. Further, if you do find out that your assigned auditor is not compatible with your organization, you can always request a re-assignment.

Because matching an auditor and the "auditees" is so critical, I was thinking of developing an auditor match making website: the www.iso-harmorny.com.

 

[GStough]

Well, we just came through our annual surveillance audit and actually did better than last year. However, we still received double-digit minor ncs (only 2 majors), and quite a few OFIs. After this experience, I have to agree with others that an individual auditor's take on things varies, because that was certainly the case this year.

 

[Barahir]

We've not had any surveillance audits yet for TS, having completed our upgrade audit just over 1 month ago. At that, our auditor found 4 minor NC's and about the same number of OFI's.

When we were QS (this company first went QS in October '98, I started a week before our first surveillance audit in October '99 and then we moved to every 6 months until October 2005 which was our last QS audit), the number of NC's gradually declined as time went on. In 1999 and 2000 we were probably getting 4 or 5 per audit, and by 2004 and 2005 we would get 1 (and on one lone occasion, none).

 

[Helmut Jilling]

You are correct. There is quite a wide variation in auditor's performance. The same system being assessed by different auditors can be rated from marginal to best-in-class. The providers of assessment services should be doing a much better job to "calibrate" their assessors. Until we are able to "clone" Dolly-the-Auditor, there will be auditor variation to be dealt with.

Well, remember that you can request to interview potential auditors that would be assigned to your account, you can ask for references. It does not to have to be a crapshoot. If you do due diligence, you can minimize the chances of an auditor incompatibility being found during the course of the audit. Further, if you do find out that your assigned auditor is not compatible with your organization, you can always request a re-assignment.

Because matching an auditor and the "auditees" is so critical, I was thinking of developing an auditor match making website: the www.iso-harmorny.com.

Sidney, I agree. I have met a lot of auditors who weren't "ripe" yet. I think registrars should have a longer mentoring period where newer, inexperienced auditors work as coauditors until they truly are ready to be Lead Auditors. A 3 day class is just not the ticket, if you want good results.