External Auditor asking for other Audit Findings

[Paul Simpson]

I can't agree that it's a lazy auditor's way, Boris! You and I both know, auditors should be looking for the actions taken after audit nc's to be effective.

Then we'll have to agree to disagree. I stand by my 'lazy' tag because I have seen it and never seen any good aspects to it. In the rest of my explanation I listed the other aspects of assessing the effectiveness of the audit programme and one of those was the close out of actions. Now if the 3rd arty auditor has raised NCs in a particular area and then (at the end of their programme) find that internal audits have raised the same thing and the NCs have been closed out then s/he would be entitled to either raise another NC on audit / corrective action or escalate their own finding to a Major - as there is evidence this is a systematic problem. The issue is with using internal / other audits to guide your own audit plan.

We both know, previous audit findings should be factored into 'this audit', especially since we know that corrective actions are often subject to the "Hawthorne Effect".

Here we disagree again. The 3rd party should use their own audit results to define the plan / programme but not the auditee's. Now if there is evidence the audit process isn't working effectively then they will concentrate on that at next visit.

That degree of bias should be observable as a competency (or lack thereof), and not confused as an effective auditor trait, following up on previous result...

Not really sure of your point here, Andy.

 

[AndyN]

Surely, Boris, if it were a significant fault, wouldn't there be some guidance in ISO 19011 or, as far as CBs go, something to the effect in ISO/IEC 17021? When I was auditing, we weren't allowed to audit the 'sensitive' stuff until last. However, many CBs and their auditors start off with management review, corrective actions and internal audits etc. What I call 'dumpster diving' - if the audit isn't aware of the bias, I agree it can have a negative effect, however, I'd think that a competent auditor would factor the observations into their audit and NOT be biased by them...

 

[Big Jim]

Then we'll have to agree to disagree. I stand by my 'lazy' tag because I have seen it and never seen any good aspects to it. In the rest of my explanation I listed the other aspects of assessing the effectiveness of the audit programme and one of those was the close out of actions. Now if the 3rd arty auditor has raised NCs in a particular area and then (at the end of their programme) find that internal audits have raised the same thing and the NCs have been closed out then s/he would be entitled to either raise another NC on audit / corrective action or escalate their own finding to a Major - as there is evidence this is a systematic problem. The issue is with using internal / other audits to guide your own audit plan.
Here we disagree again. The 3rd party should use their own audit results to define the plan / programme but not the auditee's. Now if there is evidence the audit process isn't working effectively then they will concentrate on that at next visit.

Not really sure of your point here, Andy.

Andy and I may disagree on a lot of things, but we're solidly together here.

Auditor's need to have the opportunity to probe any QMS weaknesses. Not only is he not being lazy if he doesn't look at internal audit findings, he is not being diligent if he doesn't.

Any audit pertaining to the QMS is fair game, including internal audits, customer audits, and regulatory body audits as far as they effect the QMS.

In my opinion though, QMS auditors should not be prying around in the financials. Even though we carry the title of auditor, few of us have the education, training, or experience to be there.

Added in edit. I believe the flavor of access to other QMS audits is found in the requirements of management review where management review needs to include results of audit (5.6.2 a). The results of audits isn't limited to internal audits or even to just CB audits, it would include ANY QMS audit.

 

[Paul Simpson]

Surely, Boris, if it were a significant fault, wouldn't there be some guidance in ISO 19011 or, as far as CBs go, something to the effect in ISO/IEC 17021? When I was auditing, we weren't allowed to audit the 'sensitive' stuff until last. However, many CBs and their auditors start off with management review, corrective actions and internal audits etc. What I call 'dumpster diving' - if the audit isn't aware of the bias, I agree it can have a negative effect, however, I'd think that a competent auditor would factor the observations into their audit and NOT be biased by them...

Now 19011 is specifically required to exclude 3rd party audits (wrong IMHO but there you are) and I have long despaired of getting anything useful out of 17021 but both standards do talk about avoiding bias and, in my answer to Bonhomme (what a good man he is! ). I tried to explain where the bias can come from. So my recommendation (for the avoidance of bias) is the auditor takes his / her own sample and leaves internal audits to last. It also means they have to do the job they've been paid to do rather then regurgitating the orgs own internal audit results back to itself.

I also think the 'dumpster items' you mention are all largely end of audit topics, relating as they do to the Check / Act part of the cycle.

Andy and I may disagree on a lot of things, but we're solidly together here.

... and you and I agree on next to nothing but that's OK.

Auditor's need to have the opportunity to probe any QMS weaknesses. Not only is he not being lazy if he doesn't look at internal audit findings, he is not being diligent if he doesn't.

... nobody said that. To save you looking back at the thread I answered Bonhomme's question of why looking at internal audits early on could bias an auditor's sample. Period.

Any audit pertaining to the QMS is fair game, including internal audits, customer audits, and regulatory body audits as far as they effect the QMS.

Really? What does your CB contract with the auditee say? From memory there is no requirement in 9001 that you open up customer audits (for example) for 3rd party scrutiny. If the auditee chooses to put the results through their corrective action procedure (as many do) then they're fair game. Again not in Bonhomme's original question.

In my opinion though, QMS auditors should not be prying around in the financials. Even though we carry the title of auditor, few of us have the education, training, or experience to be there.

Agreed.

Added in edit. I believe the flavor of access to other QMS audits is found in the requirements of management review where management review needs to include results of audit (5.6.2 a). The results of audits isn't limited to internal audits or even to just CB audits, it would include ANY QMS audit.

You have stretched the requirements beyond the standard but again if customer audits are summarized for management review then fine ... to be looked at at the end of the 3rd party audit.

 

[Big Jim]

Now 19011 is specifically required to exclude 3rd party audits (wrong IMHO but there you are) and I have long despaired of getting anything useful out of 17021 but both standards do talk about avoiding bias and, in my answer to Bonhomme (what a good man he is! ). I tried to explain where the bias can come from. So my recommendation (for the avoidance of bias) is the auditor takes his / her own sample and leaves internal audits to last. It also means they have to do the job they've been paid to do rather then regurgitating the orgs own internal audit results back to itself.

I also think the 'dumpster items' you mention are all largely end of audit topics, relating as they do to the Check / Act part of the cycle.
... and you and I agree on next to nothing but that's OK.

... nobody said that. To save you looking back at the thread I answered Bonhomme's question of why looking at internal audits early on could bias an auditor's sample. Period.

Really? What does your CB contract with the auditee say? From memory there is no requirement in 9001 that you open up customer audits (for example) for 3rd party scrutiny. If the auditee chooses to put the results through their corrective action procedure (as many do) then they're fair game. Again not in Bonhomme's original question.

Agreed.

You have stretched the requirements beyond the standard but again if customer audits are summarized for management review then fine ... to be looked at at the end of the 3rd party audit.

Most of us seem to look at management review near the beginning of the audit. Why would you wait until the end to discover a weakness you should have probed earlier?

Please explain this concept about how looking at the performance of the QMS creates a bias. I don't see it. Instead I see an excuse for a lack of due dilligence.

 

[JaneB]

Jim,
You might want to look at some of the considerable research that has been done over many years that supports Boris' viewpoint, including self-fulfilling prophecies, Pygmalion effect, etc etc. There's a wealth of research on how people can be, and are swayed and influenced in ways that cause them to behave in biassed ways even when they absolutely don't think they are. Why would (or could) auditors be considered to be not so.
OK, you prefer to do it early on. You have reasons for that. I think there's a very strong case for not doing so, but coming back to see if what you found - completely independently - has also been found before by other auditor/s, including internal. But to ssay that doing it this way you see 'an excuse for a lack of due diligence' is strong language and neither reasonable or justifiable.

Certainly not the case with the many, many fine and professional third party auditors I have seen do it in that order.

 

[Big Jim]

Jim,
You might want to look at some of the considerable research that has been done over many years that supports Boris' viewpoint, including self-fulfilling prophecies, Pygmalion effect, etc etc. There's a wealth of research on how people can be, and are swayed and influenced in ways that cause them to behave in biassed ways even when they absolutely don't think they are. Why would (or could) auditors be considered to be not so.
OK, you prefer to do it early on. You have reasons for that. I think there's a very strong case for not doing so, but coming back to see if what you found - completely independently - has also been found before by other auditor/s, including internal. But to ssay that doing it this way you see 'an excuse for a lack of due diligence' is strong language and neither reasonable or justifiable.

Certainly not the case with the many, many fine and professional third party auditors I have seen do it in that order.

I'm very familiar with those topics, but I truly do not see how that would create a negative effect when performing an audit.

If improvements have been made and they earlier NCRs have been effectively resolved that is only a positive thing.

We must not forget what is written in 8.2.2 about performing internal audits.

"An audit program shall be planned, taking into consideration the status and importance of the processes and areas to be audited, AS WELL AS THE RESULTS OF PREVIOUS AUDITS."

Emphasis mine.

Somehow this should be ignored when preparing for and performing a 3rd party audit? I don't think so.

The 3rd party audit is to evaluate the entirety of the QMS. How can you do that if you don't look over the internal audit? If you look over the internal audit, how can you ignore the findings and how they are resolved.

How can you say that doing our job creates a bias?

 

[JaneB]

Jim,
You perhaps didn't read or understand what I said. And then setting up a straw man "Jane said we shouldn't look at the results of internal results as part of an external audit" and attacking that.

False premise. And has nothing to do with what I actually wrote.

 

[Big Jim]

Jim,
You perhaps didn't read or understand what I said. And then setting up a straw man "Jane said we shouldn't look at the results of internal results as part of an external audit" and attacking that.

False premise. And has nothing to do with what I actually wrote.

Jane,

Please re-read what you wrote. Perhaps you ment something different than I see on that page.

What is you position on the topic? In you opinion, should CB auditors not look at internal audit findings?

 

[JaneB]

Very much

Jim, of course a CB auditor will, would and should examine relevant internal audits (by which I mean audits that are relevant to the scope of the 9001 certification). They could not recommend certification without doing so, because of the requirements stipulated in clause 8.2.2 (which I'm sure you're as aware of as I am). I don't understand why you would think that this question was even up for debate.


What was under debate (at least in my understanding and following the sequence of the discussion) was the timing of this - early (your preference, as I already said) or late in the audit. That's what I was debating. All clear now?