Measuring Effectiveness of Internal Audits - Reporting in Management Review Meeting

[Toni Herrera]

Hi All!
This is my first visit to the site and the forum. I feel as though I've struck gold!!

I have a question. I have been performing internal audits for three years and typically I report at management reviews which audits were performed, which audits were past due on the schedule, and which random audits were performed due to a customer complaint, major breakdown, or rework/scrap costs.
I tell them how many corrective actions were issued and which ISO element had the most non-compliances.

Now, we are looking at "measuring the effectiveness" of the internal audit process. Am I way off base in believing that the measurements we are looking for will be found in the corrective action follow ups?

 

[Jimmy Olson]

Hi Toni,

Welcome to the cove. It's the best resource you can have (in my opinion).

I would agree with you in using corrective actions to measure audit effectiveness. That seems like it would at least be a good starting point.

I will keep my answer short since I am still new to this. I am sure you will get responses from plenty of knowledgable people.

 

[NITIN]

You could present-
Number of disputed audits
Feedback of auditees
Reaction time analysis to close non conformances over different time periods
Process improvement findings

 

[Claes Gefvenberg]

Re: Measuring Internal Audits

Toni Herrera said:

Hi All!
This is my first visit to the site and the forum. I feel as though I've struck gold!!

I think you have, Toni. That was my feeling as well when I first found the Cove, and I still consider it a gold mine. Welcome aboard .

Ok.... Over to your question:

Concerning the audits themselves, I do an annual evaluation looking at among other things:

Planned vs carried out audits.
The number of new audit findings vs closed ones during the year.
Average time from registration of a finding to completed action.

I also do a weekly follow up on:

Total no. of "active" issues.
No. of delayed actions (Yes, we have them, I'm sorry to say)
No. of actions due within a month ( Early warning to avoid the above).

/Claes

 

[Anton Ovsianko]

Claes Gefvenberg said:

Concerning the audits themselves, I do an annual evaluation looking at among other things:

Planned vs carried out audits.
The number of new audit findings vs closed ones during the year.
Average time from registration of a finding to completed action.

I would point out Claes, that IMHO only the first one measures auditing effectiveness. The rest are concerned with corrective/preventive action. Actions are neither necessarily developed, nor performed by internal auditors. So, it is just another system being measured.

To my mind NITIN presented a more relevant list of indicators:

NITIN said:

You could present-
Number of disputed audits
Feedback of auditees
Reaction time analysis to close non conformances over different time periods
Process improvement findings

Yours,

Anton

 

[M Greenaway]

In order to consider the effectiveness of internal audits I would have thought that we need to think about the purpose of internal audits.

The 'trouble' with auditing is that it is not an exact science, as such counting numbers of non-conformances found by clause or area may not give any real insight into the effectiveness of the audit process.

 

[Anton Ovsianko]

True Mr. Greenaway,

I would rather seek for some intricate indicators showing somehow the ratio between the non-conformances found and non-conformances taking place. Clear that you will never be sure that you've found ALL the NC's.
So you will have to judge by indirect results like customer complaints and other satisfaction/dissatisfaction indicators.

We use:
- number of audits performed vs. planned
- number of NC's found vs. previous periods corrected through general indicators of quality in the area audited (like customer complaints, defect rate dynamics, cost dynamics etc.)
- comprehensiveness of audit (judging by audit reports)

You see, the indicators are qualitative rather than quantitative.

Regards,

Anton

 

[Claes Gefvenberg]

Anton Ovsianko said:

I would point out Claes, that IMHO only the first one measures auditing effectiveness. The rest are concerned with corrective/preventive action. Actions are neither necessarily developed, nor performed by internal auditors. So, it is just another system being measured.

To my mind NITIN presented a more relevant list of indicators:

Yours,

Anton

On the other hand the very reason for doing an audit in the first place is that we want improvement. I consider the actions to correct audit findings to be part of the internal audit process.

Toni asked how to measure the effectivness of the internal audit process. If we assemble a heap of findings and no action is taken, I wouldn't call it effective.

/Claes

 

[Anton Ovsianko]

Cleas,

I would say that internal audit is a part of continuous improvement process, which also includes corrective and preventive action and so on. We can as well measure effectiveness of internal audit process separately from the effectiveness of corrective action and preventive action. Of cource, we should also measure their total effectiveness as a unite system.

MHO

Anton

 

[db]

A distant measure

A distant measure I use is to compare internal findings against external findings. This could be dangerous because it tempts us into changing our audits to match the registrars. We shouldn't do that. If used properly, it can show if our auditors can find things. For example, if the internal audit finds nothing and the external finds 15 majors, then we need to find out why there is such a large difference. That goes also for if the internal finds 15 and the external finds nothing.

Some of the things already mentioned are better measure. That is why I said a 'distant' measure. The important thing is to look at YOUR system and decide what measures YOU feel will show if YOUR internal audit program is working!

Warning, this place is addictive! :thedeal: