Measuring Effectiveness of Internal Audits - Reporting in Management Review Meeting

Hi All!
This is my first visit to the site and the forum. I feel as though I've struck gold!!

I have a question. I have been performing internal audits for three years and typically I report at management reviews which audits were performed, which audits were past due on the schedule, and which random audits were performed due to a customer complaint, major breakdown, or rework/scrap costs.
I tell them how many corrective actions were issued and which ISO element had the most non-compliances.

Now, we are looking at "measuring the effectiveness" of the internal audit process. Am I way off base in believing that the measurements we are looking for will be found in the corrective action follow ups?

[Jimmy Olson]

Hi Toni,

Welcome to the cove. It's the best resource you can have (in my opinion).

I would agree with you in using corrective actions to measure audit effectiveness. That seems like it would at least be a good starting point.

I will keep my answer short since I am still new to this. I am sure you will get responses from plenty of knowledgable people.

[NITIN]

You could present-
Number of disputed audits
Feedback of auditees
Reaction time analysis to close non conformances over different time periods
Process improvement findings

[Claes Gefvenberg]

Toni Herrera said:

Hi All!
This is my first visit to the site and the forum. I feel as though I've struck gold!!

I think you have, Toni. That was my feeling as well when I first found the Cove, and I still consider it a gold mine. Welcome aboard .

Ok.... Over to your question:

Concerning the audits themselves, I do an annual evaluation looking at among other things:

Planned vs carried out audits.
The number of new audit findings vs closed ones during the year.
Average time from registration of a finding to completed action.

I also do a weekly follow up on:

Total no. of "active" issues.
No. of delayed actions (Yes, we have them, I'm sorry to say)
No. of actions due within a month ( Early warning to avoid the above).

/Claes

Claes Gefvenberg said:

Concerning the audits themselves, I do an annual evaluation looking at among other things:

Planned vs carried out audits.
The number of new audit findings vs closed ones during the year.
Average time from registration of a finding to completed action.

I would point out Claes, that IMHO only the first one measures auditing effectiveness. The rest are concerned with corrective/preventive action. Actions are neither necessarily developed, nor performed by internal auditors. So, it is just another system being measured.

To my mind NITIN presented a more relevant list of indicators:

NITIN said:

You could present-
Number of disputed audits
Feedback of auditees
Reaction time analysis to close non conformances over different time periods
Process improvement findings

Yours,

Anton

[M Greenaway]

In order to consider the effectiveness of internal audits I would have thought that we need to think about the purpose of internal audits.

The 'trouble' with auditing is that it is not an exact science, as such counting numbers of non-conformances found by clause or area may not give any real insight into the effectiveness of the audit process.

True Mr. Greenaway,

I would rather seek for some intricate indicators showing somehow the ratio between the non-conformances found and non-conformances taking place. Clear that you will never be sure that you've found ALL the NC's.
So you will have to judge by indirect results like customer complaints and other satisfaction/dissatisfaction indicators.

We use:
- number of audits performed vs. planned
- number of NC's found vs. previous periods corrected through general indicators of quality in the area audited (like customer complaints, defect rate dynamics, cost dynamics etc.)
- comprehensiveness of audit (judging by audit reports)

You see, the indicators are qualitative rather than quantitative.

Regards,

Anton

[Claes Gefvenberg]

Anton Ovsianko said:

I would point out Claes, that IMHO only the first one measures auditing effectiveness. The rest are concerned with corrective/preventive action. Actions are neither necessarily developed, nor performed by internal auditors. So, it is just another system being measured.

To my mind NITIN presented a more relevant list of indicators:

Yours,

Anton

On the other hand the very reason for doing an audit in the first place is that we want improvement. I consider the actions to correct audit findings to be part of the internal audit process.

Toni asked how to measure the effectivness of the internal audit process. If we assemble a heap of findings and no action is taken, I wouldn't call it effective.

/Claes