Measuring Effectiveness of Internal Audits - Reporting in Management Review Meeting

T

Toni Herrera

#1
Hi All!
This is my first visit to the site and the forum. I feel as though I've struck gold!!

I have a question. I have been performing internal audits for three years and typically I report at management reviews which audits were performed, which audits were past due on the schedule, and which random audits were performed due to a customer complaint, major breakdown, or rework/scrap costs.
I tell them how many corrective actions were issued and which ISO element had the most non-compliances.

Now, we are looking at "measuring the effectiveness" of the internal audit process. Am I way off base in believing that the measurements we are looking for will be found in the corrective action follow ups?
 
Elsmar Forum Sponsor
J

Jimmy Olson

#2
Hi Toni,

Welcome to the cove. It's the best resource you can have (in my opinion).

I would agree with you in using corrective actions to measure audit effectiveness. That seems like it would at least be a good starting point.

I will keep my answer short since I am still new to this. I am sure you will get responses from plenty of knowledgable people. :p
 
N

NITIN

#3
You could present-
Number of disputed audits
Feedback of auditees
Reaction time analysis to close non conformances over different time periods
Process improvement findings
 
#4
Re: Measuring Internal Audits

Toni Herrera said:

Hi All!
This is my first visit to the site and the forum. I feel as though I've struck gold!!
I think you have, Toni. That was my feeling as well when I first found the Cove, and I still consider it a gold mine. Welcome aboard :D .

Ok.... Over to your question:

Concerning the audits themselves, I do an annual evaluation looking at among other things:

Planned vs carried out audits.
The number of new audit findings vs closed ones during the year.
Average time from registration of a finding to completed action.

I also do a weekly follow up on:

Total no. of "active" issues.
No. of delayed actions (Yes, we have them, I'm sorry to say)
No. of actions due within a month ( Early warning to avoid the above).

/Claes
 
A

Anton Ovsianko

#5
Claes Gefvenberg said:

Concerning the audits themselves, I do an annual evaluation looking at among other things:

Planned vs carried out audits.
The number of new audit findings vs closed ones during the year.
Average time from registration of a finding to completed action.
I would point out Claes, that IMHO only the first one measures auditing effectiveness. The rest are concerned with corrective/preventive action. Actions are neither necessarily developed, nor performed by internal auditors. So, it is just another system being measured.

To my mind NITIN presented a more relevant list of indicators:

NITIN said:

You could present-
Number of disputed audits
Feedback of auditees
Reaction time analysis to close non conformances over different time periods
Process improvement findings

Yours,

Anton :p
 
M

M Greenaway

#6
In order to consider the effectiveness of internal audits I would have thought that we need to think about the purpose of internal audits.

The 'trouble' with auditing is that it is not an exact science, as such counting numbers of non-conformances found by clause or area may not give any real insight into the effectiveness of the audit process.
 
A

Anton Ovsianko

#7
True Mr. Greenaway,

I would rather seek for some intricate indicators showing somehow the ratio between the non-conformances found and non-conformances taking place. Clear that you will never be sure that you've found ALL the NC's.
So you will have to judge by indirect results like customer complaints and other satisfaction/dissatisfaction indicators.

We use:
- number of audits performed vs. planned
- number of NC's found vs. previous periods corrected through general indicators of quality in the area audited (like customer complaints, defect rate dynamics, cost dynamics etc.)
- comprehensiveness of audit (judging by audit reports)

You see, the indicators are qualitative rather than quantitative.

Regards,

Anton
 
#8
Anton Ovsianko said:

I would point out Claes, that IMHO only the first one measures auditing effectiveness. The rest are concerned with corrective/preventive action. Actions are neither necessarily developed, nor performed by internal auditors. So, it is just another system being measured.

To my mind NITIN presented a more relevant list of indicators:

Yours,

Anton :p
On the other hand the very reason for doing an audit in the first place is that we want improvement. I consider the actions to correct audit findings to be part of the internal audit process.

Toni asked how to measure the effectivness of the internal audit process. If we assemble a heap of findings and no action is taken, I wouldn't call it effective.

/Claes
 
A

Anton Ovsianko

#9
Cleas,

I would say that internal audit is a part of continuous improvement process, which also includes corrective and preventive action and so on. We can as well measure effectiveness of internal audit process separately from the effectiveness of corrective action and preventive action. Of cource, we should also measure their total effectiveness as a unite system.

MHO

Anton
 
#10
A distant measure

A distant measure I use is to compare internal findings against external findings. This could be dangerous because it tempts us into changing our audits to match the registrars. We shouldn't do that. If used properly, it can show if our auditors can find things. For example, if the internal audit finds nothing and the external finds 15 majors, then we need to find out why there is such a large difference. That goes also for if the internal finds 15 and the external finds nothing.

Some of the things already mentioned are better measure. That is why I said a 'distant' measure. The important thing is to look at YOUR system and decide what measures YOU feel will show if YOUR internal audit program is working!

Warning, this place is addictive! :thedeal:
 
Thread starter Similar threads Forum Replies Date
A Internal Audit Tools and Measuring their Effectiveness Internal Auditing 4
M Effectiveness Measuring and Efficiency Measuring In ISO 9001:2015 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
S Measuring OEE (Overall Equipment Effectiveness) for each Job, not for Machine Lean in Manufacturing and Service Industries 2
P Measuring Process Effectiveness - Measures of Effectiveness (MOE's) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 29
R Control Chart Metrics/KPI - Measuring Effectiveness of Control Charting System Statistical Analysis Tools, Techniques and SPC 4
Q Measuring Effectiveness in Containment Action or in Corrective Action? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
E Measuring Receiving Inspection Process Effectiveness AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
K Measuring the Effectiveness of an organization's QMS (Quality Managment System) IATF 16949 - Automotive Quality Systems Standard 11
R Measuring Effectiveness of Quoting Department Quality Manager and Management Related Issues 23
M Measuring Corrective and Preventive Action System Effectiveness Nonconformance and Corrective Action 32
Raffy How do I measure Measuring Training Effectiveness? Performance evaluation? Other ISO and International Standards and European Regulations 26
Marc ISO 9001 - Measuring Effectiveness of the QMS System ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 16
S Measuring Training Effectiveness - How do you measure Effectiveness? Training - Internal, External, Online and Distance Learning 50
R ULM Measuring Pressures General Measurement Device and Calibration Topics 1
M Missing measuring equipment General Measurement Device and Calibration Topics 6
Marco Bernardi MMC & LMC modifiers and CMM measuring techinques like diameter least squares and circularity or minimum/maximim diameter. Calibration and Metrology Software and Hardware 2
Kuldeep Singh Measuring Function MEDDEV 2.1/5 relation with MDD 93/42/EEC CE Marking (Conformité Européene) / CB Scheme 1
M Measuring FIM (TIR) - Two inside diameters - Conflicting readings between inspectors Manufacturing and Related Processes 1
ScottK Measuring thread flank angle on a screw Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
D ASTM C1064 Temperature Measuring Device General Measurement Device and Calibration Topics 2
B Measuring and monitoring equipment - Understanding which procedures to be compliant with ISO 13485 ISO 13485:2016 - Medical Device Quality Management Systems 6
C ISO 9001:2015 Monitoring and measuring resources. Application a service industry ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
V Ammeter calibration - Measuring head (on pic.) problem General Measurement Device and Calibration Topics 1
D Monitoring and measuring resources - Example of how section ISO 9001 7.1.5 would apply to a manufacture of software ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
A CE mark - Measuring Instruments Directive confusion! CE Marking (Conformité Européene) / CB Scheme 0
M Measuring Capability of Process with Multiple Specifications Capability, Accuracy and Stability - Processes, Machines, etc. 6
E ISO 13485 7.6 Control of monitoring and measuring equipment - Assess the Validity ISO 13485:2016 - Medical Device Quality Management Systems 4
D Measuring fine tubing (< 0.100") "Wall by weight" General Measurement Device and Calibration Topics 11
G Posting Measuring Equipment Accuracy for User Information General Measurement Device and Calibration Topics 4
S ISO 9001 7.1.5.2 - Actions Taken When Measuring Equipment is Found to be Unfit for Use General Measurement Device and Calibration Topics 9
G ISO 9001 and ISO 17025 - Measuring tools not in calibration system General Measurement Device and Calibration Topics 3
S Calibration - Isolated current meter - Measuring patient auxiliary leakage current Calibration and Metrology Software and Hardware 4
G Measuring customer parts on a CMM - How many decimals to report to the customer? ISO 17025 related Discussions 28
qualprod Complying with ISO 9001:2015 - 9.1.1 b - Monitoring, measuring, analyzing and evaluation ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
Tagin Measuring supplier quality with small quantities Statistical Analysis Tools, Techniques and SPC 10
G Uncertainty of staging a short line scale standard on longer measuring machine? Measurement Uncertainty (MU) 4
U Measuring a Golf Club Face/Grip/Shaft Rotation General Measurement Device and Calibration Topics 7
Q Acceptable calibration accuracy of a 60" linear measuring device General Measurement Device and Calibration Topics 16
H Measuring Design Process Design and Development of Products and Processes 4
C AS9100D 7.1.5.2 / Calibration of Employee Owned Measuring Equipment AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
johnny johnson New measuring equipment selection (Cg/Cgk) Capability, Accuracy and Stability - Processes, Machines, etc. 2
S Cleanrooms - Particle Measuring - ISO 14644 Other Medical Device and Orthopedic Related Topics 3
S IOQ protocol for a Micro-Vu measuring system - Is a PQ required? Qualification and Validation (including 21 CFR Part 11) 2
K ISO 13485:2016 Cl. 4.2.3 - Determine QMS Processes, Monitoring, Measuring, etc. ISO 13485:2016 - Medical Device Quality Management Systems 4
L Radius Measuring Tools General Measurement Device and Calibration Topics 4
Proud Liberal Measuring axial and radial surface finish (16 Rz) on a broached internal radius Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
J IATF 16949 Cl. 7.1.5.2.1 - Gauges (Measuring Jigs) Calibration/Verification Records IATF 16949 - Automotive Quality Systems Standard 6
Claes Gefvenberg Worst measuring equipment ever? Coffee Break and Water Cooler Discussions 6
J Measuring an Untrue Radius - Metal Stamped and Formed Parts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
A Calibration Interval - AS9100 - Control of Monitoring and Measuring Equipment Calibration Frequency (Interval) 8

Similar threads

Top Bottom