Internal Auditor Productivity Metric?

Elsmar Forum Sponsor
#12
Howdy y'all.

I'm trying to develop an overall productivity metric for internal auditing. It's intended as a barometer, not a super-precise, dictatorial, gotta-meet-the numbers metric. And, I don't want any auditors auditing "to hit the numbers". Productivity to me means something like: ouput / resources (or cost). Doing more with the same resources or doing the same with fewer resources. And I don't want anything like # of findings or recommendations per audit. I'm talking about AUDIT PROGRAM productivity, not results of audits. There's a bit of a wrench thrown in due to audit variability, the planning of audit time, the fact that all of the auditors are part-time, etc., i.e. sometimes output changes correlates directly with input change. Any kind of ROI analysis would be done separately at this time. Our schedule is planned around audits assigned per quarter. Get it done in a quarter and you are good to go.

Any ideas welcomed!

Thanks!
My recommendation - don't waste your time! There are much bigger fish to fry and more relevant activities than putting (meaningless) metrics to internal audits, quite frankly. There's no number you can put to findings etc., so all you are left with is requirements like turning in reports on time etc. Do you really have that many auditors doing that many audits that you need to have a measurement for this part of the audit process?

As for 'get it done in a quarter' - you mean as long as an auditor can get their assigned audit done in 3 MONTHS??:mg: I don't see the sense in that! You seriously give an auditor a 3 month window to do an audit? WOW! Tell me more, you've peaked my interest:confused:
 
S

split-cane

#13
Annual plan used to be a rolling 12-month window built off an Access database that increased frequency if the "conclusion" was bad or worse. That wasn't scalable and writing the report was tough (small boxes on screen) and output wasn't as slick as wanted for audience. Anyway, we went to an .xls. At the same time we transitioned to more of a "pull" system where Mgmt could request "custom" audits or additional audits in pre-determined areas/processes. All of our auditors are part-time with other jobs at the company. Plus, auditees, though cooperative on balance, aren't always around. These last two have gotten progressively "worse" as we have become global (USA, Europe, China, South America, South Africa, etc.). Then you don't know for sure when there will be demand for a "pull" audit. To accommodate all this, auditors basically have a set of audits assigned by quarter. When they get it done is up to them to manage, so long as it is sometime that quarter.
 

Randy

Super Moderator
#14
Andy is absolutely correct, you're wasting time and in many of the "metrics" proposed defeating and actually working against the purpose of the audit...."to verify conformance & effectiveness". If I saw that you were focusing on non-conformance I'd issue you one for that reason alone because it is contrary to the requirements of 9-14-18-16949-13485-and nearly every other standard out there requiring audits to be performed.

If absolutely must measure something measure preventive actions/opportunities for improvement your auditors find because that would be a proactive, value-added measure of productivity.
 

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#15
Thank you, thank you, Randy, Andy and Sidney, for your comments. Being a full time internal auditor for safety, quality, environmental and calibration standards, the only quantifiable metric I will even subscribe to is doing audits on time. But those metrics can be subject to scheduling too few, or too many audits in a quarter.

It's more complicated to use qualitative assessment of audit success, but it can be done IF there is a before-and-after assessment of how an audited process is performing.

If my audit revealed that we are doing a practice out of sheer tradition and it saved money to eliminate the practice, then my audit metric is that of being a team member - not the improvement's owner - of the improvement process.

To sum it up, audits are a support function. Our support of beneficial change can be measured, but that should be about the limit of it.
:2cents:
 

Jim Wynne

Staff member
Admin
#16
Howdy y'all.

I'm trying to develop an overall productivity metric for internal auditing. It's intended as a barometer, not a super-precise, dictatorial, gotta-meet-the numbers metric. And, I don't want any auditors auditing "to hit the numbers". Productivity to me means something like: ouput / resources (or cost). Doing more with the same resources or doing the same with fewer resources. And I don't want anything like # of findings or recommendations per audit. I'm talking about AUDIT PROGRAM productivity, not results of audits. There's a bit of a wrench thrown in due to audit variability, the planning of audit time, the fact that all of the auditors are part-time, etc., i.e. sometimes output changes correlates directly with input change. Any kind of ROI analysis would be done separately at this time. Our schedule is planned around audits assigned per quarter. Get it done in a quarter and you are good to go.

Any ideas welcomed!

Thanks!
Peter Drucker is credited with saying "There is nothing so useless as doing efficiently that which shouldn't be done at all." The idea can be extended to measurement--trying to measure things that can't be measured, or don't need to be. If you're struggling with what and how to measure, there's a good chance that meaningful measurement is impossible or unnecessary.

The internal audit process should be evaluated, but that doesn't mean measurement by specious metrics. What do you expect of the process? If you know that, you know how to evaluate. If you don't know that, you can't evaluate. I think that an IA process should produce opportunities for improvement and confirmation that processes are operating as designed. If your IA process does those things, you're finished with evaluation. If it doesn't, you know what needs to be remedied.

Don't get caught in the damnable trap that's common to many Six Sigma programs--that you must be able to measure everything in numbers. It ain't true.
 
Last edited:
#17
Very nice post, Jim (that's 'British' for 'great', BTW)

Probably a (the) most important part of an audit is evaluating what the auditors found compared to what was expected they'd find. If you use audits to diagnose if the quality system is (or is not) performing to plan, then auditors' reports should indicate evidence that supports process performance - and conformance.

An audit, in and of itself, is a monitoring activity of/for processes and shouldn't be subjected to measurement itself.
 
Last edited:
S

split-cane

#18
Well, thanks to everyone for their thoughtful responses. Your answers pretty much confirm my own conclusions in attempting to think this through prior to posting here. Still, I can be a bonehead (just ask my wife) and wondered if there was something obvious I was overlooking. I might still mess around a bit with some concepts. If they work, I'll post them. I'm glad I asked the question and thanks again.
 

Randy

Super Moderator
#19
Well, thanks to everyone for their thoughtful responses. Your answers pretty much confirm my own conclusions in attempting to think this through prior to posting here. Still, I can be a bonehead (just ask my wife) and wondered if there was something obvious I was overlooking. I might still mess around a bit with some concepts. If they work, I'll post them. I'm glad I asked the question and thanks again.
Just use the standard and follow what it tells you the audit must achieve...

From 9001:2008

The organization shall conduct internal audits at planned intervals to determine whether the quality management system

a) conforms to the planned arrangements (see 7.1), to the requirements of this International Standard and to the quality management system requirements established by the organization, and

b) is effectively implemented and maintained.


If you must use a metric then consider whether or not your auditors are able to meet these requirements during the audit within the planned amount of time that they are given to do so.
 

v9991

Trusted Information Resource
#20
i wouldn't agree with the proposition that 'metrics/measuring' internal-audits is not usefull; and suggest the following procedure for measuring the effectiveness of internal-audits.

one should compare the outcome of following few areas against the outcome of internal-audits.
1. opportunity areas for improvement measured against the implemented points
2. major/critical observations which are addressed and measured against
2.1 action plan/investigations concluding that the given process had lead to problems; OR
2.2 measured against the deviations/complaints in the given area. OR
2.the observations in the external audit

other aspects of measurement such as adherence to plan/schedule, closure of audit observations could be certainly relevant; but, when we speak on the 'effectiveness' of procedures, [not just compliance to requirements ] then, we need to evaluate the procedure in tandem with the associated processes.

hope this helps.
valiveti.
 
Thread starter Similar threads Forum Replies Date
J IATF 16949 Internal Audit question - Auditor's responsibility Internal Auditing 6
V Internal Auditor Competency KPI IATF 16949 - Automotive Quality Systems Standard 14
B Internal Auditor Competency - Product Auditors Internal Auditing 9
U Internal Auditor not trained but done Audit for some process Nonconformance and Corrective Action 5
M Tips on preparing for IATF 16949 Internal Lead Auditor exam Manufacturing and Related Processes 1
S ISO 13485 Lead Auditor - Debate between our Quality Team and Regulatory Auditor - Internal Auditor Training ISO 13485:2016 - Medical Device Quality Management Systems 17
B Internal and external auditor competency to CSR's IATF 16949 - Automotive Quality Systems Standard 20
D Impartiality of Internal Auditor ISO 9001/13485 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 18
M IATF 16949 7.2.3 Internal Auditor Competency - Trainer's competency Internal Auditing 7
C Recommendations for UK-based ISO 13485 internal auditor training ISO 13485:2016 - Medical Device Quality Management Systems 1
A External Auditor issue with Internal Audits Internal Auditing 7
Q Internal Auditor competence for ISO 14001 ISO 14001:2015 Specific Discussions 11
S IATF 16949: Is "Certified" Internal Auditor mandatory? IATF 16949 - Automotive Quality Systems Standard 9
S Internal Auditing for API Spec Q1 - auditor qualification requirements Oil and Gas Industry Standards and Regulations 6
J Your opinion on the better training org for IATF16949 Internal auditor and Lead Auditor IATF 16949 - Automotive Quality Systems Standard 3
Q Internal Auditor Training requirements for ISO 14001:2015 ISO 14001:2015 Specific Discussions 5
E Informational I would like to get some Internal Auditor Training Internal Auditing 31
Q What is the difference between normal and licensed internal auditor? VDA Standards - Germany's Automotive Standards 9
J IATF 16949 CAR - Internal Auditor Requirements IATF 16949 - Automotive Quality Systems Standard 15
A ISO 13485 - Internal Auditor Independence and Process Owners ISO 13485:2016 - Medical Device Quality Management Systems 3
Q AS 9100D internal auditor training requirements Internal Auditing 16
A Other Discussion Boards - Internal Auditor Coffee Break and Water Cooler Discussions 5
E Online training for an internal auditor to AS9120B AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
Q Internal Auditor Trainer's Requirement Internal Auditing 4
D IATF 16949 - Internal Auditor Training/Certification Requirements Training - Internal, External, Online and Distance Learning 2
R Internal Auditor vs. Public Accountant - Accounting Student Needing Help Career and Occupation Discussions 4
C IATF 16949 Cl. - 7.2.3 Internal Auditor Competency IATF 16949 - Automotive Quality Systems Standard 1
C IATF 16949:2016 Cl. 7.2.3 Internal Auditor Competency Requirements IATF 16949 - Automotive Quality Systems Standard 39
Q IATF 16949 Cl. 7.2.3 - Internal Auditor Competency and Records IATF 16949 - Automotive Quality Systems Standard 5
R Internal Auditor auditing Internal Audit Procedure (AS9100) Internal Auditing 18
Q How can an Internal Auditor Trainer's Competency be Evaluated - IATF 16949 Cl. 7.2.3 IATF 16949 - Automotive Quality Systems Standard 10
H ISO/IATF Internal Auditor Recommendations For Manufacturer In California, USA Internal Auditing 3
Casana 2nd Party and Internal Auditor Qualifications & Training IATF 16949 - Automotive Quality Systems Standard 13
P Problem with IATF 16949 Clause 7.2.3 Requirements (Internal Auditor Competency) IATF 16949 - Automotive Quality Systems Standard 3
B Competency of Trainer for Internal Auditor Training (IATF 16949) IATF 16949 - Automotive Quality Systems Standard 11
C ISO 13485:2016 - Internal Auditor Training vs. Lead Auditor Training Training - Internal, External, Online and Distance Learning 4
J Dinged on Internal Audits for supervising an auditor I was training Internal Auditing 10
M Internal Auditor --> Licence needed? Internal Auditing 6
V Internal Auditor Training Documents and PPT Document Control Systems, Procedures, Forms and Templates 5
V Internal Auditor Training Documents and PPT Internal Auditing 3
V Certified Internal auditor is necessary? ISO 9001 requirement? Quality Management System (QMS) Manuals 4
D Can the Plant Manager be our ISO Representative and an Internal Auditor? Internal Auditing 26
M ISO 13485 Audit Questions - Internal Auditor Training and other Requirements ISO 13485:2016 - Medical Device Quality Management Systems 10
M Questions about AS9100 Internal Auditor Training AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 14
R Can a external auditor raise a finding that is already identified in Internal Audit ? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
P Can internal ISO 17025 Auditor conduct ISO 9001 Audit ISO 17025 related Discussions 7
P Internal Auditor Training - Audit exercises Internal Auditing 5
D Training Material for Internal Auditor Requirements Training - Internal, External, Online and Distance Learning 4
U TS16949 Internal Auditor Training - Is On-line training OK? Training - Internal, External, Online and Distance Learning 1
F Internal Audit quiz - Auditor Qualifications and Requirements Internal Auditing 34

Similar threads

Top Bottom