How do you Audit Internal Audits?

C

Chris Ford

#21
Thanks for your feedback; It's not just talking about audits; at the management review top management actually discuss the plan, review training needs, assign resources, etc.




We are giving internal audit more visibility through the Management Review.
That's monitoring, and it's a Management Review input requirement to begin with. It's actually the first item on the list of required inputs. So, you're not actually giving your audit program "more" visibility by having management discuss the details - they're supposed to do that anyway.

You're missing the part in 8.2.2 that says you need to audit your quality system. It doesn't say, "audit only those parts that are not discussed in management review".

An audit is quite different than monitoring activities.

Now... if you include the internal audit program on your schedule, then have management audit it together during management review, you'd have a much stronger case to argue. But, then you're still going to need to generate an audit report, still document your nonconformities, and still take corrective action regardless of who you want to call the "auditor".

I think what you're doing could be a very good solution to the tight budget / tight resources issue. When management refuses to budge and won't offer funding for audit staff, make them conduct the audit!

Don't forget to train them as auditors - so you've got your competency issue taken care of...

I'm with Andy on this one though... as it stands, based on what you're saying, you're monitoring the audit program in management review, which is a good thing since it's required under 5.6.2, but you're not auditing your audit program. That would be a NC against 8.2.2 which requires that:

"The organization shall conduct internal audits at planned intervals to determine whether the quality management system
a) conforms to the planned arrangements (see 7.1), to the requirements of this International Standard and to the quality management system requirements established by the organization, and
b) is effectively implemented and maintained."

So, 8.2.2 requires that you audit the entire quality system and 5.6.2 requires that you talk about those audits during management review meetings. Unless you can show that an audit was conducted during the management review meeting, and back that up with an audit report, you haven't met the requirements of 8.2.2. And, if your registrar hasn't seen it yet, my guess is you're probably going to see a NC in the future.
 
Elsmar Forum Sponsor
C

Chris Ford

#22
Hi Andy...

Well, I can certainly see your point. Unfortunately, as our Quality Department has only two individuals in it... and we both have way more work on our plates than we can handle (which is probably true for a vast majority of the folks here), it becomes necessary for us to have individuals from other departments as internal auditors.

I completely agree with your sentiment that training internal auditors is critical to getting good audits. That is why I have an 8 hour class (4 hours on the standard and 4 hours on conducting audits) prior to any internal auditor being allowed to participate in audits. Then, the auditor trainees watch and participate in internal audits with a fully trained and qualified auditor for a minimum of 4 audits. Then, the auditor trainee conducts 4 audits under the supervision of the trained auditor. At that point I am comfortable with the auditor trainee's skills and they are approved as an auditor capable of conducting a process audit on their own.

Some may see this as overkill... some may see this as underkill. I'm comfortable with the process.

Cheers!!

Mike
It sounds like a thorough process to me, Mike. I created an audit training program just like this at a company once. I spent a great deal of time creating the material for it, including powerpoint presentations and multiple written documents. The course included tests, and on-the-job training requirements very similar to yours.

The problem I faced came right back to the budget issues. Most of the people who completed training didn't get passed the OJT requirements before they were assigned other tasks or simply left the company. I found an extremely high turn-over rate in internal auditors as the result.

I think a program like yours is excellent for training internal auditors who have a direct job responsibility for conducting internal audits when the company can't bear the cost of external training or certification. When "audit" is not part of a person's job description, though you're dealing with volunteers. Volunteers are unpredictable. What my program lacked was direct management support - management did not allocate dedicated resources for internal audits. I did manage to get one company to reflect a person's internal audit work on annual performance reviews. That gave people more "points" toward larger bonuses - throw money at it, and they'll always find the time to "volunteer".

In all of the situations I've encountered, I think that a thorough analysis would prove that the company actually spent more money by taking this "band-aid" approach, having other departments volunteer to audit than they would have if they hired a third-party.
 
M

Michael T

#23
The problem I faced came right back to the budget issues. Most of the people who completed training didn't get passed the OJT requirements before they were assigned other tasks or simply left the company. I found an extremely high turn-over rate in internal auditors as the result.

I think a program like yours is excellent for training internal auditors who have a direct job responsibility for conducting internal audits when the company can't bear the cost of external training or certification. When "audit" is not part of a person's job description, though you're dealing with volunteers. Volunteers are unpredictable. What my program lacked was direct management support - management did not allocate dedicated resources for internal audits. I did manage to get one company to reflect a person's internal audit work on annual performance reviews. That gave people more "points" toward larger bonuses - throw money at it, and they'll always find the time to "volunteer".

Amen to that! :biglaugh: Tie it to the paycheck and people line up.

I used to have a high turn-over rate and big problems with supervisors releasing individuals to conduct audits. I stepped out of the QA arena for a couple of years to run Customer Service then into Marketing and the guy they hired to replace me was... hmmmm... not as effective and the QMS started to slide. The management team let him go... pulled me back into the Quality Department and we finally have a Production Manager who knows the benefits of internal audits and started holding supervisors' feet to the fire concerning internal auditors committments. The president of the company held a meeting with all supervisors and area leaders and said that this will happen... no if's, no and's, no but's.

So... :D I've got management committment and no grief about the auditor trainees time out of the department. We also have this tied to performance reviews which has a direct impact on raises and bonuses.

It's all about the money.

Cheers!!

Mike
 
B

Bill Pflanz

#24
I've also found that it is undesirable to have someone from another 'dpartment' do the audit, since they often have only minimal experience of what that dept may actually do. Sure they can 'learn' but you won't get an effective audit!

Actually, I believe that a lot more care and effort has to be put into selection and assignement of auditors than has tradionally been done. Simply choosing someone from another dept. giving them a checklist and sending them off isn't going to get a really good audit result.....
After reading the posts again, I find us to be generally in agreement with some differences in style rather than substance.

I do find it surprising that Andy finds it "undesirable" to have someone from another department do the audit. That is not only common practice in many organizations but should not be an issue if the auditor is properly trained in auditing techniques. I do agree that the training must be more than a checklist. I have never liked checklists but I know some companies use it to make it easier on the auditor and auditees and ensures that needed processes are audited. Checklists can have the disadvantage of not allowing the auditor some discretion in looking at obvious problems not on it.

Bill Pflanz
 
#25
Bill, your comments are appreciated.

My point about folks from other depts. is based on my years of experience where internal auditors have been used as so much 'cannon fodder', having been primed with a training course, packed with a checklist and told 'go audit', and they have little to no idea even what basic blocking and tackling is done in the assigned auditee dept.! Worse, some have a distinct preconceived notion, for example, the 'buyers being on the take from suppliers or we wouldn't keep getting the cr*p we get"......

To witness an audit, for example, of the product engineering process, as performed by an engineer was a moment of true inspiration! Instead of tentatively dancing around the issues, their collaboration on the audit as auditor/auditee was admirable. I doubt many from Manufacturing or Purchasing etc. (or if the roles were reversed, i.e Engineering in Purchasing) would have been as effective.......
 
C

Chris Ford

#26
Amen to that! :biglaugh: Tie it to the paycheck and people line up.


It's all about the money.

Cheers!!

Mike
It sounds like you've got a good situation - management commitment is key and it really mitigates and minimizes the typical headaches you get with "volunteers". I've seen the spectrum - auditors trained, but don't show up for their own audits, or auditors sitting on their notes for weeks on end because they "got busy". I've actually been fortunate enough to never have had a NC written against an audit program. I have had auditors make comments like, "it would probably have been cheaper if you just hired a consultant," though.

In referencing auditor qualifications, Andy mentioned earlier that he felt that more thought needs to go into the selection of auditors in order to achieve the most effective results. I completely agree, and I think in many cases, if not most, companies would reduce costs by hiring independent auditors rather than try to handle it internally. Most companies fail to recognize the tremendous cost savings, because the cost is usually hidden in reallocation of indirect labor personnel.

The internal, hidden costs for creating and maintaining an audit training program for internal auditors can be astronomical. If we make some assumptions, we can actually compare costs.

Assumptions:
Small company with 100 employees, occupying a 60,000 square foot building, and reporting about $80M in sales annually.
Everyone involved is most likely considered indirect labor.
Annual salary for everyone is $60,000 or $29.00 / hour
Company overhead percentage is 220% or $64.00 / hour
Overhead cost is $93.00 / hour

Training Program Development - 8 hour course. Probably need around 24 hours to create it. The cost to develop the course would be approximately $2232.00. If we forecast six trainees for the first year, the cost of development per employee would be $372.

Classroom Training (Instructor) - 8 hour course delivery + about 3 hours prep time for a total of 11 hours or $1023 per course. Let's just assume that over the course of a year, we conduct two sessions for three students each. That's a cost of $341 per student for instructor time.

Classroom Training (Student) - 8 hours classroom time for a total of $744.

OJT Training (Student) - Assuming that the auditor isn't spending extra time training, we can average the time spent on OJT audits at about 4 hours each. If 4 audits is the magic number, that's a total of 16 hours OJT training required or $1488 per student.

The total cost to train 6 people internally would be around $2973 each or about $17,638. Granted, after the first six students over the first year, the cost per student will be reduced to around $2600 each ($15,500), assuming 2 classes per year with 3 students each.

And let's hope that overtime isn't a factor...

At any rate, we've developed the program and trained 6 people. Total investment so far is $17, 638, but they each conduct one audit in that same year for a total of 8 hours each or roughly $4400. Therefore the total cost to train 6 auditors and conduct 6 audits of 6 clauses would be roughly $22,000. If you're auditing the entire QMS... well, do the math.

An independent auditor could audit the entire quality system for less than half that amount.

The following year, the same auditors may not be available, or even working with the company any longer. So, it'll be back to training probably another 6 internal auditors and another $20K investment in the name of autonomy.

Then consider the cost of the risk involved. In Mike's case, it seems he's got a handle on a regular set of volunteer auditors. I think that's rare. But, in most cases, as Andy suggests, how qualified are the internal auditors? How effective is the audit? And how much of a commitment can we expect from volunteers?

All it takes is one costly mistake. The cost of corrective action can be much greater than the cost of conducting an audit. If the internal audit isn't effective for a lack of due diligence on the internal auditors' part, the cost of a NC on a certification audit could be rather shocking. Then, you've got a $25,000 internal audit report that ultimately didn't quite live up to what it was worth.
 
J

JaneB

#27
Some organisations I have worked with don't actually audit their own audit program.

It's the exception, but it can happen (or rather, not happen in this case), and is both justificable and accepted by external audits. In each case, these are/were organisations where top management is intimately aware of the audit program, the audit process, the auditor doing it, and the results. And there would have been no benefit whatsoever in doing such an audit, as there would have been nothing to 'report' to the MD or CEO or Director or whoever that s/he didn't already know and was intimately familiar with.

In quite small organisations, it can be acceptable not to audit internal audit. I'm sure there will be people who disagree with me.

Before you do, the original idea for this came from a question raised by a highly competent and experienced auditor from a very reputable and international certifier. He queried the value to the particular organisation of auditing its audit process (for reasons given above).
 
I

isotexas

#28
Some organisations I have worked with don't actually audit their own audit program.

It's the exception, but it can happen (or rather, not happen in this case), and is both justificable and accepted by external audits. In each case, these are/were organisations where top management is intimately aware of the audit program, the audit process, the auditor doing it, and the results. And there would have been no benefit whatsoever in doing such an audit, as there would have been nothing to 'report' to the MD or CEO or Director or whoever that s/he didn't already know and was intimately familiar with.

In quite small organisations, it can be acceptable not to audit internal audit. I'm sure there will be people who disagree with me.

Before you do, the original idea for this came from a question raised by a highly competent and experienced auditor from a very reputable and international certifier. He queried the value to the particular organisation of auditing its audit process (for reasons given above).

My org is living proof of this, I was the ONLY QA person. I walked my management team through the audit plan, the schedule, the report and the Corrective Actions specific to the audit. They then made the determination of "audit completeness" for the lack of a better term. All activities were documented, including the fact the audit would be audited during Management Review. Our registrar ( top 5 guys) never had a problem with the approach.
 
P

Phiobi

#29
Yep!! That's the way I read it... :agree:

As a member of the Quality Department, neither me nor the QA Manager can audit the internal audit process as we are directly involved in, and responsible for, the maintenance and upkeep of the QMS - part of which are internal audits.

However, my internal auditors are not part of the Quality Department but members of other departments throughout the organization. As such, when they audit the Internal Audit Process, they are not auditing their own work as they don't work for the Quality Department but provide a collateral duty (voluntarily) as Internal Auditors.

BTW... I've never had our Registrar or any other 3rd party auditors find anything amiss with this.

Cheers!!

Mike
It's funny most people agree the way it should be done and it is the way I have always done it, just every year and I bet next year will be no different I pose myself the same question....
 
E

Eloy Gomez

#30
The Quality department IA audits are coming up next month. Since Quality cannot audit their process we asked for volunteers from other departments to no avail. The Quality Manager suggested using document control to audit Internal audits and Calibration and use Calibration personnel to audit Receiving Inspection and RCCA. Well, you guys get the point. Is there a problem with this? To use various QA personnel to perform all of our QA audits as long as they do not audit their own department, yet we all still fall under the umbrella of Quality Assurance.
 
Thread starter Similar threads Forum Replies Date
lanley liao How to understand this words that the planning of internal audit shall take into consideration the results of previous audits? Oil and Gas Industry Standards and Regulations 10
O ISO13485 implementation - Are internal audits expected before stage 1 audit? Design and Development of Products and Processes 3
S Internal audit discrepancy - We missed a few audits that were scheduled Internal Auditing 12
G Non Conformance During ISO 9001 Audit - Not All Internal Audits Completed General Auditing Discussions 19
E Informational Internal Audits - Wear multiple hats what can and can't I audit (so I'm not auditing my own work) Internal Auditing 149
F API Spec Q1 9th Edition Surveillance Audit - Questions about internal audits. Oil and Gas Industry Standards and Regulations 22
J Internal Audit clarification - How to perform the audits IATF 16949 - Automotive Quality Systems Standard 6
K No Internal Audits For Upcoming IATF Trans Audit IATF 16949 - Automotive Quality Systems Standard 5
J Internal Audits - Closing Audit Deficiency Reports (ISO 13485) Internal Auditing 4
S Is Audit Plan / Agenda required for Internal Audits? Internal Auditing 2
C ISO 9001:2008 Surveillance Audit - No Internal Audits Internal Auditing 9
dubrizo Internal Audit Value - What is the point of conducting internal audits to a checklist Internal Auditing 40
O New Job 1 Month from Recertification Audit - Missing Documents, no Internal Audits ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 25
N Internal Process Audits - 7.1 Planning - How do YOU audit it? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
F Changing the Frequency of Internal Audits - ISO/TS 16949 Internal Audit Requirements Internal Auditing 5
Q Reducing the Frequency of Audits - Internal Audit Programme 2012 Internal Auditing 13
K Internal Audits - Necessary before Stage 1 Audit? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 42
G Internal Audit Planning - We are not getting the best out of our Internal Audits ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
K ISO 13485 Audit Finding on Regulatory Issues - Internal Audits ISO 13485:2016 - Medical Device Quality Management Systems 8
S Internal Audits required before Registration Audit - Distributors ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
ScottK Using Customer Audits as part of the Internal Audit Plan for ISO9001 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
M Internal Audits - Audit Departments or Processes Internal Auditing 13
M Two situations where it is appropriate to add further Hazard Audits to Internal Audit Occupational Health & Safety Management Standards 8
E Internal Audits - Presenting Audit Findings to Upper Management General Auditing Discussions 19
P No Time to Audit? Becoming "leaner" - How do you make time for Internal Audits? Internal Auditing 36
E First external audit coming up, do I need to have performed Internal Audits already? Internal Auditing 15
C New to performing internal audits - Looking for audit questions Internal Auditing 5
A Audit Database that captures internal, external audits - Prime Part 21 manufacturer Quality Assurance and Compliance Software Tools and Solutions 12
Casana How to document Internal Audits done via the "Process Audit" methodology Internal Auditing 9
S Internal Audits not performed - Useful data from internal audit schedule Internal Auditing 31
B Internal audits 3 year plan? Internal audit frequency Internal Auditing 23
Y Internal Audits of All Facilities World Wide Before the Certification Audit Internal Auditing 9
D Internal Audits and Nonconformances - Clause 8.2.2 Internal Audit (paragraph 4) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
W Internal Audit Schedule - Seeking Examples - Quality Audits Internal Auditing 14
Crusader Your Internal Audit Team: Internal or Hired External? Outsourcing Internal Audits Internal Auditing 100
L Internal Audit Schedule - Frequency of internal audits Internal Auditing 9
C Changing Internal Audit Program - Regional Manager Online Audits Internal Auditing 2
C Who audits the internal audit system? Internal Auditing 32
I Seeking: Internal Audit Training Material - Practice Audits? Internal Auditing 14
Gman2 ISO 9001:2000 Internal Audit: Someone PLEASE find me the "Shall" on "Process Audits" Process Audits and Layered Process Audits 37
Gman2 Internal Audits - Who Can Audit Whom? Internal Auditing 7
Gman2 ISO 9001 Internal Audit Checklists and Process Audits Process Audits and Layered Process Audits 31
A ISO 14001 Clause 4.5.4 - Internal Audit - Must a company perform internal audits? ISO 14001:2015 Specific Discussions 5
A Management Review and Internal Audits Before Registration Audit Management Review Meetings and related Processes 2
Raffy The Role of the Process Audit in ISO 9000 Internal Audits Process Audits and Layered Process Audits 22
Raffy Internal Audit vs. Customer Audits - Can we count Customer Audits as IAs? Internal Auditing 25
J Self Audits - Internal Audits - Area supervisors to audit their own area Internal Auditing 22
C Internal Audit Schedule - Documentation requirements for Internal Audits Internal Auditing 10
A Internal Audits - Can we to perform audit just using process maps? Process Maps, Process Mapping and Turtle Diagrams 19
B Our QS9000 audit team has been using checklists to perform internal audits Internal Auditing 37

Similar threads

Top Bottom