When is the best time to Audit the Internal Audit Process?

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#11
Re: When is the best time to audit internal audit process?

Can you please describe what an enough audit of the audit process looks like?
At a minimum (keeping in mind we are registered to TS16949), the following are expected to be included where I am:

4.1f Are requirements in specifications met (sample 2 – 3 specification in the area)?
4.2.3d Are current issues of specifications available in the area?
4.2.4 Are there documented procedures for identifying, collecting, indexing, filing, storing, accessing, maintaining, and disposing of audit records?
4.2.4 Are records complete, legible, readily retrievable & protected from damage or loss?
5.3 Does everyone know the quality policy and how it relates to their job?
5.6.2 Are audit reports submitted to management for review?
8.2.2 Are audits conducted at planned intervals?
8.2.2 Do audits look at compliance AND effectiveness?
8.2.2 Are results of previous audits reviewed and taken into account during the audit?
8.2.2 Are auditors selected to ensure they are independent of the area being audited?
8.2.2 Is there a documented procedure for the audit program including responsibilities and requirements for planning, conducting, reporting results and maintaining records?
8.2.2 Does the management responsible for the audited area ensure that actions are taken without undue delay?
8.2.2 Do follow-up activities include the verification of the actions taken?
8.2.2.1 Is the quality management system audited to verify compliance with TS-16949?
8.2.2.2 Is each manufacturing process audited?
8.5.2-8.5.3 Do Corrective Action Procedures include:
a) Investigating and recording the cause of noconformities?
b) Determining corrective actions to eliminate the cause?
c) Ensuring that the corrective action is taken and is effective?
8.2.2.4 Do audits cover all quality management processes, activities and shifts, and are scheduled according to an annual plan?
8.2.2.4 When internal/external non-conformities or customer complaints occur, the audit frequency is appropriately increased.
8.2.2.4 Are specific checklists used for each audit.
8.2.2.5 Do all personnel have current training and/or certifications ?
7.5.1 (To be asked before closing the audit) Is there anything in your work area that makes it difficult/impossible to do your job (suitable work environment)
 
Elsmar Forum Sponsor
K

Ka Pilo

#12
Hi Jennifer,

I enjoy your posts, they are very thoughtful.

My next question –

Which is better, it might be best audited while the audit is actually occurring or audit the internal audit process at the end of an audit cycle?
 
#13
Re: When is the best time to audit internal audit process?

It’s not really just auditing audit records or audit files. As we all know, records is one way to tell or show how a process is conducted and if it is conducted properly. Observing and interviewing auditors are also another way of checking if audit was done correctly or on time. In my experience, external auditors just check records to check if the internal audit process was done.

Observing another auditor at work I think is more of a preparation or training for new auditors or applicable for external auditors. In the case of our current CB, they have observers during their audit. Or sometimes they have auditors from the accreditation body to verify if they are complying with the requirements as auditors from a certifying body.
I believe you are confusing the various activities. Auditing the records is auditing the output, not the process of audits. Sure you can observe an auditor - why is it only during training that you'd do this? What the CB auditor does is not a place to emulate them - we know that it's costly for them to audit an auditor once qualified - that's what sets a good CB from a cheap one! Internally, we should fall into the same type of trap. Things change, internal auditors 'go off' task etc. Mission creep I think it's called.

There is much to be observed when actually seeing what another auditor does. Looking at records only doesn't tell you HOW they arrived at the result - what they missed, if it was low hanging fruit, whether they had a good relationship with the auditee etc and more...
 
K

Ka Pilo

#14
Re: When is the best time to audit internal audit process?

Auditing the records is auditing the output, not the process of audits.
Nice point.
There is much to be observed when actually seeing what another auditor does. Looking at records only doesn't tell you HOW they arrived at the result - what they missed, if it was low hanging fruit, whether they had a good relationship with the auditee etc and more...
I agree, but I think auditing while the audit is actually occurring is not easy to put into practice considering a lot of factors, including the schedule etc. Any tips?

Thank you for your thoughtful response to my case.
 

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#15
Hi Jennifer,

I enjoy your posts, they are very thoughtful.

My next question –

Which is better, it might be best audited while the audit is actually occurring or audit the internal audit process at the end of an audit cycle?
I do not support the idea of watching an audit in progress to audit the internal audit process. Such an activity would be suitable for assessing readiness of an auditor-in-training to conduct an audit unattended, but audits do not run to a process in the same way as, for example heat treatment and quenching process of a steel plate. Each audit can be different from the last, and unless the auditor is being a bully during interviews such an assessment would be mostly based on preference of the observer. When asked how I determine effectiveness, how I get to the "heart of the matter" or how I establish a rapport with the auditee I would be happy to describe it in detail. I will offer examples and show how our tracking logs have recorded results of changing how we do certain things to be more effective in auditing.

A second reason for my not supporting the notion is how hard it is to avoid any change in behavior in auditor or auditee. I will explain. While I was working in the school I sat through a class during which the teacher was being assessed by the Assistant Principle. I was in awe: he was an almost completely different guy. His lesson, its delivery and interaction with the students were flawless. It was like he'd been abducted by aliens and replaced with a copy! I watched the students' reaction to all this too - it was subtle, but they clearly knew they were on display. In other words, the result was affected by the experiment itself.
 
Last edited:
#16
Re: When is the best time to audit internal audit process?

Nice point.

I agree, but I think auditing while the audit is actually occurring is not easy to put into practice considering a lot of factors, including the schedule etc. Any tips?

Thank you for your thoughtful response to my case.
What do you consider factors to make this difficult? I can address each specific item, rather than generalities.
 
#17
I'd suggest that, in the majority of cases, Jennifer, your experience is not really applicable. I'd fully agree that for professional auditors, many have the skill to 'clean up' their act for an Accreditation Audit or whatever. It used to happen when I sold fine British cars and the manufacturers rep came to see how the dealer handled the sales/service processes. We were all on our best behaviour, schooled in the criteria etc.

But for most internal auditors, it's basic blocking and tackling we're looking for - asking the right questions, seeking evidence, confirming, not overlooking things, keeping to scope - how can you tell that without seeing them in action?

As we know from many posts here, some auditors rarely get a chance to do many audits and develop the skills that can be picked up with someone observing them. Let's also not forget the 4 Phases of Adult learning which can also affect even seasoned auditors...
 

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#18
I'd suggest that, in the majority of cases, Jennifer, your experience is not really applicable. I'd fully agree that for professional auditors, many have the skill to 'clean up' their act for an Accreditation Audit or whatever. It used to happen when I sold fine British cars and the manufacturers rep came to see how the dealer handled the sales/service processes. We were all on our best behaviour, schooled in the criteria etc.

But for most internal auditors, it's basic blocking and tackling we're looking for - asking the right questions, seeking evidence, confirming, not overlooking things, keeping to scope - how can you tell that without seeing them in action?

As we know from many posts here, some auditors rarely get a chance to do many audits and develop the skills that can be picked up with someone observing them. Let's also not forget the 4 Phases of Adult learning which can also affect even seasoned auditors...
These are good points. I have always been a full time auditor. And it's true that most telephone help desks I call have some beeping message informing me "This call may be recorded for quality control purposes." Hair stylists are observed while they perform processes in order to get their licenses.

And so, to me it makes sense to go about this as casually as possible. It makes sense for the manager to have enough ongoing presence to not raise a sense of unease; to instead have discussions with the auditors and when the subject arises of schedule, say "Oh can I come with you on that audit?" Immediately after the audit interview, it makes sense to ask, "So what now? Are you satisfied, or did you get a whiff of something you want to look into more?" or some such. One could ask leading questions and discuss ways to pursue this-or-that to improve the audit's effectiveness.
 
T

treesei

#19
"Audit an internal audit during its process"...

We should be careful using words because every and each of them has defined meanings. What is the definition of the word audit?

From Cambridge Dictionary, an audit is: “An official examination of the accounts of a business.”

Regardless of the word's financial origin, ISO 19011:2002 defines an audit as “systematic, independent and documented process for obtaining audit evidence (3.3) and evaluating it objectively to determine the extent to which the audit criteria (3.2) are fulfilled.”

I propose that to "systematic"ally audit the internal audit program, include the audit of IA in the IA cycle as an independent IA using the same methodology for other IA's. Frequency is the same as other QMS areas. For example, if your calibration gets audited once a year, do the same for IA audit. As part of the audit, one or multiple sessions may be arranged to watch an internal audit in process. However, the scope of the internal audit of IA program must cover the whole cycle in every audit aspects.

I don't think watching an in-process internal audit satisfies an audit of the internal audit program. It can be part of an audit but does not equal to an audit.
 
#20
I can't agree Treesei:

If you consider the 'status and importance' of the audit process, you wouldn't have to audit the whole thing at all! Why not audit just what's new/changed/not working?
 
Thread starter Similar threads Forum Replies Date
G Internal Audit Planning - We are not getting the best out of our Internal Audits ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
J The Best Way to Make an Internal Audit Checklist Internal Auditing 30
I What kind of wine best complements the Friday that you close out your external audit findings? Opinions are welcome. Coffee Break and Water Cooler Discussions 12
G Best Practices for IT auditing - Is a session-id necessary for a complete audit trail? IEC 27001 - Information Security Management Systems (ISMS) 0
N Best practices for capturing audit objective evidence in a practical manner? Internal Auditing 3
W Audit Protocol Manual which describes best practice Auditing Standards General Auditing Discussions 1
M What is the best way to ensure retraining when changes occur? ISO 9001 Audit ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 18
A Best Way to Audit Customer Feedbacks Customer Complaints 3
O What is the best approach for Audit Result Ranking? General Auditing Discussions 4
J Best Audit Approach for IT Department with sub-departments ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
M What is the best method to audit AS9100 clause 7.2.1 and 7.2.2 AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
M Guidelines in writing an audit report and determing the best auditor General Auditing Discussions 11
P DFMEA - Machinery Design Best Practices FMEA and Control Plans 0
G Best way to calc uncert for measuring microscope without duplicating repeatability? Measurement Uncertainty (MU) 2
William55401 Distributed By Product - Best Practices for Configuration Management and Purchasing Controls ISO 13485:2016 - Medical Device Quality Management Systems 0
P Best european location to set up for a virtual medical device manufacturer? EU Medical Device Regulations 4
optomist1 Best Optical Comparators General Measurement Device and Calibration Topics 9
qualprod Best practice to ensure inputting of data in production Lean in Manufacturing and Service Industries 19
M Best profilometer for the money General Measurement Device and Calibration Topics 22
P Best Global Option to become an OBL/PLM/Virtual medical device brand in 2020 Other Medical Device Regulations World-Wide 2
earl62 What is the best way to control special characteristics in Control plan? Is it Mandatory to have SPC for IATF 16949? IATF 16949 - Automotive Quality Systems Standard 12
J ISO 13485 System 'soft start' - How to best reflect this in initial audits, management review minutes and other records? ISO 13485:2016 - Medical Device Quality Management Systems 3
S What is the best method to bond urethane foam to urethane foam? Manufacturing and Related Processes 4
Watchcat Best Regulatory Oversight for Off-Shore Device Manufacturing? Other Medical Device Regulations World-Wide 1
E Best way to keep up with pharmaceutical quality knowledge and current affairs Service Industry Specific Topics 1
K Best Measurement Systems Demos in California? General Measurement Device and Calibration Topics 1
qualprod Best approach to get a real value as average? Statistical Analysis Tools, Techniques and SPC 6
CPhelan Best metrics for monitoring low volume (<1000) parts per vendors Supply Chain Security Management Systems 1
DuncanGibbons Best practice for identifying "items" of parts for DFMEA analysis AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
I Non-Conformance vs OFI -- your best descriptions ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 61
A What are Practical data center best practices IEC 27001 - Information Security Management Systems (ISMS) 1
S Best software for customer support/complaints? Customer Complaints 0
R Bill of Materials for a complex product - Industry best practice ISO 13485:2016 - Medical Device Quality Management Systems 2
G Oversized Threads - What sized stab pin would be best to quickly check for oversized thread holes General Measurement Device and Calibration Topics 5
D What is the best software used for the pharma compliance management? Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 0
M Informational MHRA launches consultation on how to best engage patients and the public Medical Device and FDA Regulations and Standards News 0
optomist1 Mobility Field Manufacturing Best - Worst Practices Manufacturing and Related Processes 5
qualprod Best criteria to measure Corrective Action effectiveness - Poor Maintenance ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
S How to Learn all aspects of ISO 27001:2013 | The best way to grab the knowledge on 27001:2013 (Step by Step) IEC 27001 - Information Security Management Systems (ISMS) 7
I Best visual description for SOPs? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 18
Z Best Practices - small volume medical device tube cutting (0.020" to 0.080" OD)? Manufacturing and Related Processes 3
M Over-labelling - Any requirements or best-practices? Other Medical Device Regulations World-Wide 0
M Informational DITTA White Paper on Cybersecurity: Best Practices in the Medical Technology Manufacturing Environment Medical Device and FDA Regulations and Standards News 0
M Informational BSI – MDR Documentation Submissions Best Practices Guidelines Medical Device and FDA Regulations and Standards News 0
K Which clause would best fit this Nonconformance? (Supplier Related) Internal Auditing 2
P How to have employees buy-in of quality procedures. Best practices? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
S Expiry date vs. Best Before Date - Reagents ISO 13485:2016 - Medical Device Quality Management Systems 9
C Will anyone please share training material for ISO:13485:2016 for best practices Training - Internal, External, Online and Distance Learning 0
E Best GD&T training - Your suggestions and recommendations please Training - Internal, External, Online and Distance Learning 3
M Best Practice for setting tolerances on a Drawing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8

Similar threads

Top Bottom