Internal Audit of the Internal Audit System

A

andygr

#41
For us in the AS9100 process approach realm is not the various processes metric's not the true tell of how effective the audit process is?
If the audits of a process are really looking at the process and all its elements proper function the measure used for the process should always be showing improvement as the audit should be identifying any shortcomings that result in poor metrics. This is not even touching on the PM process.

If there is a downward trend to the process measures then it would appear to me that the audits to that point were not effective in some way either by missing a “poor” element of the process or some element of the overall processes of the system.

You would at this point of a degrading metric have to review the applicable audit process to try and determine why they did not identify elements of the process resulting in it's poor performance. You would not have teh person who performed the audit audit themself but have some one else put new fresh eyes on it.
:2cents:
 
Elsmar Forum Sponsor
J

JaneB

#42
They were different auditors, not by my choice.

I did mention that possibly the standard was interpreted differently, but the auditor said it was not a matter of opinion / interpretation but fact.
I do understand that different auditors may have different views.

Purely as a matter of interest, I would query this 'not opinion but fact' statement by asking specifically which clause and which specific words in the clause the auditor is relying on when they say 'every' process must be audited.

Because yes, while specifically internal audit is a process, at what point does one stop checking the checking of the checking? (etc).

If the only reason they are saying 'you must audit this' is because they believe one must 'audit everything' (and I contend it IS a belief/opinon rather than fact) rather than having any other evidence that internal audit is not effective, I'd definitely be querying the value of same.

I repeat: I am definitely not saying internal audit should not be audited at all. What I am saying is that I believe there are cases where such an activity is so low risk, and the putative value of the activity so minimal, that it is not worth doing. And that resources can be put to much better use. Which in my opinion is important.
 

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#43
For us in the AS9100 process approach realm is not the various processes metric's not the true tell of how effective the audit process is?
If the audits of a process are really looking at the process and all its elements proper function the measure used for the process should always be showing improvement as the audit should be identifying any shortcomings that result in poor metrics. This is not even touching on the PM process.

If there is a downward trend to the process measures then it would appear to me that the audits to that point were not effective in some way either by missing a “poor” element of the process or some element of the overall processes of the system.

You would at this point of a degrading metric have to review the applicable audit process to try and determine why they did not identify elements of the process resulting in it's poor performance. You would not have teh person who performed the audit audit themself but have some one else put new fresh eyes on it.
:2cents:
I've often wondered what really good audit process metrics would look like, but whatever I come up with has so many factors in the equation that I can't pin down the single independent variable upon which my performance is reflected.

You see, it starts with management resourcing (definition and resourcing the QMS) and ends with management resourcing (management review). The actual auditing bit comes in the middle.

Factors that can depend on other factors include:

- Which process is being audited (Some are more complex than others, some people are more open and willing while others are standoffish and try to hide things),

- What else is going on (I've been made Document Control Admin too, so my auditing is only part time at best now)

- What tools I use to audit (sometimes I'll make flow charts for my audit and for my customer to use later, and this can take time)

- What is going on in the organization (stresses in the market, current issues and new products/services coming on line can slow down the audit process.)

And so I trot out the usual: number of audits done, nonconformances closed, but I also throw in some others: time to close, response time, number of extensions. Even though these are also impacted by others (there are people who can certainly make me look bad if they set their minds to it) they seem to come as close to audit process performance as is practical. My true, innately developed assessment of my progress is more qualitative and spans years.
 
J

JaneB

#44
For us in the AS9100 process approach realm is not the various processes metric's not the true tell of how effective the audit process is?
Really?
While it sounds black and white and simple, I don't fully agree with it.

I read your post as saying that the audit process is entirely responsible for ensuring all other system processes are showing improvement (per metrics) and if not, the audit process should have found deficiencies/identified the problems. And if it didn't, it's not functioning properly.

Audits to me are an input to management review and I don't think the audit process has the full responsibility you ascribe to it. Plus there are simply too many possible variables for my liking.
 
B

baby12

#45
What normally happened is that the internal auditor does not audit their own work, they rather get a external auditor or get someone to come and audit their job so that gaps can be identified before the company get audited by certification company is so doing ready yourself before actual audit by the certification company to check the level of compliance to their requirements:)
 
B

baby12

#46
Is it good to perform an internal audit of the internal audit system?

I have not seen it done before. I have more than one internal auditor. I could have auditor A audit all audits where she was not involved and have a different auditor audit the audits where auditor A was involved. What do you guys think?
remember internal auditors audit internal audit system by doing this you help yourself to see the level of compliance becuase when you get audited by certification body they want to see the records of previous audit to see that really the way they want is happening
 
J

Jason PCSwitches

#47
One of the 1st things audited during any given audit by a CB is the internal audit process. That being said I want to make sure that my program is running appropriately. Having someone audit (review/analyze) the process prior to a CB audit is, IMHO, valuable. Not only for the obvious reason of passing the CB audit but also to have a fresh set of eyes or an outside perspective take a look and providing feedback. This allows the opportunity for improvement. Due to the critical nature of the IA program I usually schedule an audit of the internal audit process 1-2 months prior to a CB audit. This allows us time to implement any changes that may need to be made prior to their arrival.

It should not be taken personal, just like standard audits should not be perceived as a personal assessment.

Auditors - you need to be audited too, bottom-line. Otherwise how can you grow??
 
A

andygr

#48
“I read your post as saying that the audit process is entirely responsible for ensuring all other system processes are showing improvement (per metrics) and if not, the audit process should have found deficiencies/identified the problems. And if it didn't, it's not functioning properly.”

Not so, the responsibility for improvement lays with the process owner not the audit function in my view. The audit is a more of an independent verification process with an output that should be used by the process being audited to make some judgment on how well it is working based on the metrics they use and addressing identified issues from the metrics.
Management uses this same output as the process from the audit to verify that the process is using the measure of their process performance as well as any audit information to address concerns and show improvement.

The audit should be looking at process performance and based on the indicators see if the process, as defined, might have some areas that need improvement and not just correction based on poor performance.
If down the line there are issues that arise in a process there has to be some question as to why they were not identified as part of the audit process. To me this review of why the issues occurred would be the most objective determination metric of how to judge the methodology of the audit performed on the process and how the audit might need to improve.

In my view all too often the audit process is given more responsibility than it should be given to fix or make sure issues are fixed. I feel that this responsibility best lies with management who has the responsibility for overseeing processes.

My view but keep in mind the one thing that I do excell in is serving as a very good bad example.
 
J

Jason PCSwitches

#49
Not necessarily. Audits cannot examine every aspect of a process, so to state that if a process if having problems down the line that it's also connected to the AI process is premature. You must 1st examine what the last IA of that process examined, then make a decision.

If the aspect of the process that went askew was not audited, well, that's why the standard directs us to take "previous audit findings" into consideration - so it will be reviewed the next go-round.
 

RoxaneB

Super Moderator
Super Moderator
#50
Re: Internal Audit of the Internal Audit Process ?

Quality-Geek said:
As the MR/Lead Auditor (we're a small plant and all wear lots of hats), I review each audit for several reasons. Was it completed? Were problems found? Were they legitimate problems? What evidence was reviewed? Was it submitted with the audit? And the list goes on...
Is this really an audit of an audit or simply a checklist to be completed so that the audit can be 'formally' completed within your system?

Quality-Geek said:
We also have an audit of the internal audit process. This looks primarily at scheduling, timely completion of audits, scope of the audits, training, and the connections between audit results and the CAPA system and management review. This particular audit is completed by one of the auditors (not me), and is done at least once a year. If the audit reports justify it, I'll schedule this audit as often as quarterly.
Quarterly? That sounds a bit much, in my opinion, and means that the focus is on individual issues instead of something that hints towards a systematic problem.

Quality-Geek said:
There have been years where the only way audits are completed is because I do them or literally drag the assigned auditor around to complete it.
Again, systematic. WHY are you doing them instead of having the schedule adhered to? WHAT is management commitment like to the audit process? WHERE is the value of the audits?

Quality-Geek said:
Sadly, this hasn't always made the internal audot process stronger. I keep telling myself this takes time...
Before you ask management what they see as the value of audits, might I suggest you ask yourself that same question. WHY is your organization doing audits? If the reason is to simply meet requirements of some standard or to keep a piece of paper on your wall, then you're right, things will not improve.

IF, however, you see value in audits, you need to explain this value to management...and your auditors. It's not about simply writing down every single little fault with a document. It's about making a recommendation to overhaul the document control process and include the value/risk of such an activity. It's not about auditors not finishing their paperwork. It's about developing an audit process that is simple, user-friendly, understandable and transparent. Auditing can be a value-added activity that helps to bind your systems and processes even more tightly, integrating their tools and language.

I recall being part of an audit programme once that was like yours. It was a chore. No one saw the value. When it became my responsibility to oversee the programme, I gave it an overhaul. We added a section to findings, clearly showing the risk of being in nonconformance. We added a section to opportunities, showing the value of being implemented. We paired up our audit of the details with a view of the 'big picture'.

And how did we know that we had done the right thing?

Because during an external audit, the auditor said "Everything I would have found, you already found and have a plan to address. Unconditional approval."
 
Thread starter Similar threads Forum Replies Date
Q ISO 9001-2015 Internal audit finding Internal Auditing 12
lanley liao How to understand this words that the planning of internal audit shall take into consideration the results of previous audits? Oil and Gas Industry Standards and Regulations 10
A Add MDSAP to Internal Audit Schedule Medical Device Related Regulations 0
J IATF 16949 Internal Audit question - Auditor's responsibility Internal Auditing 6
S IATF 16949 Internal Audit Example IATF 16949 - Automotive Quality Systems Standard 7
R AS9100D internal audit checklist or ISO 9001 2015 to AS9100 D AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
M ISO 13485:2016 internal audit checklist Medical Device and FDA Regulations and Standards News 5
A Internal Audit Questions ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
salaheddine96 Internal audit planning Internal Auditing 2
M ISO 9001 Major Nonconformance Internal Audit Schedule/COVID-19 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 18
E MDR internal audit Internal Auditing 1
U Internal Auditor not trained but done Audit for some process Nonconformance and Corrective Action 5
B Looking for 10 Internal Audit Online Training Participants ISO 17025 related Discussions 2
H AS9100 Checklist for Internal Audit needed AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
A What are the pros and cons of using an audit software for internal auditing? General Auditing Discussions 7
F Internal Audit before Pre-Assessment ISO 17025 related Discussions 2
Q Internal audit plan template Internal Auditing 12
L Internal audit during COVID-19 restrictions ISO 13485:2016 - Medical Device Quality Management Systems 5
O ISO13485 implementation - Are internal audits expected before stage 1 audit? Design and Development of Products and Processes 3
B Using Unreleased Documents & Process Maps for Internal Audit purposes ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
N Small Company - Internal audit process - Who does the audit? Internal Auditing 16
J Does anyone have an excel IATF 16949 Internal Audit checklist I could use? IATF 16949 - Automotive Quality Systems Standard 7
G Addressing Non-Conformances from an Internal Audit that are not product related ISO 13485:2016 - Medical Device Quality Management Systems 11
S Internal audit discrepancy - We missed a few audits that were scheduled Internal Auditing 12
Raffy ISO 14001 9.2.2 Internal Audit Programme Content Internal Auditing 10
N Internal Audit Schedule – Who gets to set the schedule? Internal Auditing 16
V IATF 16949 9.2.2.1 Internal Audit Program - "Process Changes" IATF 16949 - Automotive Quality Systems Standard 11
G Non Conformance During ISO 9001 Audit - Not All Internal Audits Completed General Auditing Discussions 19
B Using external FDA and ISO 13485 audit as internal audit Internal Auditing 6
T Internal Audit Schedule when Hiring Out Internal Auditing 7
D ISO 9001:2015 Internal Audit Training Advice Internal Auditing 10
M Internal audit consultant ISO 13485 (English speaker) Consultants and Consulting 3
S Implementing a 45001 Health & Safety standard - Internal audit plan wanted Internal Auditing 1
F Internal Audit - Procedure example Internal Auditing 5
C Internal Audit - Process Clause Matrix / Audit Checklist ISO 13485:2016 - Medical Device Quality Management Systems 7
CPhelan Internal audit - Combine similar nonconformities in one or keep separate? Internal Auditing 6
M Internal Audit Plan in Retail Internal Auditing 10
D Management of NC after internal system audit IATF 16949 - Automotive Quality Systems Standard 7
A Purchasing - Internal Audit Questions Internal Auditing 8
N Comprehensive Compliance Matrix for Internal Audit Checklist Other Medical Device Regulations World-Wide 1
W Where to begin with an ISO 9001:2015 internal audit Internal Auditing 13
D Internal audit forms or checklists for a medical/veterinary laboratory General Auditing Discussions 5
E Informational Internal Audits - Wear multiple hats what can and can't I audit (so I'm not auditing my own work) Internal Auditing 149
E ISO 9001:2015 - Internal Audit Plan Clauses General Auditing Discussions 8
S Internal Audit Checklist for Application/Software development IEC 27001 - Information Security Management Systems (ISMS) 1
S Internal Audit - Risk and Opportunity (ISO 9001:2015 ) Internal Auditing 1
F API Spec Q1 9th Edition Surveillance Audit - Questions about internal audits. Oil and Gas Industry Standards and Regulations 22
Ashland78 IATF 16949 Internal Audit Checklist Manufacturing and Related Processes 11
Ed Panek Root Cause CAPAs from internal audit ISO 13485:2016 - Medical Device Quality Management Systems 16
K Student wanting Case Study about Internal Audit Report Internal Auditing 1

Similar threads

Top Bottom