Internal Audit Sampling Plans - Determining Internal Audit Sample Size

Mark Meer

Trusted Information Resource
#41
The value is in the auditor digging into why and also reporting that to management for their action. Not signed? Did you find out why?
I'd say this is the realm of management reviews and CAPA investigations, which would followup from the audit findings.

The auditor points to the "ugly kids", and it is the responsibility of the company to investigate why and take actions.
The auditor can certainly suggest actions, but ultimately their role is in the identification of non-conformances, not fixing them.

You're missing a vital point here Mark: What's working?
Again, my interpretation of "working" is simply: do the procedures meet audit requirements, and are the procedures being followed?

To return to training as an example:

Suppose the company has a bloated 50-page training SOP that calls for 10 signatures from every manager, supervisor and employee on every record.

Is this procedure efficient?
Probably not. (I might make an observation and suggestion)

Are all these signatures necessary or add value?
Probably not. (again, an observation)

Is their process effective at meeting the requirements of the standard I'm auditing against?
This is what I'm interested in. As long as the procedure meets the audit requirements and I'm comfortable they are following it, then I'd say yes, it is effective, regardless of concerns for efficiency or superfluous requirements/controls.
 
Elsmar Forum Sponsor
#42
We aren't talking efficiency. We've got to audit effectiveness and you have yet to demonstrate where you do that. As a result your auditing isn't complying with the ISO 9001 requirements! The effectiveness of a process is shown by the results. Nothing in your descriptions shows you've ever considered that in your audits. How can following a procedure be considered effective? It hasn't been that way since 2000. Since that version, your internal audits have had to report on the effectiveness of the various processes - which is the validation to be discussed at management review.
 
R

Reg Morrison

#43
Since that version, your internal audits have had to report on the effectiveness of the various processes
The ISO 9001 standard requires
The organization shall conduct internal audits at planned intervals to determine whether the quality management system is effectively implemented and maintained.
Most management system professionals that I know would not agree that requirement equates to what you said above. I agree with you that, IN THEORY, internal audits that provide value to the organization need to go beyond compliance checks. However, in the real world, most internal auditors out there barely can tackle the conformance piece of auditing, much less delve into effectiveness. And the really sad part is to realize that day in, day out, thousands of organizations get audited by their respective registrars and in extremely rare occasions the 3rd party auditors write up nonconformities about the sad state of the companies internal audit programs. So, and once again, it is about time for registrars to keep their client's internal audit programs in check.

It would be interesting if we had a poll here for covers to report if their internal audit program effectiveness has ever been reported as nonconforming by their respective registrar. Rarer than an albino unicorn in my experience.
 
#44
Thanks for your agreement, Reg. Since this thread is about INTERNAL audits, wouldn't it be much more effective to keep on TOPIC rather than use it for a platform to slam CB auditors (again)? If you must bring in off topics comments, how about the regulatory auditors, supplier auditors et al?

Part of my interest in the discussion with Mark is to help elevate his understanding of how internal audits are supposed to be implemented. Since most of those internal and external auditors got their training from the same basic IRCA/Exemplar courses, your last comments regarding the audit program being written up is hardly a revelation is it?
 

Helmut Jilling

Auditor / Consultant
#45
So many juicy comments to reply to...this is one of my pet topics....

1) internal auditors definitely need better training, not the conventional IRCA/ANAB et al type of training. But, value added, improvement oriented INTERNAL audits. Help the company perform better and save more money....


2) look for things that are "not the way they ought to be!" CA's, PA's, OFI's, Action Items....all of the above! Nothing wrong with OFI's in internal audits...they make the company better.


3) ask people you audit " How can we do things BETTER?" They have ideas...hear them out!


4) understand WHY you are auditing...to make things better! To help make things better! In my training, there are 5 types of findings...external auditors can only write one type...how utterly regressive is that?
 

RoxaneB

Super Moderator
Super Moderator
#46
It's all well and good for internal auditors to be trained on and have experience in identifying issues that will add value to the management system, but there also needs to be a baseline for that. Too often I see documented processes with no related metrics.

If the documented procedure is followed but the desired results are not being achieved, that can spark several discussions.

If the results are being achieved but the documented procedure is not being followed, that's a different conversation.

If both are spot on, yay!

If both are not where they could be, well, now there's a big discussion to be held.

Getting back to the original topic of sample size, it's basic stats here. If the first record I pull is perfect, odds are the majority of the records will be good. Yes, I'll check more than one to verify this.

If the first record I pull is not so good, odds are the majority of the records will be not so good. And yes, I'll check more than one to verify this.

For an internal audit - at least to 9001/14001 - there is no prescribed rule for sample size. Although, it is a sample...so 100% review cannot be called a "sample". I'd probably call a 100% check a witch hunt, non-value-added, nitpicking and a few other choice terms that are not positive.

An audit - be it internal or external - is a snap shot in time. It is a picture of a moment within the organization's creation of a product/service. A sample of records is the same.

No organization is perfect 100% of the time. There will be things missed. This is a reality. As an auditor, what I look for is a mechanism to self-identify and self-correct such discrepancies, depending on the level of risk.

To use the signed training records as an example, if I found some that were not signed, yet the procedure said they were to be signed, there would be a discussion on how the oversight occurred AND the value of signing them in the first place AND if there is a mechanism in place to verify the signatures if this is considered a critical activity. The result of those conversations would help me to create a meaningful finding (if appropriate). It also helps the organization start the journey of improving the process's effectiveness beyond "We retrained folks to follow the procedure."

To simply say, "Hey, ya'll aren't following your procedure for signing off on training records" is like sitting in a meeting room and saying "Oh, it's dark in here" if the power goes out.
 
Thread starter Similar threads Forum Replies Date
T ISO 13485: 2016 Internal Audit - Is sampling on projects allowed? ISO 13485:2016 - Medical Device Quality Management Systems 6
E Internal Process Audit Sampling using MIL-STD-105-E - 500 operators on 3 shifts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
Q ISO 9001-2015 Internal audit finding Internal Auditing 12
lanley liao How to understand this words that the planning of internal audit shall take into consideration the results of previous audits? Oil and Gas Industry Standards and Regulations 10
A Add MDSAP to Internal Audit Schedule Medical Device Related Regulations 0
J IATF 16949 Internal Audit question - Auditor's responsibility Internal Auditing 6
S IATF 16949 Internal Audit Example IATF 16949 - Automotive Quality Systems Standard 7
R AS9100D internal audit checklist or ISO 9001 2015 to AS9100 D AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
M ISO 13485:2016 internal audit checklist Medical Device and FDA Regulations and Standards News 5
A Internal Audit Questions ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
salaheddine96 Internal audit planning Internal Auditing 2
M ISO 9001 Major Nonconformance Internal Audit Schedule/COVID-19 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 18
E MDR internal audit Internal Auditing 1
U Internal Auditor not trained but done Audit for some process Nonconformance and Corrective Action 5
B Looking for 10 Internal Audit Online Training Participants ISO 17025 related Discussions 2
H AS9100 Checklist for Internal Audit needed AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
A What are the pros and cons of using an audit software for internal auditing? General Auditing Discussions 7
F Internal Audit before Pre-Assessment ISO 17025 related Discussions 2
Q Internal audit plan template Internal Auditing 12
L Internal audit during COVID-19 restrictions ISO 13485:2016 - Medical Device Quality Management Systems 5
O ISO13485 implementation - Are internal audits expected before stage 1 audit? Design and Development of Products and Processes 3
B Using Unreleased Documents & Process Maps for Internal Audit purposes ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
N Small Company - Internal audit process - Who does the audit? Internal Auditing 16
J Does anyone have an excel IATF 16949 Internal Audit checklist I could use? IATF 16949 - Automotive Quality Systems Standard 7
G Addressing Non-Conformances from an Internal Audit that are not product related ISO 13485:2016 - Medical Device Quality Management Systems 11
S Internal audit discrepancy - We missed a few audits that were scheduled Internal Auditing 12
Raffy ISO 14001 9.2.2 Internal Audit Programme Content Internal Auditing 10
N Internal Audit Schedule – Who gets to set the schedule? Internal Auditing 16
V IATF 16949 9.2.2.1 Internal Audit Program - "Process Changes" IATF 16949 - Automotive Quality Systems Standard 11
G Non Conformance During ISO 9001 Audit - Not All Internal Audits Completed General Auditing Discussions 19
B Using external FDA and ISO 13485 audit as internal audit Internal Auditing 6
T Internal Audit Schedule when Hiring Out Internal Auditing 7
D ISO 9001:2015 Internal Audit Training Advice Internal Auditing 10
M Internal audit consultant ISO 13485 (English speaker) Consultants and Consulting 3
S Implementing a 45001 Health & Safety standard - Internal audit plan wanted Internal Auditing 1
F Internal Audit - Procedure example Internal Auditing 5
C Internal Audit - Process Clause Matrix / Audit Checklist ISO 13485:2016 - Medical Device Quality Management Systems 7
CPhelan Internal audit - Combine similar nonconformities in one or keep separate? Internal Auditing 6
M Internal Audit Plan in Retail Internal Auditing 10
D Management of NC after internal system audit IATF 16949 - Automotive Quality Systems Standard 7
A Purchasing - Internal Audit Questions Internal Auditing 8
N Comprehensive Compliance Matrix for Internal Audit Checklist Other Medical Device Regulations World-Wide 1
W Where to begin with an ISO 9001:2015 internal audit Internal Auditing 13
D Internal audit forms or checklists for a medical/veterinary laboratory General Auditing Discussions 5
E Informational Internal Audits - Wear multiple hats what can and can't I audit (so I'm not auditing my own work) Internal Auditing 149
E ISO 9001:2015 - Internal Audit Plan Clauses General Auditing Discussions 8
S Internal Audit Checklist for Application/Software development IEC 27001 - Information Security Management Systems (ISMS) 1
S Internal Audit - Risk and Opportunity (ISO 9001:2015 ) Internal Auditing 1
F API Spec Q1 9th Edition Surveillance Audit - Questions about internal audits. Oil and Gas Industry Standards and Regulations 22
Ashland78 IATF 16949 Internal Audit Checklist Manufacturing and Related Processes 11

Similar threads

Top Bottom