Internal Audit Sampling Plans - Determining Internal Audit Sample Size

#11
That's just fine..........

Greg B said:
Andy, I cannot agree with this at ALL. Whilst I do not audit everything an external auditor would audit I think it is part of any full process audit that I establish that everyone is competent in their positions. I audit the Process not the Standard (I leave that to the 3rd party) and by process I mean All the Inputs and Outputs and everything involved in the process steps. I look from a business angle and look at waste, efficiency etc. Personally I don't care if the auditee is a bit put out by my randomness and I never set a figure or percentage.
In general, I don't agree that we require a formalised sampling number. We are not there to Pass or Fail as in a QC setting. We are there to ensure that the business processess are efficient and effective. We do this by obeservation, questioning and the the collection of facts.
I'm not looking for an endorsement! 'H' and I don't see eye to eye on audit stuff, either. What I'm sharing is based on training and consulting over 20 years of implementation and meeting very frustrated auditors and disengaged management, who don't have a clue what value to expect from their internal audit program.:eek:

'H' - I'm guessing you do quite a lot of external audits. Third party audit clients don't complain (well, rarely, since they risk reprisal) to their registrar - they complain to me, when they come to my classes. When I consult, I constantly meet management who are gun shy of third party auditors and cannot show any tangible benefit from audits.:mg:

Sure, they smile and tell auditors "Great, we welcome your comments", but behind the scenes, it's different, in my experience.

My comments are based on trying to offer my clients an alternative to finding a 'sampling plan' to audit. I'm attempting to give auditing the same kudos that 6 Sigma gets - think about it, the process isn't so different:read:

Andy (self confessed audit heretic.........):lmao:
 
Elsmar Forum Sponsor

Helmut Jilling

Auditor / Consultant
#12
AndyN said:
I'm not looking for an endorsement! 'H' and I don't see eye to eye on audit stuff, either. What I'm sharing is based on training and consulting over 20 years of implementation and meeting very frustrated auditors and disengaged management, who don't have a clue what value to expect from their internal audit program.:eek:

'H' - I'm guessing you do quite a lot of external audits. Third party audit clients don't complain (well, rarely, since they risk reprisal) to their registrar - they complain to me, when they come to my classes. When I consult, I constantly meet management who are gun shy of third party auditors and cannot show any tangible benefit from audits.:mg:

Sure, they smile and tell auditors "Great, we welcome your comments", but behind the scenes, it's different, in my experience.

My comments are based on trying to offer my clients an alternative to finding a 'sampling plan' to audit. I'm attempting to give auditing the same kudos that 6 Sigma gets - think about it, the process isn't so different:read:

Andy (self confessed audit heretic.........):lmao:

I don't disagree with any of these thoughts. I also consult and see some of the same stuff.

What I disagreed with was your comment questioning why an internal auditor was auditing linakges to training. My comment was of course he needs to audit linkages to training. That is one of the most critical linkages and needs to be regularly audited. It is rarely as robust as we expect. I wrote an nc just this morning at a company that was benchmark caliber, but still had a training issue.

(PS: As to the other comments, of course some managers are gun shy of some auditors. Sometimes they have had bad auditors, and sometimes the auditors were fine but the managers did not know what they were doing. That is why you and I offer training and consulting. But neither auditors nor consultants are always the bad guys.)
 

Miner

Forum Moderator
Staff member
Admin
#13
hjilling said:
Or, I tend to select based on who I interviewed on the floor. Those that demonstrate some weakness are worth selecting their records.
Speaking as an auditee, I have found that auditors that collect large, random samples rarely find issues, but auditors that used small targeted samples such as that described by hjilling, or doing a deep dive following a process from start to finish across multiple procedures usually find the weak points.

Targeted samples are usually driven by changes in the process/organization. Have you hired anyone or used a temporary agency since the last audit? Let me see their training records. Have you added a new customer? Let me see your contract reviews and design records. And on and on.
 
#14
Yup - totally!

Miner said:
Speaking as an auditee, I have found that auditors that collect large, random samples rarely find issues, but auditors that used small targeted samples such as that described by hjilling, or doing a deep dive following a process from start to finish across multiple procedures usually find the weak points.

Targeted samples are usually driven by changes in the process/organization. Have you hired anyone or used a temporary agency since the last audit? Let me see their training records. Have you added a new customer? Let me see your contract reviews and design records. And on and on.
Hence, not random and the 'sample' is based on something management would be interested in..................didn't I explain that????:lol: :agree1: :yes:

Thanks, Miner:applause:

Andy
 
G

Greg B

#15
So many new Moderators...

Wow there are a lot of new moderators that I have not yet met. Just to let everyone know that I am back and I am taking names. Glad you guys worked all of this out BTW. It's a bit hard, sometimes, to convey our collective POVs when distance and circumstances are all different, as in audits the scenario and the people always change. The scope of the audit and what the customer/Client wants or needs also differs. Good luck and nice meeting you all. Some may say that at times I can be very blunt but I like blunt! :lmao:
 
B

Bill Pflanz

#16
I think we lost the person originally posting the question in all the chaos of responses. No wonder he is weary. I hope chaosweary lets us know what happens the next time he is asked about audit sampling. :)

Bill Pflanz
 
C

chaosweary

#17
Just Listening

Actually I am still very much here! There are a lot of viewpoints and all subjectively valid. I got my answer early on for sample sizes which is that there is no formal standard for internal audit (process) sample sizes. "phew" now I know that my boss was just being ...well a boss. You guys are great!
Auditing training records and the training process is mandatory even if its not directed by management to comply with ISO 9001 and TS 16949 I am sure Juran would be totally against it, but Juran doesn't pay my bills.
The sore spot of this thread I think hits on value add. Now here is where I get a lot of lets say "disagreement". The value add of internal audit can be huge if management decides to use it as a tool for feedback. But what if they do not? What if they don't even have one little bitsy tiny weeny quality objective in their business plan? Should all of us auditors in those circumstances throw up our hands, quit and find a "quality" oriented organization? I think not, those types of organizations are few and far between. I am not going to try to convince someone who makes 20x may pay that they need to listen to their internal audit group or use them in a value added fashion. That is Andy's job! :lol: The other day we got a news release email that we had acquired another company. It was news to my manager as well. I didn't see him running to set up a meeting to see if they needed to be put under our ISO umbrella as well...The struggle goes on.
....I was really sad when I found out there wasn't a Santa Claus too...:mg:
 
S

schclark

#18
I agree with others that, for a system audit, such as for training confirmation, you need to audit the training records of persons that have been interviewed as part of the audit of work methods. I believe, however, that for audits of products, where the sampling is without replacement, a sample size of 5 is valid for this reason:
Using the Hypergeometric distribution (for non-replacement of sample items), for a 1% rate of non-conformities in a population, the probability of finding 1 non-conformity in the sample of 5 from a population of 100 is approximately 0.05. For a population of 1000, the probability of 1 non-conformity in a sample of 5 is 0.048.
 
Last edited by a moderator:
S

samsung

#19
I agree with others that, for a system audit, such as for training confirmation, you need to audit the training records of persons that have been interviewed as part of the audit of work methods. I believe, however, that for audits of products, where the sampling is without replacement, a sample size of 5 is valid for this reason:
Using the Hypergeometric distribution (for non-replacement of sample items), for a 1% rate of non-conformities in a population, the probability of finding 1 non-conformity in the sample of 5 from a population of 100 is approximately 0.05. For a population of 1000, the probability of 1 non-conformity in a sample of 5 is 0.048.
Welcome to Cove.

Why do you insist on auditing training records? What if you interview a person, find him weak somewhere and also find a good training record of him. On the contrary, there may be persons with excellent track record but no or little training records. For some, training may not be applicable, they may learn the things through other means and hence no 'training' records are necessary.
 
S

schclark

#20
My submittal was only my first one to this thread, therefore, and you don't know me personally, therefore, why do you say "Why do you insist on auditing training records?". That comment should not be addressed to me, but ISO 9001 6.2.2 e), requires registered companies to "maintain appropriate records of education, training, skills and experience...", therefore, I believe companies audit training records to meet that requirement.
 
Thread starter Similar threads Forum Replies Date
T ISO 13485: 2016 Internal Audit - Is sampling on projects allowed? ISO 13485:2016 - Medical Device Quality Management Systems 6
E Internal Process Audit Sampling using MIL-STD-105-E - 500 operators on 3 shifts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
Q ISO 9001-2015 Internal audit finding Internal Auditing 12
lanley liao How to understand this words that the planning of internal audit shall take into consideration the results of previous audits? Oil and Gas Industry Standards and Regulations 10
A Add MDSAP to Internal Audit Schedule Medical Device Related Regulations 0
J IATF 16949 Internal Audit question - Auditor's responsibility Internal Auditing 6
S IATF 16949 Internal Audit Example IATF 16949 - Automotive Quality Systems Standard 7
R AS9100D internal audit checklist or ISO 9001 2015 to AS9100 D AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
M ISO 13485:2016 internal audit checklist Medical Device and FDA Regulations and Standards News 5
A Internal Audit Questions ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
salaheddine96 Internal audit planning Internal Auditing 2
M ISO 9001 Major Nonconformance Internal Audit Schedule/COVID-19 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 18
E MDR internal audit Internal Auditing 1
U Internal Auditor not trained but done Audit for some process Nonconformance and Corrective Action 5
B Looking for 10 Internal Audit Online Training Participants ISO 17025 related Discussions 2
H AS9100 Checklist for Internal Audit needed AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
A What are the pros and cons of using an audit software for internal auditing? General Auditing Discussions 7
F Internal Audit before Pre-Assessment ISO 17025 related Discussions 2
Q Internal audit plan template Internal Auditing 12
L Internal audit during COVID-19 restrictions ISO 13485:2016 - Medical Device Quality Management Systems 5
O ISO13485 implementation - Are internal audits expected before stage 1 audit? Design and Development of Products and Processes 3
B Using Unreleased Documents & Process Maps for Internal Audit purposes ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
N Small Company - Internal audit process - Who does the audit? Internal Auditing 16
J Does anyone have an excel IATF 16949 Internal Audit checklist I could use? IATF 16949 - Automotive Quality Systems Standard 7
G Addressing Non-Conformances from an Internal Audit that are not product related ISO 13485:2016 - Medical Device Quality Management Systems 11
S Internal audit discrepancy - We missed a few audits that were scheduled Internal Auditing 12
Raffy ISO 14001 9.2.2 Internal Audit Programme Content Internal Auditing 10
N Internal Audit Schedule – Who gets to set the schedule? Internal Auditing 16
V IATF 16949 9.2.2.1 Internal Audit Program - "Process Changes" IATF 16949 - Automotive Quality Systems Standard 11
G Non Conformance During ISO 9001 Audit - Not All Internal Audits Completed General Auditing Discussions 19
B Using external FDA and ISO 13485 audit as internal audit Internal Auditing 6
T Internal Audit Schedule when Hiring Out Internal Auditing 7
D ISO 9001:2015 Internal Audit Training Advice Internal Auditing 10
M Internal audit consultant ISO 13485 (English speaker) Consultants and Consulting 3
S Implementing a 45001 Health & Safety standard - Internal audit plan wanted Internal Auditing 1
F Internal Audit - Procedure example Internal Auditing 5
C Internal Audit - Process Clause Matrix / Audit Checklist ISO 13485:2016 - Medical Device Quality Management Systems 7
CPhelan Internal audit - Combine similar nonconformities in one or keep separate? Internal Auditing 6
M Internal Audit Plan in Retail Internal Auditing 10
D Management of NC after internal system audit IATF 16949 - Automotive Quality Systems Standard 7
A Purchasing - Internal Audit Questions Internal Auditing 8
N Comprehensive Compliance Matrix for Internal Audit Checklist Other Medical Device Regulations World-Wide 1
W Where to begin with an ISO 9001:2015 internal audit Internal Auditing 13
D Internal audit forms or checklists for a medical/veterinary laboratory General Auditing Discussions 5
E Informational Internal Audits - Wear multiple hats what can and can't I audit (so I'm not auditing my own work) Internal Auditing 149
E ISO 9001:2015 - Internal Audit Plan Clauses General Auditing Discussions 8
S Internal Audit Checklist for Application/Software development IEC 27001 - Information Security Management Systems (ISMS) 1
S Internal Audit - Risk and Opportunity (ISO 9001:2015 ) Internal Auditing 1
F API Spec Q1 9th Edition Surveillance Audit - Questions about internal audits. Oil and Gas Industry Standards and Regulations 22
Ashland78 IATF 16949 Internal Audit Checklist Manufacturing and Related Processes 11

Similar threads

Top Bottom