Internal Audit Sampling Plans - Determining Internal Audit Sample Size

C

chaosweary

#1
So I am doing an internal assessment and pull 5 training records to verify compliance. After the audit my manager asks me how did I come up with pulling 5 records, how did you determine not to pull more or less? I said the number was 10% of the personell working in the area, no other logic. Is there any sample plans for how you pull samples internal audits? I thought about using the mil standard 105E or ANSI Z1.4 but those are really for product acceptance sampling, what sample plan or reference can I start using to get him off my back as to the logic of the "determination" of the sample plan?

:thanx:
 
Elsmar Forum Sponsor
#2
Well I guess..........

it depends on the purpose of the audit. Since there are a number to choose from, it'd be helpful to know what that was.;)

Personally, unless you are trying to replicate a 3rd party audit, as in before registration/surveillance, I can't see why you'd (randomly) want to see training records:mg: What are you trying to establish?

Have you had a change in personnel? In such a case you'd be better off finding which specific people haven't had training to perform their assignments.

I'm not surprised that you were challenged on the sampling. I'm thinking the manager was (covertly) challenging the whole purpose of the audit. Using a sampling plan is of no consequence to you and most of all, him. You probably don't have a 'focus' for the audit that management would endorse, so until the planning addresses an issue that they see as value added, the audits will miss vital opportunities.:bonk:

By way of example; there was a company which always had product quality issues during the summer. They employed local 'interns' to cover for vacation working. Because they chose to audit 'an element a month' (QS-9000 standard), they missed '4.18' every year (one year completely, then by 1 month the next year.....). Shouldn't they have audited the training process when it was implemented to see what was going on?? (i.e. how effective it was).:yes:

So, my advice is look at your overall audit scheduling, not the sampling. Don't be random, that's an external audit technique and not for an internal auditor.:eek:

Andy
 

Helmut Jilling

Auditor / Consultant
#3
Andy, I completely disagree! Chaosweary was entirely correct. Of course he is supposed to audit training records as a linkage to shop floor personnel. It is one of the expected linkages between the shop processes and the training process. That is the whole point of this "process approach" to connect the linkages. Or, are you looking at a different angle?

As to what "sampling plan" was followed, that is up to the auditor. You can randomly select some, and 5 of 50 is fine. Or, I tend to select based on who I interviewed on the floor. Those that demonstrate some weakness are worth selecting their records.
 
C

chaosweary

#4
Not a choice

The audit purpose is to ensure all processes that affect quality meet the intent of the ISO/TS 16949 Automotive Quality Technical Specification and the company's Quality Manual.
Part of my internal audit training is to verify all employees have had training according to their job responsibilities (that affect quality) and the training is logged in our training system. Auditing for training, corrective/preventive action, records, measurement & process improvement etc are all basics of every single audit I do, so I am not sure where your coming from. I don't work for QA that samples products I work for the corporate internal audit organization:bonk:

The problem is that I cannot find any formal guidance on sample size other than for product audits. I know the boss is just challenging me to find a quantitative answer, but I have to say that I can't find a reference (staring at the ASQ quality audit handbook right in front of me). I wonder if I should throw in the towel and say, "I don't know, how many records should I sample?" and then say, "how did you come up with that sample size?" I think Andy is not understanding the intent. As an internal auditor I audit everything a third party auditor audits and much much more!
 

RoxaneB

Super Moderator
Super Moderator
#5
There really is no set guideline on sample size. Some people like 10%. Some like more...some like less. What matters is that:
  • The Auditor explains to the Auditee that sampling is not an exact science and is conducted in a finite period of time with finite resources;
  • The Auditor has properly used the method of sampling to be confident is his/her audit conclusions; and,
  • The sampling can be verified (i.e., records looked at are documented so that the evidence can be traced).

I agree with looking at the training records but for sampling and to show an audit trail, I usually leave it to the end of audit when the team has compiled a list of auditees (and positions). With HR we then look at the required training for those positions and verify that they are required to be trained on their job and that they have indeed been trained...that whole qualified and competent thing.
 
B

Bill Pflanz

#6
When I was studying for my Certified Quality Auditor exam, I tried to find a sampling plan method for auditing. All the references to sampling were very vague and none used any formal sampling methods. The use of 5 samples appears to have its roots in financial auditing.

Dennis Arter is well known in the auditing field and in his book on quality audits he did provide some guidelines. He felt the sample size should depend on how sure you wanted your conclusion to be correct and how many occurrences there are. The problem is that you do not know how many defects there are. He then went on to say that 30 to 100 samples are sufficiently precise but also said that being statistically precise is not valuable if management doesn't understand it. He also said that concluding that you have a major problem based on 5 samples is not valid either due to its lack of preciseness. The practical issue with the number of samples that Arter suggested is getting the audit done in a reasonable time and cost.

The real response should have been that you were using a sample of 5 but it does not confirm either a problem or lack of problem. Just because you did not find one does not mean there was not a problem. If you did find one by taking just 5 samples, the manager should consider that it is more than an isolated incident and the process should be at least reviewed in more detail. Now if you find 3 or more problems with just 5 samples than everyone should agree that there is a likely problem.

Bill Pflanz
 
R

ralphsulser

#7
hjilling said:
Or, I tend to select based on who I interviewed on the floor. Those that demonstrate some weakness are worth selecting their records.
I think this is the better approach because you have met and interviewed the employees, and you have formed an opinion their competentcy, then look at the training records to follow the audit trail.
 
C

chaosweary

#9
Thanks!

Bill Pflanz said:
The real response should have been that you were using a sample of 5 but it does not confirm either a problem or lack of problem. Just because you did not find one does not mean there was not a problem. If you did find one by taking just 5 samples, the manager should consider that it is more than an isolated incident and the process should be at least reviewed in more detail. Now if you find 3 or more problems with just 5 samples than everyone should agree that there is a likely problem.

Bill Pflanz
I think I will use this explanation (in my own words). I am glad there is a lot of subjective material out there on this one. It give some breathing room as to the explanation.
:agree1: :thanx:
 
G

Greg B

#10
AndyN said:
Personally, unless you are trying to replicate a 3rd party audit, as in before registration/surveillance, I can't see why you'd (randomly) want to see training records:mg: What are you trying to establish?Andy
Andy, I cannot agree with this at ALL. Whilst I do not audit everything an external auditor would audit I think it is part of any full process audit that I establish that everyone is competent in their positions. I audit the Process not the Standard (I leave that to the 3rd party) and by process I mean All the Inputs and Outputs and everything involved in the process steps. I look from a business angle and look at waste, efficiency etc. Personally I don't care if the auditee is a bit put out by my randomness and I never set a figure or percentage.
In general, I don't agree that we require a formalised sampling number. We are not there to Pass or Fail as in a QC setting. We are there to ensure that the business processess are efficient and effective. We do this by obeservation, questioning and the the collection of facts.
 
Thread starter Similar threads Forum Replies Date
T ISO 13485: 2016 Internal Audit - Is sampling on projects allowed? ISO 13485:2016 - Medical Device Quality Management Systems 6
E Internal Process Audit Sampling using MIL-STD-105-E - 500 operators on 3 shifts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
Q ISO 9001-2015 Internal audit finding Internal Auditing 12
lanley liao How to understand this words that the planning of internal audit shall take into consideration the results of previous audits? Oil and Gas Industry Standards and Regulations 10
A Add MDSAP to Internal Audit Schedule Medical Device Related Regulations 0
J IATF 16949 Internal Audit question - Auditor's responsibility Internal Auditing 6
S IATF 16949 Internal Audit Example IATF 16949 - Automotive Quality Systems Standard 7
R AS9100D internal audit checklist or ISO 9001 2015 to AS9100 D AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
M ISO 13485:2016 internal audit checklist Medical Device and FDA Regulations and Standards News 5
A Internal Audit Questions ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
salaheddine96 Internal audit planning Internal Auditing 2
M ISO 9001 Major Nonconformance Internal Audit Schedule/COVID-19 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 18
E MDR internal audit Internal Auditing 1
U Internal Auditor not trained but done Audit for some process Nonconformance and Corrective Action 5
B Looking for 10 Internal Audit Online Training Participants ISO 17025 related Discussions 2
H AS9100 Checklist for Internal Audit needed AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
A What are the pros and cons of using an audit software for internal auditing? General Auditing Discussions 7
F Internal Audit before Pre-Assessment ISO 17025 related Discussions 2
Q Internal audit plan template Internal Auditing 12
L Internal audit during COVID-19 restrictions ISO 13485:2016 - Medical Device Quality Management Systems 5
O ISO13485 implementation - Are internal audits expected before stage 1 audit? Design and Development of Products and Processes 3
B Using Unreleased Documents & Process Maps for Internal Audit purposes ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
N Small Company - Internal audit process - Who does the audit? Internal Auditing 16
J Does anyone have an excel IATF 16949 Internal Audit checklist I could use? IATF 16949 - Automotive Quality Systems Standard 7
G Addressing Non-Conformances from an Internal Audit that are not product related ISO 13485:2016 - Medical Device Quality Management Systems 11
S Internal audit discrepancy - We missed a few audits that were scheduled Internal Auditing 12
Raffy ISO 14001 9.2.2 Internal Audit Programme Content Internal Auditing 10
N Internal Audit Schedule – Who gets to set the schedule? Internal Auditing 16
V IATF 16949 9.2.2.1 Internal Audit Program - "Process Changes" IATF 16949 - Automotive Quality Systems Standard 11
G Non Conformance During ISO 9001 Audit - Not All Internal Audits Completed General Auditing Discussions 19
B Using external FDA and ISO 13485 audit as internal audit Internal Auditing 6
T Internal Audit Schedule when Hiring Out Internal Auditing 7
D ISO 9001:2015 Internal Audit Training Advice Internal Auditing 10
M Internal audit consultant ISO 13485 (English speaker) Consultants and Consulting 3
S Implementing a 45001 Health & Safety standard - Internal audit plan wanted Internal Auditing 1
F Internal Audit - Procedure example Internal Auditing 5
C Internal Audit - Process Clause Matrix / Audit Checklist ISO 13485:2016 - Medical Device Quality Management Systems 7
CPhelan Internal audit - Combine similar nonconformities in one or keep separate? Internal Auditing 6
M Internal Audit Plan in Retail Internal Auditing 10
D Management of NC after internal system audit IATF 16949 - Automotive Quality Systems Standard 7
A Purchasing - Internal Audit Questions Internal Auditing 8
N Comprehensive Compliance Matrix for Internal Audit Checklist Other Medical Device Regulations World-Wide 1
W Where to begin with an ISO 9001:2015 internal audit Internal Auditing 13
D Internal audit forms or checklists for a medical/veterinary laboratory General Auditing Discussions 5
E Informational Internal Audits - Wear multiple hats what can and can't I audit (so I'm not auditing my own work) Internal Auditing 149
E ISO 9001:2015 - Internal Audit Plan Clauses General Auditing Discussions 8
S Internal Audit Checklist for Application/Software development IEC 27001 - Information Security Management Systems (ISMS) 1
S Internal Audit - Risk and Opportunity (ISO 9001:2015 ) Internal Auditing 1
F API Spec Q1 9th Edition Surveillance Audit - Questions about internal audits. Oil and Gas Industry Standards and Regulations 22
Ashland78 IATF 16949 Internal Audit Checklist Manufacturing and Related Processes 11

Similar threads

Top Bottom