Determination of Sample Sizes when Auditing

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#1
An oldie but goodie...

How do you determine sample size when auditing records?

How do you determine sample size when auditing a process across multiple production lines?

How do you determine sample size when asking employees how their company quality policy?

What statistically valid method do you use to determine various sample sizes?
 
Elsmar Forum Sponsor

John Broomfield

Staff member
Super Moderator
#2
An oldie but goodie...

How do you determine sample size when auditing records?

How do you determine sample size when auditing a process across multiple production lines?

How do you determine sample size when asking employees how their company quality policy?

What statistically valid method do you use to determine various sample sizes?
Marc,

The sample has to be large enough for the auditee and the audit client to accept the audit findings as fact.

And the findings or records always include the sample size.

But, like you, I'm looking forward to the answers in this thread.

John
 

Jim Wynne

Staff member
Admin
#3
There is no general requirement for audit samples to be statistically valid. That could be prohibitive in many cases. Auditors look for objective evidence. Regardless of the sample size, if an auditor looks at n records and finds no nonconformity, there is no assurance that record n+1 wouldn't be a bad one.
 

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#4
I am asking because like many here, my auditing experience goes back quite a few years. The other thread got me to thinking about how I *really* determine sample size in various situations whether it be records or even how many employees to ask the old "What does your company quality policy mean to you and how does it affect what you do?" which we have been asking people for years.

I have always been at odds with this aspect of auditing because it has always been something you typically have to do "on the fly" since until you are in the middle of doing an audit you often don't know things such as how many purchase orders there will be.

That said, how do you come up with an audit plan and how does "on the fly" determination of sample size affect the iplanned audit schedule.

I know the old "what ever you and your client are comfortable with", not to mention aspects such as "the risk" level (etc.), but again - How do you determine that?

I will say I think audit plans are better today than say 10 or 15 years ago. It has evolved into a more defined planned audits. I'm not trying to make a big deal out of this. I'm simply curious as to how others handle determining sample sizes in various situations.
 

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#5
There is no general requirement for audit samples to be statistically valid. That could be prohibitive in many cases. Auditors look for objective evidence. Regardless of the sample size, if an auditor looks at n records and finds no nonconformity, there is no assurance that record n+1 wouldn't be a bad one.
I agree there is no requirement, or at least none that I am aware of. If I have 5000 records (as an example), is a sample size of 100 significant enough to even warrant sampling any more than a sample size of 10 - Or of not bothering to take a sample at all.

I think we all know that in any sampling plan other than 100% there is no assurance that record n+1 wouldn't be a bad one. That is a given.
 

Sidney Vianna

Post Responsibly
Staff member
Admin
#6
Audit sampling of management system processes basically needs to provide confidence that the sample REPRESENTS/IS REPRESENTATIVE the item(s) being assessed. In most cases, sampling is determined by (non-scientific) professional judgement of the auditor(s) at hand. However, the latest version of ISO 19011(2011) in Appendix B also talks about Statistical sampling

B.3.3 Statistical sampling
If the decision is made to use statistical sampling, the sampling plan should be based on the audit objectives and what is known about the characteristics of overall population from which the samples are to be taken.

— Statistical sampling design uses a sample selection process based on probability theory. Attribute-based sampling is used when there are only two possible sample outcomes for each sample (e.g. correct/incorrect or pass/fail). Variable-based sampling is used when the sample outcomes occur in a continuous range.

— The sampling plan should take into account whether the outcomes being examined are likely to be attribute-based or variable-based. For example, when evaluating conformance of completed forms to the requirements set out in a procedure, an attribute-based approach could be used. When examining the occurrence of food safety incidents or the number of security breaches, a variable-based approach would likely be more appropriate.
 

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#7
OK - Determination by "professional judgement". That's pretty open? :notme:
 

Ajit Basrur

Staff member
Admin
#8
From IMDRF (International Medical Device Regulators Forum)

Guidelines for Regulatory Auditing of Quality Management Systems of Medical Device Manufacturers ? Part 2: Regulatory Auditing Strategy -

http://www.imdrf.org/docs/ghtf/fina...-guidelines-for-regulatory-auditing-part2.doc

Auditors may select samples based on factors which are most likely to affect the safety. In planning quality management system audits (see also section 6.6 Audit Planning), auditors need to consider many factors (e.g., the scope of the audit, the classification of the medical device(s), the complexity of the medical device(s), the intended use, applicable regulatory requirements, results of prior audits, etc.). Within each subsystem, sampling may need to occur in order to evaluate the effective implementation of the particular subsystem (and related subsystems). Tables 1 and/or 2 depicted in Appendix 1 may be used in determining appropriate statistical sample sizes.
.
 

Attachments

Colin

Quite Involved in Discussions
#9
I think that the type of record/evidence you are looking at also dictates the sample size e.g. if you were checking some purchase orders you may take 5 or 6 examples whereas if you were checking construction project files you most likely won't have the (or the will) to check so many.

Remember also that we are auditing processes now so it may be that we take just a small number of samples and follow them through the system - it takes time.
 
#10
Colin has a good point - and another "dimension" can be the "scope" of the process records. By that I mean, not just a quantity, but the type of record. If we take purchasing, there may be raw materials purchased, services (calibration, maintenance,) and out sourced value-added processes (machining, painting) and each has it's own specific requirements. This adds further dimension to what is chosen - take 5 - 7 of each and you've got a pretty large task on your hands!

However, if you take a "risk based" approach, you'll spend less time looking at suppliers and products/services which have never been a problem, and focus on the new or changed supplier(s), or the new/changed product spec. etc and so on. Now that's a sample...
 
Thread starter Similar threads Forum Replies Date
M EU IVD Risk Determination of sample collection EU Medical Device Regulations 0
C AS9138 Sample Size Determination AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
J Interesting Discussion Sample Size Determination for Medical Device Other Medical Device and Orthopedic Related Topics 19
V When and How to apply the Bayes Success Run approach for Sample Size Determination Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
Statistical Steven Continuous Data Analysis Sample Size Determination Six Sigma 3
A Destructive Test Sample Size Determination Statistical Analysis Tools, Techniques and SPC 4
P Repetition (sample size) for measuring equipment accuracy determination General Measurement Device and Calibration Topics 1
K Sample Size Determination - Medical Devices Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
R Sample determination using minitab if everything elses is a variable Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
S Determination of sample size for product audit IATF 16949 - Automotive Quality Systems Standard 1
M ISO 14971 Determination of Competent Persons ISO 14971 - Medical Device Risk Management 4
U Does *anyone* know a lab that will test to EN 455-4 Medical Gloves shelf life determination? EU Medical Device Regulations 1
F Plastics cans shelf life determination under static load Manufacturing and Related Processes 1
C ISO/IEC 17021-1 clause 7.1.2 - Determination of competence criteria Document Control Systems, Procedures, Forms and Templates 2
D Standard guide for shelf life determination of non sterile component with degradation materials Reliability Analysis - Predictions, Testing and Standards 1
cscalise Determination of CPD (Continuing Professional Development) hours for a training course Training - Internal, External, Online and Distance Learning 1
T Determination of TENSION SAFETY FACTORS - Table 21 IEC 60601-1 Other Medical Device Regulations World-Wide 5
H Max and Min Load Determination for Steam Sterilization of Liquds EU Medical Device Regulations 1
O Determination of Test Cycles of Dynamic Forces (IEC 60601) IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
R Determination of IP (International Protection Marking) Rating IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
V Determination of the Optimal Sampling Number Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
J Root Cause Determination Training for Production Managers Nonconformance and Corrective Action 6
somashekar Determination of Clean Room Class for Medical Devices ISO 13485:2016 - Medical Device Quality Management Systems 1
S Shelf Life Determination - Sterility test (Stasis test) ISO 13485:2016 - Medical Device Quality Management Systems 12
G Determination of dental plaque disclosing products Other ISO and International Standards and European Regulations 1
A Determination of Production Equivalence for Design Validation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
kedarg6500 Determination of Metal Grade from known Chemical Composition Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
S Root Cause Determination of Epoxy Encapsulated Electronics Modules Nonconformance and Corrective Action 8
P Determination of Subgroup Size for P Charts Statistical Analysis Tools, Techniques and SPC 4
GStough 7.2.1 Determination of req'ts related to the Product...in Contract/Service Industry Service Industry Specific Topics 2
M 8.2.2 Internal Audit - Determination of Effective Implementation IATF 16949 - Automotive Quality Systems Standard 4
N Chemical Oxygen Demand (COD) Determination in Industrial Waste Water General Measurement Device and Calibration Topics 5
J How IAQG plans to ensure that correct PEAR level determination is done by Auditors? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
L Determination of AMS 2750D Furnace Classifications AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
sagai New Draft Guidance from FDA - Factors considered for Risk/Benefit Determination Other US Medical Device Regulations 4
M AS9100 Rev C Initial Certification - Determination of Competence AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
K Determination of Calibration Frequency by an Outside Laboratory Calibration Frequency (Interval) 5
S Medical Device Software Level of Concern Determination 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 23
E Differences between "Significant Change" determination - Canada vs. FDA Canada Medical Device Regulations 3
N Thermocouple Drift Determination - Drift test done by NIST General Measurement Device and Calibration Topics 3
N ISO 34: Rubber, vulcanized or thermoplastic - Determination of tear strength Other ISO and International Standards and European Regulations 1
T Determination of UMDNS Code for various types of Contact Lens ISO 13485:2016 - Medical Device Quality Management Systems 4
P SPICE (Software Process Improvement and Capability Determination) Certification Software Quality Assurance 3
6 Method for Determination of Amount of Air in Orange Juice Manufacturing and Related Processes 7
S Electronic Coordinate Determination System Inspection, Prints (Drawings), Testing, Sampling and Related Topics 21
S Criteria for Determination of Significant Aspect/Hazard in IMS Miscellaneous Environmental Standards and EMS Related Discussions 3
M Environmental Aspect Significance Determination and Scoring Miscellaneous Environmental Standards and EMS Related Discussions 6
E ISO14971 - Determination of possible hazards (SU, UU, PC) ISO 14971 - Medical Device Risk Management 3
J Documentation of the determination that an NCMR is invalid ISO 13485:2016 - Medical Device Quality Management Systems 4
R New Equipment Calibration and determination of Calibration Frequency General Measurement Device and Calibration Topics 8

Similar threads

Top Bottom