Random Sampling vs. Thorough Review - Records to be Reviewed - Audit Finding

P

Polly Pure Bread

#1
The audit is being carried out according to what was planned. The objective and scope was defined, identified auditees who have significant responsibilities were being interviewed, reference documents such as Procedures, SOP, WI, etc. were being verified against actual practice.

It was stated in the audit plan and audit notice that records to be reviewed are from February to August 2009. Auditor started gathering data and other evidences; he scrutinized the quality records thoroughly however had gone beyond the specified time frame to check consistent implementation, perhaps quality records of January 2009 and older. Auditee reacted and told the auditor that the records that should be audited are from Feb. to Aug. 2009 only, thus don’t include quality records beyond the specified timeframe. Auditee added that review should be done on a sampling not that going through each record one by one.

Who is correct the Auditor or the Auditee?
 
Elsmar Forum Sponsor

SteelMaiden

Super Moderator
Super Moderator
#2
Re: Random sampling vs thorough review

Why would you place date limitations on the review of records? I can see if the auditor was looking at stuff from 3 years ago, but I don't understand the Feb-August thing. Unless....there was a corrective action completed in there and everything from Feb was supposed to be done per the new corrective action? And even then, you'd want to look a ways back to make sure that things had changed, no? I'd say neither the auditor, nor the auditee was wrong, but the person creating the plan might be?
 
A

arios

#3
Re: Random sampling vs thorough review

Hola Amigo

I agree sampling should be more representative but since auditing is not a exact science there is always a likelihood that the auditor will focus on a certain fraction of a population. That may happen if the auditor sees a trend and wants to confirm or discard a possible finding or a trace to another area of interest within the QMS.

I cannot say if the auditor or the auditee are right or wrong, but since the auditor is the one who selects the sample he/she should be granted the freedom to take the sample that is considered convenient.

If the auditor selects an "older" sample and an issue arises in that sample which is not representative of newer improvements, you as auditee can claim that "improvements have been done since" and use them to try to make the auditor reconsider or even to explain in your response (if raised) what was done to improve the area of concern.

Alberto
 
P

Polly Pure Bread

#4
Re: Random sampling vs thorough review

Why would you place date limitations on the review of records? I can see if the auditor was looking at stuff from 3 years ago, but I don't understand the Feb-August thing.

Apart from monthly in-house audit, we also conduct full QMS Internal Quality Audit twice a year and the January 2009 and older records were covered in the previous audit.

It’s possible that there are issues that were reported previously. Auditee said that reviewing previously audited records would over kill them especially the audit team has difficulty in tracking previous audits since our documentation is still hardcopy format.
 
T

Tony C

#5
Re: Random sampling vs thorough review

Apart from monthly in-house audit, we also conduct full QMS Internal Quality Audit twice a year and the January 2009 and older records were covered in the previous audit.

It’s possible that there are issues that were reported previously. Auditee said that reviewing previously audited records would over kill them especially the audit team has difficulty in tracking previous audits since our documentation is still hardcopy format.
Hi dQApprentice

I wouldn't want to be auditing old records again but I would want to be following up on any non-conformances identified and corrective actions from the previous audit.

Regards,

Tony :)
 
A

arios

#6
I still believe that the auditor may have valid reasons to select older samples for review. It's completely up to the auditor and he or she should be afforded the opportunity to review what it is felt as appropriate.

It may not make sense to everybody why the auditor chooses a certain part of the population, but it is completely valid provided that he asks for records thart are part of those within the defined retention time.

I can list various reasons that can prompt me as auditor to assess along a period of time or just a section of the population. That's why is important for an auditor to have experience so he can understand that there exist advantages or disadvantages for doing one way or the other. If the auditor ask you for records that were created three years ago, then he must expect the records to take a little longer to be retrieved and those may not be necessarily representative or present activities, if on the other hand he just chooses records from a given month then he must realize that he may be making a conclusion just based on that period of time, at the end the system will be challenged one way or the other.

Bottom line: it's a random sample. Stratified sampling is better but may not always be used
 
Last edited by a moderator:
P

Polly Pure Bread

#7
Re: Random sampling vs thorough review

Hi dQApprentice
I wouldn't want to be auditing old records again but I would want to be following up on any non-conformances identified and corrective actions from the previous audit.
Thanks SteelMaiden, Arios (amigo) and Tony

Yes, we do verify correction of previously identified deficiencies. I understand that relevant records should be reviewed for completeness and accuracy as well as for conformance to the requirements as applicable. It just that the QMR and/or Audit Team Lead notified the auditee, they requested records (Feb. to Aug. 2009) that need to be ready.
 
T

Tony C

#8
I still believe that the auditor may have valid reasons to select older samples for review. It's completely up to the auditor and he or she should be afforded the opportunity to review what it is felt as appropriate.

It may not make sense to everybody why the auditor chooses a certain part of the population, but it is completely valid provided that he asks for records thart are part of those within the defined retention time.
:confused:

I fail to see the usefulness of this especially if they have already been audited other than to prove they have been retained.

Regards

Tony
 
J

JaneB

#9
I still believe that the auditor may have valid reasons to select older samples for review. It's completely up to the auditor and he or she should be afforded the opportunity to review what it is felt as appropriate.
I agree with you to some extent and only within reason.

If the auditor is testing something to do with record retention, and there has been a problem with it in the past, so they want to see records from 2 or 3 or whatever years ago to test that, OK. Or if they have to store records as a contractual/statutory/regulatory requirement, perhaps yes then also.

But if not, why? Quite often there's just zero point in going back a long way: the company has moved on/changed/improved things, so auditing to then report the incredibly unuseful finding that 'you weren't following your brand new records retention procedure' 2 years ago provides what value?
 
J

JaneB

#10
It was stated in the audit plan and audit notice that records to be reviewed are from February to August 2009. Auditor started gathering data and other evidences; he scrutinized the quality records thoroughly however had gone beyond the specified time frame to check consistent implementation, perhaps quality records of January 2009 and older.
Depends if there was a valid reason in the records he looked at to then follow up a particular line of questioning or not. If there was, the auditor has every right to. But if not, and if the plan really said 'we will look at X but the auditor looked at X + Y' I can understand this could well annoy the auditee.

Auditee added that review should be done on a sampling not that going through each record one by one.
Sampling is normal practice... but an auditee doesn't really have the right to tell the auditor how to audit, and I wasn't there to know the details.

Who is correct the Auditor or the Auditee?
Again, not enough information supplied to answer.

However, what does sound clear is that this is not an audit that went well. And it's definitely a good idea to do some corrective action to prevent recurrence.

:topic:
dQApprentice, I don't know why, but your posts are always full of superfluous HTML font and Verdana tags. I don't know why, but they're irritating, because I have to clear them out of the way in order to respond, which is time-consuming, so I'm getting less inclined to respond to your posts because of this. Any way you could get rid of them yourself?
 
Thread starter Similar threads Forum Replies Date
M Stratified Random Sampling Calculations Statistical Analysis Tools, Techniques and SPC 1
A Random Sampling Statistical Analysis Tools, Techniques and SPC 10
G Sampling Question - Random Sample of 100 pcs and 11 pcs. have been rejected Inspection, Prints (Drawings), Testing, Sampling and Related Topics 30
S Random Sampling of Chickens Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
P Random Sampling at Receiving Inspection: A Practical Implementation needed Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
D How to perform Random Sampling Statement Inspection, Prints (Drawings), Testing, Sampling and Related Topics 10
A Random Sampling for in-process inspection Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
S Random Sampling question - taking samples from the same spot! Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
S Using AQL to get true random sampling using General Inspection Level II AQL - Acceptable Quality Level 4
Ron Rompen Random data generation in Minitab Statistical Analysis Tools, Techniques and SPC 2
L Do all non-random patterns apply to attributes and variable charts including R chart? Statistical Analysis Tools, Techniques and SPC 5
S Probability Density Function of a Function of a Random Variable Statistical Analysis Tools, Techniques and SPC 13
I Multiple Comparisons for a Random Factor Using Minitab Software 2
P Individuals Chart for Non-Sequential Data - Random Sample without Reference to Time Statistical Analysis Tools, Techniques and SPC 4
H Part Number Assignment, Should be coded or Random? Manufacturing and Related Processes 11
Marc Solving "Random" Puzzles Key To Scoring A-List Tech Jobs Career and Occupation Discussions 8
Ajit Basrur Random Insult Generator (RIG) Coffee Break and Water Cooler Discussions 2
M Random 'Normal Distribution' Numbers in Excel Excel .xls Spreadsheet Templates and Tools 26
M Creating a "random disturbance" for each observation via MINITAB Using Minitab Software 1
C Random Uncertainties - Can anyone explain "Random Uncertainties"? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
Marc Unannounced Random Audits General Auditing Discussions 1
J IVD sampling sizes for buyer Reliability Analysis - Predictions, Testing and Standards 0
S Which Sampling Plan(s) Should I Use? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
C Sampling - ISO 16269-6 vs ISO 2859 Other ISO and International Standards and European Regulations 0
L Sampling Plan Risks AQL - Acceptable Quality Level 6
A Sampling plan for in-process QC (medical devices) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
M Minitab Capability of the Population (no sampling) Using Minitab Software 11
W LTPD, AQL, Ppk and Cpk validation sampling plan table Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
Q Sampling method for Distributor verification activities EU Medical Device Regulations 1
E Change in control plan - Do I have to do sampling? IATF 16949 - Automotive Quality Systems Standard 1
S Is using ANSI/ASQ Z1.4-2008 the correct sampling plan to determine Pass/Fail of Apparel measurements? AQL - Acceptable Quality Level 4
N Sampling Plan for Internal Audits - ISO 2859 or 3951 - Or Neither? Internal Auditing 6
F Product audit sampling plans IATF 16949 - Automotive Quality Systems Standard 3
R Reduced sampling plan for sterial products APQP and PPAP 0
A ISO 2859-1 - 9.3.3.2 Switching score - Sampling Plans and PPM AQL - Acceptable Quality Level 0
R BS EN 12060:1998 - Zinc and zinc alloys Method of sampling Specifications Manufacturing and Related Processes 4
S High voltage testing - ISO 17025 - 7.2.2 Validation of methods and 7.3 Sampling ISO 17025 related Discussions 3
8 Activity Sampling Exercise Lean in Manufacturing and Service Industries 11
T Defining sampling plan for different AQL AQL - Acceptable Quality Level 3
B AS9100D Sampling Plan AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
R Design verification for interim design outputs - sampling rationale ISO 13485:2016 - Medical Device Quality Management Systems 2
Nicole Desouza Sampling plan for a customer who wants AQL 1.0 (per ANSI Z1.4) AQL - Acceptable Quality Level 5
V How to interpret AQL sampling tables AQL - Acceptable Quality Level 5
A AQL sampling bags of parts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
C Applying Weibayes to Attribute sampling Reliability Analysis - Predictions, Testing and Standards 0
I Sampling processes - Who must define the AQL level? AQL - Acceptable Quality Level 9
E Sampling for capability studies for variables and attributes Capability, Accuracy and Stability - Processes, Machines, etc. 4
O Statistical justification of sampling size in V&V tests ISO 13485:2016 - Medical Device Quality Management Systems 5
M Sampling Plan for Alumin High Pressure Die Castings Manufacturing and Related Processes 0
E Sampling plan for orthopedic implant - Process Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4

Similar threads

Top Bottom