Audit Sampling methods - Discovery Sampling Defined and Discussed

[Govind]

I came across with this topic called "Discovery Sampling" in the ASQ CQA Body of Knowledge in the past. Also I saw again today in another (financial) audit related article. This triggered few questions in my mind. I decided to put infront of this knowledgeable Quality community and ask for their feedback.

Discovery Sampling is defined as:
" a procedure for determining the sample size required to have a stipulated probability of observing at least one occurrence when the expected population deviation rate is at a designated level."

My questions are:
1. Do you sample more than one document, record in a given process while auditing?
2. Do you random sample documents and records based on " a scientific sampling Plan" ?
3. Have you ever used this "discovery Sampling" method for your Quality System auditing?
4. If anyone using this approach, any pros and cons? practical? cost effective?

Iam not aware of any registrar using this approach.

Since there is no thread on this subject from this forum, I would appreciate your feedback on this topic.

Thanks,
Govind.

[C Emmons]

ok, I will bite. I am in the transportation industry, and the internal auditor for 19 facilities. I have several different documents which are sampled and in some cases the same document is audited to check different processes. I have found that be setting my sample size as a "days worth" it allows me to get a good across the board sample. For example: Our drivers are required to turn in confirmation of their deliveries at the end of the day. A "Delivery Receipt". This receipt serves as a clear inspection by the customer, time of arrival, departure, pcs delivered etc. By sampling a days worth it allows me to view a sample of all drivers at a particular site and discover if there may be an issue related to just one specfic driver etc.

[Randy]

The ASQ dribble above is just that...dribble.

You need to base your sampling (we're talking ISO system auditing here) on being able to provide the objective evidence of conformance to whatever the applicable requirement is and not to identify a pre-concieved/pre-determined deviation. Good gosh...we should never go into an audit with a pre-disposition for finding the deviations, we look for conformance. We're not looking for "scientific sampling" (a horrendous process at best). Sampling has to take into consideration things like the size of the organization, the volume of documentation available, the time available, the scope of the audit and others. In the end adequate sampling is going to have to depend upon the "competence" of the auditor and the physical factors of the audit itself.

[The Taz!]

Thanx Randy. . . I was biting my tongue reading this thread. . .

and I think the right word is Drivel

[Rob Nix]

Arrrrrggghhhh!

You took the words right out of my mouth, Randy. I'll bet the framers of "discovery sampling" never did an hour of auditing in their lives. The expertise of the auditor is what determines how much documentation is audited. There are TOO MANY VARIABLES to establish "sampling" rules. Sigh!

Like Taz repeated, DRIVEL!

[RCBeyette]

Like Taz and Rob said, you took the words right out of my mouth, Randy! I had to read this twice and adjust the contact lenses to make sure I wasn't mis-reading.

Originally Posted by Govind

1. Do you sample more than one document, record in a given process while auditing?

Yes and yes. To sample just one means nothing...depending on the item and time constraints, three to five samples of a document (or a record) in an area tends to be my norm.

Originally Posted by Govind

2. Do you random sample documents and records based on " a scientific sampling Plan" ?

Please clarify "scientific sampling plan." I mean, it's not like I thow suction darts at the monitor to determine which documents/records to look at, but I also do not allow the auditee to select the documents/records for me to look at...I'm don't play the "pick the best and hide the rest" game with people.

Originally Posted by Govind

3. Have you ever used this "discovery Sampling" method for your Quality System auditing?
4. If anyone using this approach, any pros and cons? practical? cost effective?

No, I have not used and if it is truly meant for an auditor to keep digging until they find evidence of a nonconformity, I will not use it.

Pro - Theoretically, I guess, over time there would never be any discrepancies in the system.
Con - Digging and digging and digging....doesn't make audits very auditee-friendly now, does it?
Pro - Ummm....
Con - Reactive instead of proactive
Pro - Still thinking...
Con - When does it end? What if you've been sampling documents for 3 hours and found nothing...do you keep going?
Pro - *gone to get a coffee*
Con - Okay...you've reviewed 763 documents. One of them had a mismatching Revision Number and corresponding hard copy. Now what? Do you honestly issue a finding 0.13% of nonconforming documentation? Is this a value-added activity to do so?

Practical - No...all of the time and resources focusing on finding errors instead of improvement.
Cost Effective - see above

Govind, I hope we don't sound like we're bashing the idea (okay...maybe we are saying our opinions much stronger than usual), but based on the definition of discovery sampling has us auditing to discover problems instead of auditing to discover ways of improvement.

Are we misinterpreting the defintion?

[J Oliphant]

I trust my intuition a little more than this.

why- Your purpose is to do the efficient survey of the organization's compliance to requirements. As an auditor, it always seemed that possible issues betrayed themselves in little unscientific (and sometimes unrecordable) observations. A book that is unorganized, things that aren't signed, things that the auditees didn't seem to bring up or remember when asked basic questions about procedures. one minor defect, can lead to another.

now with your sampling technique, you'd spend a non-value added time trying to discover if the issue is signficant rather than going from one issue to another trying to characterize the breakdown on quality.

The registrars auditors all seemed gifted at sifting for clues and part of there rigorousness was the fact that they dived beyond the initial offerings of procedure and manual that was undoubtably perfected by QA. That is to my mind their power.

Dennis arter called them "sniffs" and advocated that auditors spend and unscientifically large amount of time looking at documentation that may indicate the larger problem.

I think that sampling plans would then make auditors overall, less penetrating, enlightening and valuable.

Such a sampling plan might be more useful in analyzing your own dept.

[Al Dyer]

Auditors are supposed to be trained and certified. That said, auditors are people that need to have experience in some type of business setting. There are many here that have "general" business sense and know when they are being given a line of bologna and asked to swallow geese!!!

An auditor uses his/her accumulated knowledge to ascertain whether a company is following the system that "THEY" initiated.

In my 23 odd years I have only conflicted with 1 auditor, and he was not happy when he gave our company a "major" for an unrsesolved CA. He was upset because I brought up the fact that he was the person that closed the CA and then tried to give us the gig (in front of the entire management team).

Oh well,
Al...