Auditing Statistically - A statistical approach to sample sizes

[M Greenaway]

This topic could have gone under the statistics heading - but anyway.

Various discussions have talked about statistics and variance of processes, and i'm sure we all have an appreciation of what it is all about.

Therefore when we conduct an audit, which is a method of 'measurement' like any other (maybe) should we adopt a statisitical approach to sample sizes, plotting and interpretation of data (findings). i.e. just because we find an NC in an audit couldnt it just be that this is a common cause of variance in the process ?

Or am I way off beam ?

 

[db]

Common Cause NC?

Quote:
-----------------------------------------------------------
Originally posted by Greenway
"just because we find an NC in an audit couldnt it just be that this is a common cause of variance in the process ? "
------------------------------------------------------------

Yes, it could. In fact I suspect a lot NCs are. That could explain repeating nonconformances. The problem may come in the CA. I do not know an auditor (internal/external) that would allow a CA to come to the conclusion that the NC was common cause variation and that no action would be taken. We tend to assume that NCs are special cause variation and can be corrected. This is another case of some good mental chewing material. Thanks I know I’m not going to get much work done today!

 

[JRKH]

If we're going to conduct QMS measurment statistically,
how would we conduct a MSA on auditing? Don't we have to proof the measurment system first?

James

 

[Laura M]

Interesting question.

I would think that the requirements of internal auditing based on the relative importance of an activity and results of previous audits is the basis for anything statistical. P-chart of N/C's by clause, department, can drive decisions. Of course there is measurement system variation, isn't that what has been extensively discussed in other threads regarding certification audits?

But of course - wouldn't common cause = minor and special cause = major? Well at least a special cause in the negative direction - multiple NC's for a given requirement or in a given department.
They may have not had statistics in mind when they were named minor/major - but it correlates in my mind.

Laura

 

[M Greenaway]

Excellant twist

James

Excellant twist to the original post - yes I think we certainly should proof the measurement system first. Applying MSA methodology sounds good - dont know how it could be applied in practice.

Laura - it sounds like a good correlation between common causes, special causes and minor, major findings. A statistical approach would enable the auditor to classify his findings as minor/major with much greater accuracy, as at present it can be quite subjective.

I think we could be on to something here !

 

[Atul Khandekar]

Twist !

Applying MSA methodology sounds good - dont know how it could be applied in practice.

Maybe have 3 auditors conduct the audit twice each?

Is there any sort of consensus as to what constitues a minor NC and what makes a major NC? How do you remove the operator (auditor) bias?

 

[M Greenaway]

Its not beyonf possibility that you could set up a controlled situation that your various auditors have to audit, and then compare results.

Removing bias could be done through additional training to the group so that a common understanding exists between them all.

The more I think about it the better it sounds !

 

[Al Dyer]

Iv'e seen audit results succesfully reported using a pareto chart for guidence as long as all NC's have a C.A.

When thought about, all internal audits are controlled statistically because the results need to be reported to management which in turn uses the results to increase or decrease audit frequencies which is basic sampling..

Hope I made sense, the codeine hasn't kicked in yet!

 

[M Greenaway]

Al

Do you think that the current method of auditing, reporting results and determining corrective actions really has any basis on statistcal theory and causes of variation ?

Reviewing audit results to determine whether to increase or decrease audit frequency is only effective if the audit results themselves are based on statistical analysis of the causes of variation.

 

[Al Dyer]

Originally posted by M Greenaway
Al

Do you think that the current method of auditing, reporting results and determining corrective actions really has any basis on statistcal theory and causes of variation ?

Reviewing audit results to determine whether to increase or decrease audit frequency is only effective if the audit results themselves are based on statistical analysis of the causes of variation.

-----------------------------------------------------------------

1a: Statistical theory: Yes, but only rudimentary not to the point of D.O.E.
1b: Causes of variation: See below.

2: I don't see how the cause of a nonconformance in an internal audit has any affect on the +/- frequency of an audit. The causes would come up during the actual corrective action. If there are nonconformances there is increased frequency, more-so with ongoing nonconformances. And if there are ongoing nonconformances it "probably" (depending on the structure if the audit program) is not caused by the audit system but a faulty C/A system.

I believe it comes down to cost effectiveness as to how intricate the internal audit system is linked to statistical methodology.