Statistical Analysis during Document Audit

[clipps]

I am trying to determine a significantly appropriate number of pieces of equipment to review during an audit of IMTE. The ANSI/ASQC Z1.4 applies to attributes and QSIT sampling using binomial staged sampling plans at 95% confidence limit is confusing. There are 106 total units and I would like to perform the audit on a representative sample. Any help would be appreciated.
Jeff

[Jim Wynne]

Originally Posted by clipps

I am trying to determine a significantly appropriate number of pieces of equipment to review during an audit of IMTE. The ANSI/ASQC Z1.4 applies to attributes and QSIT sampling using binomial staged sampling plans at 95% confidence limit is confusing. There are 106 total units and I would like to perform the audit on a representative sample. Any help would be appreciated.
Jeff

Welcome to the Cove, Jeff.

I'm confused. I'm guessing that IMTE is Inspection Measurement and Test Equipment, but your thread title indicates that you're doing a document audit. Can you tell us a little more about what you're trying to do? It might be difficult to define "representative" in a case like this.

[clipps]

Hello Jim, thanks for the response. I am auditing IMTE as you identified. We have 106 pieces and I am going to be auditing the actual equipment (labels, storage conditions, status, etc.) and associated files (records of cal, purchase history, rationale for accuracy limits, etc.), so a sample would be appropriate to determine compliance. A third-party audit is scheduled for January and I want to ensure that a statistically valid sample was chosen for the internal audit. A binomial approach seems to be the right path since the equipment and/or files are either compliant or not. Does this help?

[Jim Wynne]

Originally Posted by clipps

Hello Jim, thanks for the response. I am auditing IMTE as you identified. We have 106 pieces and I am going to be auditing the actual equipment (labels, storage conditions, status, etc.) and associated files (records of cal, purchase history, rationale for accuracy limits, etc.), so a sample would be appropriate to determine compliance. A third-party audit is scheduled for January and I want to ensure that a statistically valid sample was chosen for the internal audit. A binomial approach seems to be the right path since the equipment and/or files are either compliant or not. Does this help?

In anticipation of a third-party audit, what you should be trying to determine is whether or not there is evidence of systemic issues, not single nonconforming condition as might be the case in acceptance sampling. Finding a single instance (out of multiple "opportunities" for each device and file) isn't necessarily evidence of a system problem, so trying to use a statistical sampling approach probably won't be more helpful than just doing a random sample of eight or ten.

[Jennifer Kirley]

Jim is dead-on, as usual.

The sampling plans are based on achieving confidence that the lot is good when it's shipped, based on inspecting a (usually small) sample.

But while auditing is also verification to ensure goodness, Jim correctly points out it is about the process and system. Please view this thread: The art of auditing calibration. I think it gives good information.

I hope this helps!

[clipps]

Jennifer, thanks for the suggested thread. The areas identified in the threads as requiring good vigilence (e.g., supplier qualification) are noted.

Jim, I was trying to determine a statistically significant number of samples to inspect out of the total sample population in order to identify if a systemic issue was present. According to your suggestion, a "random sample of 8 or 10" would be sufficient. If I found two issues, would this be evidence that there was a systemic problem (20% rate)? During my last notified body audit the auditor wanted a rationale for how a sample number was determined.

In the FDA QSIT manual worldwideweb.fda.gov/ora/inspect_ref/igs/qsit/QSITGUIDE.PDF section entitled "Sampling Plans: Instructions & Tables" inspectors are given direction to use a binomial staged sampling plan to determine the number of records to review in identifying the "potential prevalence of the objectionable condition". I was using this as a starting point but hoping for a simpler, published method for determining sampling requirements that I could also cite as rationale.

Jeff

[Jennifer Kirley]

Jeff, I reiterate that the sampling and inspection you are referring to is the sampling and inspection of product.

Auditing is different because it is the examination of a process, not just a sampling to see if a group appears to be sound based on a narrow check of outcomes (inspection).

If I saw an example of one type of nonconformity, I would check further to see if there was any more. If I saw two of the same, I would look still further to determine if there was a breakdown in that aspect of the process.

If I found nonconformities in two different aspects of a calibration process, I would look further in each of these areas to see if there is a systemic breakdown.

If I were looking at the process for the first time, I would take a fair sized sample--maybe 10% or more, depending on the complexity and population size. If there were no problems, the next time I would probably look at fewer unless I found something amiss.

If I had any reason to believe there was something amiss before starting the internal audit, I would draw my sample size and focus to suit.

What I am trying to say is that the soundest method I can recommend is not an ANSI sampling table created for a controlled production process. I would instead make a methodical series of process examinations that verify the process is functioning and not just outcomes. How big a sample to take is largely intuitive.

I hope this helps!

[Jim Wynne]

Originally Posted by clipps

Jennifer, thanks for the suggested thread. The areas identified in the threads as requiring good vigilence (e.g., supplier qualification) are noted.

Jim, I was trying to determine a statistically significant number of samples to inspect out of the total sample population in order to identify if a systemic issue was present. According to your suggestion, a "random sample of 8 or 10" would be sufficient. If I found two issues, would this be evidence that there was a systemic problem (20% rate)? During my last notified body audit the auditor wanted a rationale for how a sample number was determined.

In the FDA QSIT manual worldwideweb.fda.gov/ora/inspect_ref/igs/qsit/QSITGUIDE.PDF section entitled "Sampling Plans: Instructions & Tables" inspectors are given direction to use a binomial staged sampling plan to determine the number of records to review in identifying the "potential prevalence of the objectionable condition". I was using this as a starting point but hoping for a simpler, published method for determining sampling requirements that I could also cite as rationale.

Jeff

I have to apologize for not paying attention to the forum this was posted in. I have no experience in medical stuff, so please disregard my earlier response. I will not that there is a rather disturbing footnote on page 105 of the document you linked to:

When at all possible, all samples should be chosen at random.

Samples must be chosen at random, or the tables are meaningless.