The use of Statistics at a Medical Device Manufacturer

[Statistical Steven]

Steven, can you point us to a reference or other source for that FDA preference? I've wondered about our use of Z1.4, but I haven't previously been able to find any such guidance or statement of policy.

Not sure there is a guidance document. I am basing off my experience with clients who have received warning letters, 483s and non-approvable letters from the agency for inadequate sample size justification. I would search the FDA website to see if there are specific citations in the FOI section. It is a response to the Q8, Q9 and Q10 documents from ICH.

 

[dolphinix]

Hi Steven, et al,

Although this post is old it seems to me that is still a very interesting and actual subject to be expanded on.

* Did you ever found some information regarding this statement of not using ANSI for validation?? one question that came up is, Is the use of ANSI a valid tool for the PQ stage? we currently use a different sampling rate for validation (OQ), but we come to think that the use of ANSI is valid for PQ.

* what would be the expectations from an auditors point of view regarding the use of statistical tools as per CFR 820.250?

 

[Statistical Steven]

820.250 specifically talks about sampling plans which requires a sample size and a accept/reject number. The FDA wants to know for any sampling plan used for validation, acceptance or other statistical activities, what are the risks, both Type I and Type II and how it fits your risk management plan.

 

[Bev D]

the answer lies not in guidance documents but in the fundamental nature of "statistical studies"

Acceptance sampling (including Z1.4) is an enumerative study: we want to have some assurance that the lot in front of us has a certain quality level. Sample size is one dimensional - there is a single sample taken randomly from the lot. The sampel size is based solely on the defect rate we want to allow (accept) 95% of the time or to not allow (reject) 95% (or 90%) of the time. (remember the AQL level is the defect rate we will accpet - if it exists - 95% of the time. In validation studies, we (and the FDA) are more interested in understanding what the defect rate will actually be in future lots.

Validation studies are analytic studies; they are used for prediction of future of future performance. The sample size for validation studies is more complex than for acceptance sampling. We must first consider what difference we want to detect. We also must include the expected defect rate. We must consider replication across varying conditions and true independence of the samples created. In enumerative studies the structure of the study is more critical than the actaul sampel size.

From the above we can also see that so called type I and type II errors have different meanings in an enumerative study than in an analytic study.

 

[Jose Ch]

It's been hard to find an written statistical justification for the classical 3 operators x 3 trials each for 10 parts in GR&R.
Any enlightenment here would be of great apreciation.

 

[Statistical Steven]

There is no statistical justification for the 3 x 3 x 10 trial (n=90) for continuous measurements without understanding the expected uncertainty, and sources of the uncertainty (between and withing assessors). For attribute studies, you can get estimate confidence widths based on the number of samples and concordance rates.

 

[Bev D]

Steven is correct. there is no statistical justification for this study design.
There was a paper written by Robert Traver in 1959 expounding on General Motor's approach. It is difficult to find and doesn't really justify the approach but it does explain it somewhat. the paper:

"Measuring Equipment Repeatability - The Rubber Ruler" Published in the Annual Convention Transactions, 1962 American Society for Quality Control. It may be available from the ASQ archives.

A much better resource is one of Donald Wheeler's articles on the topic: (these are free)

Wheeler, Donald J., Craig Award Paper, “Problems With Gauge R&R Studies”, 46th Annual Quality Congress, May 1992, Nashville TN, pp. 179-185.

Wheeler, Donald, Problems with Gauge R&R Studies, reprint, Quality Digest January 2011

Also try:

Donald S. Ermer and Robin Yang E-Hok, “Reliable data is an Important Commodity”, The Standard, ASQ Measurement Society Newsletter, Winter 1997, pp. 15-30.

Ermer, Donald S., “Improved Gauge R&R Measurement Studies (Part 1)”, Quality Progress, March 2006

One of the better original papers:
Youden, William John, “Graphical Diagnosis of Interlaboratory Test Results”, Industrial Quality Control, May 1959, Vol. 15, No. 11