Which Normality Test more acceptable to FDA; Also, Non-Normal Threshold?

S

snakepitt

We are in the process of determining samples sizes for various tests to support an FDA submission. Prior data are being analyzed to do this, and also being checked using Minitab for normality. Two questions:

1. Is one of the three tests (Anderson-Darling, Ryan-Joiner [similar to Shapiro-Wilk], Kolmogorov-Smirnov) more acceptable to FDA than the others? I have found some background on each that speaks to application but it did not help me. I did find a thread in the QA forum from one "Statistical Steven" several years back in which he commented that he prefers Shapiro-Wilk for sample sizes greater than ~50 and Kolmogorov-Smirnov for smaller Ns. Does anyone have additional input re FDA preference? What about A-D?

2. Between "non-normal" threshold of P=0.050 and P=0.100, is one more accepted than the other?

Thanks for the help.

Miner

Forum Moderator
Staff member
My background is not FDA, but is in statistics. So I cannot advise what is acceptable to the FDA. I will however explain some of the differences between your options.

These articles explain the basis for each of the three tests. The Anderson-Darling and Ryan-Joiner tests are equally powerful. Power = 1 - Beta Risk. The more powerful the test the less likely that you will miss a truly non-normal distribution. The Kolmogorov-Smirnov test has fallen out of favor in many (though not all) circles, primarily because it is less powerful than the other two tests. This means it is more likely to miss a truly non-normal distribution. If I were to guess, I would say that the FDA would be more likely to accept the first two tests, but as I said, I am not an FDA expert.
• The threshold is properly called Alpha. The p-value is then compared to alpha to make a decision on whether to reject the null hypothesis. Alpha is the risk that you are willing to accept that you may incorrectly reject the null hypothesis. In the case of a normality test, the null hypothesis is that the data comes from a normally distributed population. An alpha risk of 0.10 means that you are willing to take a 10% risk of saying the data did not come from a normally distributed population when, in fact, it did come from a normally distributed population. An alpha risk of 0.05 means you are willing to take a 5% risk of saying the same.
This is a decision best made by determining the consequence of your making an incorrect decision. If that consequence is low, you may decide on an alpha level of 0.10. If that consequence is moderate, go with 0.05. If it is high, you may even want to go with 0.01.

B

Barbara B

In addition to Miner's excellent posting: My customers in the medical devices area work usually with alpha=0.05 as a default value. This wasn't questioned by the FDA in the past.

Regards,

Barbara

S

snakepitt

Thanks to both of you, Miner and Barbara B, for the detailed and rapid replies. I had run across the Ryan-Joiner paper in early searches but the equations intimidated me from really reading through - this time I did.

Thinking some about the alpha level, it seems to make sense that selection of a level could dovetail with one's risk assessment process, with maybe 0.100 linked to the more benign issues while at the other end of the spectrum critical ones might require 0.010; and 0.050 for those in between.

Thanks again.

Bev D

Heretical Statistician
Staff member
Super Moderator
In general, each FDA statistician or statistical goup will have their own ideas about best practices. I have foudn that establishing a relationship with your statistical group at the FDA to be the best way to go. Typically (depending on where your product is the 'food chain') the statistical group will also prefer SAS or R analysis not Minitab or JMP. A decent 'omnibus' test for Normality that is generally accepted by the FDA (and all cautions about the uniqueness of your specific situation apply) is covered in: "A suggestion for Using Powerful and Informative Tests of Normality" by Raph D'Agostino, Albert Belanger and Ralph D'Agostino Jr. published as a commentary in Teh American Statistician, November 1990, Vol. 44 No 4.

O Chi Square Normality Test using Microsoft Excel Excel .xls Spreadsheet Templates and Tools 2
S Normality Test Chart Title Information in Minitab 15 - Label Modifications Using Minitab Software 2
E Run Chart in Minitab - Can I use Run Chart for Normality Test in Minitab Statistical Analysis Tools, Techniques and SPC 6
M Macro that will perform a normality test and capability analysis - Minitab help Using Minitab Software 1
D Normality Test Before the Xbar R Diagram is Made? Statistical Analysis Tools, Techniques and SPC 5
D Normal Distribution - How do I test for normality? All the data or just the averages? Statistical Analysis Tools, Techniques and SPC 8
K Normality Assumption - Most Tests 'Assume' a Normal Distribution - t test statistic Statistical Analysis Tools, Techniques and SPC 5
Transformation of Data Normality Failed Using Minitab Software 11
J What can I do to give the Pp and Cp when data is Non normality data Quality Tools, Improvement and Analysis 11
C Data Normality using Anderson Darling (Template) Document Control Systems, Procedures, Forms and Templates 1
J Checking Variance/Normality of my two Samples (question) Using Minitab Software 2
Normality of Data Question - Minitab Analysis Statistical Analysis Tools, Techniques and SPC 12
M Minitab 16 doesn't like perfect data for Cpk & Normality? Using Minitab Software 5
Normality, Tail Probabilities, and SPC Statistical Analysis Tools, Techniques and SPC 3
William A. Levinson speaks on When the Bell Curve Doesn?t Fit, Part 2: Non-Normality Statistical Analysis Tools, Techniques and SPC 0
R Testing Data for Normality - Need help on my essay Statistical Analysis Tools, Techniques and SPC 9
M Plain English Explanation of when to use which Normality Tests Quality Tools, Improvement and Analysis 7
B Minitab: Understanding Normality and What Chart To Use. Using Minitab Software 6
L Capability Calculation/Error associated to faulty normality distribution assumption Statistical Analysis Tools, Techniques and SPC 2
K Normality of distribution prior to Cpk and PpK Statistical Analysis Tools, Techniques and SPC 4
S Capabilty indicators in case of non normality of a distribution Statistical Analysis Tools, Techniques and SPC 5
S Should we check the normality of measurements when the calculation of Gage RR is done Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
Test Method Validation ISO 13485:2016 - Medical Device Quality Management Systems 6
Test compulsory for Sutures? Manufacturing and Related Processes 6
Informational DRAFT: The Medical Devices (Coronavirus Test Device Approvals) (Amendment) Regulations 2021 UK Medical Device Regulations 0
Identification test in ph.eur. Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 1
ISO-17025 accredited TEST lab performing internal calibrations (Yes: Here is how) ISO 17025 related Discussions 6
Cost per test tool/formula Oil and Gas Industry Standards and Regulations 1
Cost per test tool/formula Oil and Gas Industry Standards and Regulations 1
R chart from ANOVA gage R&R test Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
Ground Bond Test for Class I Medical Electrical Equipment - calibration problems IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
Burst Test Acceptability Criteria ISO 13485:2016 - Medical Device Quality Management Systems 2
60601-1 clause 8.11.3.6 Test Clarification IEC 60601 - Medical Electrical Equipment Safety Standards Series 0
Test report to certify compliance with IEC 62304 IEC 62304 - Medical Device Software Life Cycle Processes 3
Tensile test machine, tinius olsen 602 Manufacturing and Related Processes 1
Variations between ASTM A29 Standard for steel bars and Mill test certificates specs Manufacturing and Related Processes 1
Tip test 10° -> 5° IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
Sponsor name presented in the test report EU Medical Device Regulations 4
Packing Validation & Accelerated Aging Test ISO 13485:2016 - Medical Device Quality Management Systems 2
OEM full range calibrations vs Validation special test points Medical Device and FDA Regulations and Standards News 1
IEC 60601 - 8.8.3 Dielectric Strength test. 4kv being applied to the ground conductor?! IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
Shelf Life Test for IFU and Implant Card Other Medical Device Related Standards 1
Mechanical Test Under FDA Freedom of Information Act Medical Device and FDA Regulations and Standards News 5
Critical characteristic on manufacturing operations test procedure Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
IEC 60601-1 Clause 15.3.2, Push test IEC 60601 - Medical Electrical Equipment Safety Standards Series 0
Knee Implant (Femoral -Cobalt chrome)-Sub chronic toxicity test (ISO 10993-11)choice of root Medical Device and FDA Regulations and Standards News 2
Defining a lower ESD test level in IEC 60601 safety test IEC 60601 - Medical Electrical Equipment Safety Standards Series 5
Is there a need for clinical test of Class IIa products (for MDR)? EU Medical Device Regulations 2
Is calibration of test weight required General Measurement Device and Calibration Topics 4