Acceptance-Support Testing?

#1
I am reviewing procedures at a new organization that reference accept-support testing methodolgies. It appears it is based off null hypothesis testing. Here is an excerpt:

Accept-Support (AS) Testing Methodology: In AS Testing, the null hypothesis (H0) is the condition that the tester is attempting to support. For (organization), the condition is: the medical device(s) under evaluation meet(s) the defined requirements.

Accepting the null hypothesis supports that the aforementioned condition is true (synonymous with saying the device meets its requirements). Rejecting the null hypothesis affirms that the aforementioned condition is false (synonymous with saying the device does not meet its defined requirements). Type I Error (α) is the error in rejecting the null hypothesis when it is true and is known as manufacturer’s error. Type II Error (β) is the error in accepting the null hypothesis when it is false and is known as consumer’s error. Table 1 displays the AS Testing hypothesis matrix.

This verbiage is contradictory to everything that I understand about hypothesis testing, which it seems to be based on. You cannot prove a null hypothesis to be true. You can only gather information to reject it, or fail to reject it. So how could the verbiage in the procedure be correct. Is Accept-Support testing the same? Has anyone heard of this?

A quick google search reveals limited results, the most prominent being a page that contains a very simple definition:
In this type of statistical test, the statistical null hypothesis is the hypothesis that, if true, supports the experimenter’s theoretical hypothesis. Consequently, in AS testing, the experimenter would prefer not to obtain "statistical significance." In AS testing, accepting the null hypothesis supports the experimenter’s theoretical hypothesis.
(Google search: Accept-Support)

This definition appears to support the procedure, but also contradicts my personal knowledge-- PLEASE HELP!

Thanks!
 
Elsmar Forum Sponsor

Marc

Hunkered Down for the Duration
Staff member
Admin
#2
A quick "Bump" - My Thanks in advance to anyone who will help with this one.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#3
I recommend that you read: The Null Ritual What You Always Wanted to Know About Significance Testing but Were Afraid to Ask. (it's free)

A couple of comments:

First I've never found the null hypothesis / alternative hypothesis thing to be particularly useful; it certainly isn't simple or clear. adn it isn't necessary. at all. I don't use it, nor do I teach it. I teach my organization how to determine if a factor causes a change, or if a deesign meets requiremetns and with how much margin. There is SO much more to a study design that will yield conclusions that one can rely on than the null hypothesis.

From my understanding the null hypothesis is generally taken to be that 'no difference exists'. so in the case of determining if requirements are met the null hypothesis would be that the device performance is not different from the requirements. as for the idea of not proving the null hypothesis, there are two considerations to think about: there really will be no situation where two things are exactly the same or that a thing exactly meets a goal or requirement. So the method doesn't use the 'prove' word it uses the 'accept' word. there will always be some difference, the real question is whether or not the difference is of any practical importance. And since there is always some uncertainy that uncertainty is codified in the lack of the use of the term 'prove'. Of course while this is of grave importance to some theoreticians; it is wholly unuseful to people trying to do work and make decisions in the practical world. and so real people will use the term 'prove'. It is true that the null hypothesis can be what the experimenter theorizes - that a difference exists; it is simply convention that the null is that no difference exists.

As a side note: the whole idea of the p value and null hypothesis is really in support of the use of small sampel sizes without real replication true confidence and 'proof' come from replication. replication is the cornerstone of all science and cannot be circumvented by desire for a 'quick' answer.
 
#4
I'd like to thank everyone for their input!

Ultimately I have reached the conclusion that the author misunderstood null hypothesis testing and the test is actually flawed. Null hypothesis testing is designed to identify if there is sufficient evidence to reject the null hypothesis. In these cases, the null hypothesis is that there is no significant difference (or some variant). It appears the author did not understand this aspect of null hypothesis testing and decided that if you can reject the null hypothesis, why not make the null hypothesis that you "did not meet the acceptance criteria". Then you can reject that and conclude that you have met the acceptance criteria... *sigh*.:blowup:
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#5
this is one of the primary problems with the whole 'null/alternative hypothesi's approach; its' just mumbo-jumbo.

I can't say they did the test incorrectly or that they do not meet the acceptance criteria until I understand the study design and see the data...neither should you...
 
Thread starter Similar threads Forum Replies Date
R Procedure, Frequency and Acceptance Criteria for Replicate, Recalibration, Before-After and Intermediate Checks ISO 17025 related Discussions 8
M Informational US FDA Final Guidance – Acceptance Review for De Novo Classification Requests Medical Device and FDA Regulations and Standards News 1
S Boeing D6-51991 Product Acceptance Software certificate and Artefact? AS9100, IAQG 9100, Nadcap and related Aerospace Standards and Requirements 0
W Customer Signature / Acceptance Service Reports Service Industry Specific Topics 2
A Acceptance p-value for linearity and bias analysis in minitab results Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 0
D Good Acceptance Activity/Inspection flowchart for reference - Wanted Please Process Maps, Process Mapping and Turtle Diagrams 2
S Acceptance criteria for gauges General Measurement Device and Calibration Topics 6
S Is there any connection between PFMEA and Acceptance Plan FMEA and Control Plans 1
M Characterization Testing - NO acceptance criteria, no minimum performance requirement Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
J AIAG PPAP: 2.2.11.3 Acceptance Criteria for Initial Study (page 9, 4th edition) APQP and PPAP 1
C FDA on changing acceptance criteria re: analytical method validation US Food and Drug Administration (FDA) 1
M Informational USFDA Final Rule – Human Subject Protection; Acceptance of Data From Clinical Investigations for Medical Devices Medical Device and FDA Regulations and Standards News 0
M Informational USFDA Guidance – Acceptance and Filing Reviews for Premarket Approval Applications (PMAs) Medical Device and FDA Regulations and Standards News 0
khotchandrakant Defining Acceptance Quality Level, I need clarity on AQL 1.5, 2.5, 4.0 AQL - Acceptable Quality Level 5
M Acceptance of remote auditing techniques - Can you help me with my research? General Auditing Discussions 0
K Guard-banding and relaxed acceptance for dimensional gauges General Measurement Device and Calibration Topics 1
E IPC/WHMA-A-620C: Requirements and Acceptance for Cable and Wire Harness Assemblies Other ISO and International Standards and European Regulations 2
B IATF 16949 Cl. 8.6.6 - Acceptance Criteria - Zero Defects Attribute Data Sampling IATF 16949 - Automotive Quality Systems Standard 2
P Medical Device Validation Protocols and Reports - Acceptance Criteria Other Medical Device and Orthopedic Related Topics 3
J ISO 9001:2015 Cl. 8.7.1 d) Obtaining Authorization for Acceptance under Concession ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
P Setting Acceptance Quality Limits (AQL) for New Medical Devices ISO 13485:2016 - Medical Device Quality Management Systems 4
C Software Unit Acceptance Criteria (5.5.4) IEC 62304 - Medical Device Software Life Cycle Processes 3
J Acceptance Criteria in 8.1.1 e) of IATF 16949 IATF 16949 - Automotive Quality Systems Standard 2
O Acceptance criteria bacterial endotoxins immunosera for human use Manufacturing and Related Processes 3
I As a part of validation, should i include the acceptance criteria of CPk index >1.33? Qualification and Validation (including 21 CFR Part 11) 4
V Application and differences of OC Curve and Acceptance Sampling Statistical Analysis Tools, Techniques and SPC 2
V Acceptance Criteria Gage R&R - Simple vs Complex Techniques Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 8
bio_subbu USFDA issues draft Guidance on the Acceptance of Medical Device Clinical Data Other US Medical Device Regulations 1
A 5.5.3 - Software Unit Acceptance Criteria (Risk Control Measures) IEC 62304 - Medical Device Software Life Cycle Processes 3
M A Question on EASA's "Release and Acceptance of Components" EASA and JAA Aviation Standards and Requirements 2
M Establishment of Acceptance Criteria for a Weigh Balance General Measurement Device and Calibration Topics 3
L Choosing the correct Distribution for Acceptance Sampling Inspection, Prints (Drawings), Testing, Sampling and Related Topics 19
X How to define Calibration Acceptance Criteria General Measurement Device and Calibration Topics 3
T Scoring 8Ds for Acceptance/Rejection Nonconformance and Corrective Action 6
G Need help on Device Description in the Acceptance Checklist for Traditional 510(k) Other US Medical Device Regulations 2
P Gage R (no reproducibility) - Acceptance Criteria Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
B Health Canada Acceptance Policy of Foreign Clinical Trial Data for Devices Canada Medical Device Regulations 2
M Kindly help me to understand MSA Acceptance Level Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
R Policy Statement for FDA CFR 820 Acceptance Activities of Incoming Products. Other US Medical Device Regulations 1
R Long Range OC Curves - Acceptance Sampling Switching Rules Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
kedarg6500 Acceptance Sampling Plan for Variables - Product Performance Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
V Validation Test and Acceptance Criteria for Compressed Air Qualification and Validation (including 21 CFR Part 11) 3
I Dual Acceptance Criteria: Cpk and Individual Values Statistical Analysis Tools, Techniques and SPC 3
L IEC 60601-1-2 & 60601-1-11- Acceptance Levels IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
M Customer Changes their Quality Acceptance Level Requirements Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
M FDA Acceptance of ISO 13485 Reports from a Recognized International Program Other US Medical Device Regulations 4
N Risk Acceptance - Consequence Matrix ISO 14971 - Medical Device Risk Management 12
R Results: Monte Carlo Simulation of Acceptance Sampling Plan AQL - Acceptable Quality Level 3
I AS 9003A (2012) Acceptance - Why Not? AS9100, IAQG 9100, Nadcap and related Aerospace Standards and Requirements 9
Marc Definition IFAT - Integration Factory Acceptance Testing Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 0
Similar threads


















































Top Bottom