Need help wrapping my head around confidence vs beta error


Starting to get Involved
So I've tried reading into this topic many times in the past and my understanding of these terms is as follows:

Reliability- percent of products we say meets spec
Confidence- percent of time we will be correct on a given reliability
(i.e. 95% confidence/95% reliability means when accepting a lot, we want to be correct 95% of the time that 95+% of parts are acceptable).

So with that in mind, is Beta error simply 1-[confidence]? So if we're right 95% of the time, we will be wrong 5% of the time?

I couldn't really find many resources that flat-out compared beta to confidence with sampling plans, but it seems logical?

I ask because we have a procedure that discusses sample size determination, and it says that "a min 95% confidence with a max 5% beta is recommended".. If my prior statement is correct, then this seems redundant? (i.e. if you have 95+% confidence, you will automatically have no more than 5% beta)

If I am wrong on anything here, please correct me!!!!
Elsmar Forum Sponsor

John Predmore

Quite Involved in Discussions
There are 3 separate concepts in your example, but you used 95% for all three concepts. If I explain using different percentages, there may be less confusion about the different percentages.

The terms confidence and beta error come from the universe of hypothesis testing. In a manufacturing quality context, we start with the hypothesis the overall quality of supplier’s entire shipment is acceptable. There is the true (unknown) state of the shipment, and there is a decision made based on an experiment or a sample. There are 4 possible outcomes:
In reality, the product is good, and decision is made to accept <- this is desired outcome
In reality, the product is bad, and decision is made to reject <- this is desired outcome
In reality, the product is good, and decision is made to reject <- this is Type I error
In reality, the product is bad, and decision is made to accept <- this is Type II error

You used the term reliability to indicate that some fraction of the shipment can be non-conforming and the batch would still be acceptable. For my example, out of a batch 1000 widgets I want assurance 90% are within spec, so reliability must be 90%. Based on testing the sample, I accept or reject the entire lot. Because we are counting rejects rather than evaluating the mean of a continuous variable, I use a discrete probability function rather than a Z-test or a t-test. (I used the binomial instead of the hypergeometric since the batch >> sample size.) If there are 10% bad parts in the batch of 1000, there is 35% chance of seeing none in a sample of 10 (which I calculated using the BINOM function in Excel) and therefore 65% cumulative probability of seeing one or more out of 10. To be 95% confident to detect a lot of 1000 which is 10% bad, I need to see zero in a sample of 29 (to be better than 95% confident in this 1-sided test).

In a single shipment, there is 5% risk I reject the entire batch of parts which are actually 90% quality, due to luck of the draw in the sample. In this scenario, the producer’s risk of loss is 5%, which is also called the alpha risk, the risk of wrongly rejecting the hypothesis (that the quality of parts is better than 90%). The significance level of the hypothesis test (as illustrated in this scenario) is 95%.

For all possible quality levels, there is the possibility that a shipment with unacceptable quality might be accepted. Let’s say I want no more than 20% chance of a Type II error where the sample of widgets from an overall defective lot contains zero nonconforming parts (I am willing to accept a little more risk because I have automatic in-process gaging). This consumer’s risk is called Beta, where the hypothesis of 90% quality parts is wrongly accepted. The Beta risk weighs the range of possible scenarios with the defined statistical result. The inverse of the beta risk is called the power of a statistical test, the probability of correctly rejecting a hypothesis (when the alternate is true in reality). The power in this scenario would typically be determined using statistical tables or a computer.

See also Other 1-Sample Binomial | Power and Sample Size Calculators | HyLown

Bev D

Heretical Statistician
Staff member
Super Moderator
We can try a slightly different approach to understanding.
There are two important parameters of any inspection sampling plan: the defect rate that is acceptable and the defect rate that is NOT acceptable (Rejectable). The first is called the Acceptable Quality Level (AQL) and the second is the Rejectable Quality level (RQL or LTPD - Lot Tolerance Percent Defective)

AQLs have alpha risk - the probability of REJECTING a lot at the ACCEPTABLE defect rate. Here Confidence is 1-alpha (The probability of ACCEPTING the lot at the AQL defect rate)

RQLs have beta risk - the probability of ACCEPTING a lot a the RQL defect rate. Here Confidence is 1-beta (The probability of REJECTING the lot at the RQL defect rate)

The type of plan you reference is an AQL only plan (There is an RQL for it of course but it is not specified or often not calculated for the Confidence/Reliability type of plan)
Thread starter Similar threads Forum Replies Date
SocalSurfer AS9100 new certificate, but need QMS software, help Quality Assurance and Compliance Software Tools and Solutions 2
S Need help with analysing a survey on minitab Using Minitab Software 1
K A proposal for the model Quality Management - I need help for the project ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
R Probability - Need a help to solve the below question Statistical Analysis Tools, Techniques and SPC 5
8 Need Help - Runout - Function Gage Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
S New to FAIR, need help in filling it out AS9100, IAQG 9100, Nadcap and related Aerospace Standards and Requirements 8
R Pls help --- Need expert advice on Video Measurement Measurement Uncertainty (MU) 0
Ajit Basrur Need help to understand HIPAA requirements ISO 13485:2016 - Medical Device Quality Management Systems 17
Z I need help with getting a CFS in China Manufacturing and Related Processes 2
P Received a minor for not having good measureables/goals. Need help with KPIs. IATF 16949 - Automotive Quality Systems Standard 52
A Failed the AS9120B Exam Twice.....Need Help AS9100, IAQG 9100, Nadcap and related Aerospace Standards and Requirements 7
Ajit Basrur Need help in Conditional Formatting for the entire row Excel .xls Spreadsheet Templates and Tools 3
H Need help on Quality Improvement Plan (QIP) Quality Tools, Improvement and Analysis 5
S Help: Need Control Self Assessment Questionnaire for HR Departmental Functions Process Audits and Layered Process Audits 0
M Need help in Calibration of Directional Coupler General Measurement Device and Calibration Topics 2
L Need HELP with Internal Audit Program ISO 13485.2003 Quality Management System (QMS) Manuals 3
J Supporting Processes - Internal Audits - Need help settling a debate IATF 16949 - Automotive Quality Systems Standard 4
A Cpk Formula seems off, need help!? Capability, Accuracy and Stability - Processes, Machines, etc. 11
A Need Help With AEA Auditor Related Situational Questions AS9100, IAQG 9100, Nadcap and related Aerospace Standards and Requirements 1
E Transformer Overload Test - Need help IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
C I Need Help with Drafting Conventions!!! Design and Development of Products and Processes 4
M Need help on QMS Development in an AS9100C certified organization AS9100, IAQG 9100, Nadcap and related Aerospace Standards and Requirements 10
R Need Help on Analysis: How to know potential causality model from historical data Problem Solving, Root Cause Fault and Failure Analysis 5
B Need help tracking training for my business Quality Assurance and Compliance Software Tools and Solutions 3
B Need help with a Control Plan for Encapsulation Mold / Trimming / Forming Tooling FMEA and Control Plans 9
S I need help from all of you bros Oil and Gas Industry Standards and Regulations 1
LRE67 Need help identifying this "thing" Coffee Break and Water Cooler Discussions 9
G Need help on Device Description in the Acceptance Checklist for Traditional 510(k) Other US Medical Device Regulations 2
K Need help finding a book on Documentation Book, Video, Blog and Web Site Reviews and Recommendations 8
J Need help/advice on video editing software etc..... Coffee Break and Water Cooler Discussions 8
L Need Help with Case Study Questions Manufacturing and Related Processes 16
W Help explaining the need for the Process Approach AS9100, IAQG 9100, Nadcap and related Aerospace Standards and Requirements 4
Ron Rompen Unusual problem with Excel - Need some help Excel .xls Spreadsheet Templates and Tools 7
S Need help on new TS 16949 Rules IATF 16949 - Automotive Quality Systems Standard 5
M Need help with Network Analyzers and which DUT can be Calibrated with it Calibration and Metrology Software and Hardware 2
R Need help on defining scope for Design Verification File for Class III IVD 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 8
J Help I need your opinion on a News Letter Misc. Quality Assurance and Business Systems Related Topics 15
N Need Help Preparing for Medical Device Quality Engineer Interview Career and Occupation Discussions 2
B Need help on an application we are working on Coffee Break and Water Cooler Discussions 4
S Need help with Project Charter for my Six Sigma Green Belt Project Six Sigma 1
Q Need help on MSA study on a torque analyzer Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
B Need help in adding second location to FDA Registration Other US Medical Device Regulations 4
T A survey on problems during ISMS implementation - need help IEC 27001 - Information Security Management Systems (ISMS) 2
Z My first Gage R&R - Paper/Poly/Foil Material Bond Test, need help Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 15
R Need help on cost analysis for copper tube manufacturing Manufacturing and Related Processes 1
T SPSS for Analysis of Medical Data - Need help Statistical Analysis Tools, Techniques and SPC 12
T Metal Stamping Defect--Need Help with Corrective Action Manufacturing and Related Processes 5
E New to Quality & Need Help Career and Occupation Discussions 6
T Need help developing an ECO (Engineering Change order) procedure Document Control Systems, Procedures, Forms and Templates 3
S Need Help with Response Surface Optimization Using Minitab Software 4
Similar threads

Top Bottom