# Profound Statistical Concepts 2019-10-31

#### Ronen E

##### Problem Solver
Staff member
Moderator
Miner can provide a better expalanation than I can, but I’ll give it a start. Confidence intervals (not to be confused with confidence LEVELS) display the amount of precision of the AVERAGE of a data set. Tolerance intervals display the likely range of some portion of the INDIVIDUAL VALUES of the population. Two very different things. I don’t use tolerance intervals.
I think I understand quite well what Tolerance Intervals are. As far as I can tell, the argument I mentioned from that book is quite well substantiated, it's not just a random comment they made. I didn't want to go into too much detail so I briefly "summarized" the claim in my own words ("similar but better"), but the authors did much more than that. In very general terms, if you want to bracket 95% of the sampled population at a certain confidence level you can construct the appropriate Tolerance Interval, but you could also bracket Q0.025 and Q0.975 (by constructing a Confidence Interval for each, at the appropriate confidence level) and thus bracket the same Coverage (95% of the sampled population). According to the authors, the latter approach will yield a smaller interval than the former, for the same sample size.
I do not use tolerance interval either. In industries that have stringent requirements, they have very limited application.
Why?

#### Watchcat

Trusted Information Resource
much of the misuse arises from the lack of planning and poor experimental design
I thought about making one last comment/post before I moved on, but couldn't think of a way to put it that might not be taken as being dismissive of the importance of statistics. But I can bounce off of this. When people start talking about the nitty gritty of statistical tools, my reaction is that this is the least of our worries. So many studies are so bad, using different statistical tools wouldn't have made a dent. I came to the same conclusion about the nitty gritty of good writing when I was an editor. If anything, editing an article to make the study rationale, design, results more clear can sometimes serve only to make it painfully more clear that the whole thing is a mess.

Last edited:

#### Bev D

##### Heretical Statistician
Staff member
Super Moderator
Why not use tolerance intervals? Because it’s - relatively - a lot of work for very little gain in ‘accuracy’ over a simple +/- 3 SD or +/- 2.5 SD. It is still dependent on the Normality and homogeneity of the distribution. I’ve never needed tolerance intervals in 40 years. They have a limited use mostly in only providing an estimate of the likely spread of the individual values of a population based on a relatively small size. That is less than 1% of the questions and analyses I need to do...

The bigger issue for me is that there is too much reliance on descriptive statistics and enumerative study approaches when what we really need is analytic studies and real understanding of our processes that are rarely Normal or homogenous...

#### Ronen E

##### Problem Solver
Staff member
Moderator
Why not use tolerance intervals? Because it’s - relatively - a lot of work for very little gain in ‘accuracy’ over a simple +/- 3 SD or +/- 2.5 SD. It is still dependent on the Normality and homogeneity of the distribution. I’ve never needed tolerance intervals in 40 years. They have a limited use mostly in only providing an estimate of the likely spread of the individual values of a population based on a relatively small size. That is less than 1% of the questions and analyses I need to do...

The bigger issue for me is that there is too much reliance on descriptive statistics and enumerative study approaches when what we really need is analytic studies and real understanding of our processes that are rarely Normal or homogenous...
1. Based on that book (or Dr. Wheeler) I don't see how constructing a Tolerance Interval is more work than most other statistical methods I see around, once the initial mathematical "hard work" has been done, which is currently the case.

2. There are non-parametric methods to construct Tolerance Intervals, i.e. assuming Normality is not a must.

3. The way I understand it, if the process isn't homogeneous, most of the statistical modelling collapses. Isn't it so? What I'm being taught is when we realise that the process we try to study is non-homogeneous we should stop and try to understand what's going on, that is, why it looks like a non-homogeneous process. Perhaps it's not a single process, perhaps it's not in control, etc. Then either we try to fix it (if we can and if it makes practical sense) or we change our study perspective. Do you treat non-homogeneous processes differently in your practice?

4. I also "didn't need" Tolerance Intervals throughout my career, until recently. I simply wasn't taught that practice, so I used other tools. I'm not saying that they're absolutely necessary - they probably aren't. But currently I feel that this may be an elegant solution to some problems that I've seen around for many years, with solutions I've never been happy with. Isn't that how we sometimes progress?... We do something in a way that is "okay", then at some point we try a different way and realise it's better. We don't feel a burning need for the new method, but once we become aware of it and realise it makes more sense than the old ways, we hardly ever go back. Perhaps you and me have different perspectives - most of the situations I need to handle involve rigidly-small (sometimes very small) sample sizes and a strong preference to completing the study in one round. Maybe if I had the luxury of increasing the sample size at will or running multiple iterations I wouldn't be that excited about Tolerance Intervals, too. But I usually have to work with what I'm given, and since I wear the "engineer" hat rather than the "statistician" I'm usually expected to come up with a practical solution rather than just present unpleasant mathematical/scientific truths.

5. I accept that this may not be useful in your line of work, but in my field/practice "providing an estimate of the likely spread of the individual values of a population based on a relatively small size" is extremely useful. This is exactly what we're looking for in developing new medical devices (or changing them).

Thread starter Similar threads Forum Replies Date
Deming's SoPK (System of Profound Knowledge) Discussion Philosophy, Gurus, Innovation and Evolution 220
Deming's SoPK (System of Profound Knowledge) Challenge Philosophy, Gurus, Innovation and Evolution 66
Design Verification & Process Validation - Statistical sample sizes Design and Development of Products and Processes 2
Interactive visualization through graphical simulation of statistical concepts Statistical Analysis Tools, Techniques and SPC 3
Statistical Analysis - Check if these organisms at different concentrations affect the growth of wheat seedlings Using Minitab Software 4
Statistical Techniques Procedure - What should be included Document Control Systems, Procedures, Forms and Templates 4
Statistical justification of sampling size in V&V tests ISO 13485:2016 - Medical Device Quality Management Systems 5
It’s time to talk about ditching statistical significance Statistical Analysis Tools, Techniques and SPC 6
Steve Prevette's Statistical Process Control (SPC) "Library" Statistical Analysis Tools, Techniques and SPC 0
A Balanced view of statistical tests Statistical Analysis Tools, Techniques and SPC 3
Statistical basis and justification while comparing / changing sampling plans Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
SPC (Statistical Process Control) for Unilateral Tolerance - Questions Statistical Analysis Tools, Techniques and SPC 6
IATF 16949 9.1.1.3 Application of statistical concepts - Our technicians are quizzed for statistical knowledge IATF 16949 - Automotive Quality Systems Standard 3
Please help identify appropriate statistical treatment Statistical Analysis Tools, Techniques and SPC 13
Statistical basis for 30 pieces for FAI 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
IATF 16949 clause 7.1.5.1.1 - Statistical studies shall be conducted IATF 16949 - Automotive Quality Systems Standard 3
Statistical Process Control and Inspection in Footwear Production Statistical Analysis Tools, Techniques and SPC 0
IATF 16949 Cl. 7.1.5.1.1 - Statistical studies shall be conducted IATF 16949 - Automotive Quality Systems Standard 3
Statistical Process Control Library Statistical Analysis Tools, Techniques and SPC 17
Happy Birthday Statistical Steven - 2015 Covegratulations 10
L When are Statistical techniques not applicable? Service Industry Specific Topics 16
FDA 21 CFR 820.250 - Does "valid statistical" always mean math? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
Common Statistical Errors Using Minitab Software 1
Y Statistical Analysis of Road Traffic Data Statistical Analysis Tools, Techniques and SPC 11
Class II Medical Device Manufacturer - SOP for 820.250 Statistical 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
E Correct Statistical Test comparing 2 Groups Statistical Analysis Tools, Techniques and SPC 14
A Statistical Software Calibration using Ford's "Sample Calibration File" Statistical Analysis Tools, Techniques and SPC 8
Defining Martial Arts and Gymnastics Statistical Techniques Statistical Analysis Tools, Techniques and SPC 4
J Capability Analysis - Unusual Statistical Distribution of my Proccess Capability, Accuracy and Stability - Processes, Machines, etc. 5
M PhD Thesis Data Statistical Analysis Methods Statistical Analysis Tools, Techniques and SPC 2
J Statistical Significance and SPC Control Chart Reports Statistical Analysis Tools, Techniques and SPC 9
N Statistical Quality Improvement Action for Small Batch Production Statistical Analysis Tools, Techniques and SPC 17
Validation of macro - scripts - programs used in statistical software (Minitab-SAS... Qualification and Validation (including 21 CFR Part 11) 5
Statistical Process Control Crash Course - Question Quality Manager and Management Related Issues 10
H Statistical Models for Predictive Management of Software Processes Software Quality Assurance 2
A Statistical Correlation between ordered SKUs Statistical Analysis Tools, Techniques and SPC 8
S Minitab and Crystal Ball Statistical Analysis Software Using Minitab Software 13
M Determining if two different X's have any Statistical Significance on the Y's Statistical Analysis Tools, Techniques and SPC 4
Statistical Stability for the PQ of Analytical Equipment Qualification and Validation (including 21 CFR Part 11) 1
F Statistical Comparison of Product: High Average vs. Low Range Capability, Accuracy and Stability - Processes, Machines, etc. 13
E Using ANOVA during the PQ Validation Run to evaluate Statistical Differences Statistical Analysis Tools, Techniques and SPC 4
W Gage R&R for gage pins used to inspect a hole ID called a Statistical Tolerance Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
AS 9100C sec 8.2.4 - " Recognized Statistical Principles" meaning AS9100, IAQG 9100, Nadcap and related Aerospace Standards and Requirements 7
R What is PSW - Statistical Process Package + Level 5 APQP and PPAP 7
O Is SPC (Statistical Process Control) always required? Statistical Analysis Tools, Techniques and SPC 4
Any experience/feedback on statistical software tools...minitab -design expert -JMP Using Minitab Software 4
M SPC (Statistical Process Control) Book Recommendations wanted Book, Video, Blog and Web Site Reviews and Recommendations 1
How to use Statistical Process Control (SPC)? Statistical Analysis Tools, Techniques and SPC 8
TS 16949 8.1.1 Identification of Statistical Tools - Requirement Scope? IATF 16949 - Automotive Quality Systems Standard 13
D Appropriate Statistical Methods to set up an Alert/Warning Limit Quality Assurance and Compliance Software Tools and Solutions 15