Search the Elsmar Cove!
**Search ALL of Elsmar.com** with DuckDuckGo including content not in the forum - Search results with No ads.
Profound Statistical Concepts

Profound Statistical Concepts 2019-10-31

Ronen E

Problem Solver
Staff member
Super Moderator
#61
Miner can provide a better expalanation than I can, but I’ll give it a start. Confidence intervals (not to be confused with confidence LEVELS) display the amount of precision of the AVERAGE of a data set. Tolerance intervals display the likely range of some portion of the INDIVIDUAL VALUES of the population. Two very different things. I don’t use tolerance intervals.
I think I understand quite well what Tolerance Intervals are. As far as I can tell, the argument I mentioned from that book is quite well substantiated, it's not just a random comment they made. I didn't want to go into too much detail so I briefly "summarized" the claim in my own words ("similar but better"), but the authors did much more than that. In very general terms, if you want to bracket 95% of the sampled population at a certain confidence level you can construct the appropriate Tolerance Interval, but you could also bracket Q0.025 and Q0.975 (by constructing a Confidence Interval for each, at the appropriate confidence level) and thus bracket the same Coverage (95% of the sampled population). According to the authors, the latter approach will yield a smaller interval than the former, for the same sample size.
I do not use tolerance interval either. In industries that have stringent requirements, they have very limited application.
Why?
 

Watchcat

Quite Involved in Discussions
#62
much of the misuse arises from the lack of planning and poor experimental design
I thought about making one last comment/post before I moved on, but couldn't think of a way to put it that might not be taken as being dismissive of the importance of statistics. But I can bounce off of this. When people start talking about the nitty gritty of statistical tools, my reaction is that this is the least of our worries. So many studies are so bad, using different statistical tools wouldn't have made a dent. I came to the same conclusion about the nitty gritty of good writing when I was an editor. If anything, editing an article to make the study rationale, design, results more clear can sometimes serve only to make it painfully more clear that the whole thing is a mess.
 
Last edited:

Bev D

Heretical Statistician
Staff member
Super Moderator
#63
Why not use tolerance intervals? Because it’s - relatively - a lot of work for very little gain in ‘accuracy’ over a simple +/- 3 SD or +/- 2.5 SD. It is still dependent on the Normality and homogeneity of the distribution. I’ve never needed tolerance intervals in 40 years. They have a limited use mostly in only providing an estimate of the likely spread of the individual values of a population based on a relatively small size. That is less than 1% of the questions and analyses I need to do...

The bigger issue for me is that there is too much reliance on descriptive statistics and enumerative study approaches when what we really need is analytic studies and real understanding of our processes that are rarely Normal or homogenous...
 

Ronen E

Problem Solver
Staff member
Super Moderator
#64
Why not use tolerance intervals? Because it’s - relatively - a lot of work for very little gain in ‘accuracy’ over a simple +/- 3 SD or +/- 2.5 SD. It is still dependent on the Normality and homogeneity of the distribution. I’ve never needed tolerance intervals in 40 years. They have a limited use mostly in only providing an estimate of the likely spread of the individual values of a population based on a relatively small size. That is less than 1% of the questions and analyses I need to do...

The bigger issue for me is that there is too much reliance on descriptive statistics and enumerative study approaches when what we really need is analytic studies and real understanding of our processes that are rarely Normal or homogenous...
1. Based on that book (or Dr. Wheeler) I don't see how constructing a Tolerance Interval is more work than most other statistical methods I see around, once the initial mathematical "hard work" has been done, which is currently the case.

2. There are non-parametric methods to construct Tolerance Intervals, i.e. assuming Normality is not a must.

3. The way I understand it, if the process isn't homogeneous, most of the statistical modelling collapses. Isn't it so? What I'm being taught is when we realise that the process we try to study is non-homogeneous we should stop and try to understand what's going on, that is, why it looks like a non-homogeneous process. Perhaps it's not a single process, perhaps it's not in control, etc. Then either we try to fix it (if we can and if it makes practical sense) or we change our study perspective. Do you treat non-homogeneous processes differently in your practice?

4. I also "didn't need" Tolerance Intervals throughout my career, until recently. I simply wasn't taught that practice, so I used other tools. I'm not saying that they're absolutely necessary - they probably aren't. But currently I feel that this may be an elegant solution to some problems that I've seen around for many years, with solutions I've never been happy with. Isn't that how we sometimes progress?... We do something in a way that is "okay", then at some point we try a different way and realise it's better. We don't feel a burning need for the new method, but once we become aware of it and realise it makes more sense than the old ways, we hardly ever go back. Perhaps you and me have different perspectives - most of the situations I need to handle involve rigidly-small (sometimes very small) sample sizes and a strong preference to completing the study in one round. Maybe if I had the luxury of increasing the sample size at will or running multiple iterations I wouldn't be that excited about Tolerance Intervals, too. But I usually have to work with what I'm given, and since I wear the "engineer" hat rather than the "statistician" I'm usually expected to come up with a practical solution rather than just present unpleasant mathematical/scientific truths.

5. I accept that this may not be useful in your line of work, but in my field/practice "providing an estimate of the likely spread of the individual values of a population based on a relatively small size" is extremely useful. This is exactly what we're looking for in developing new medical devices (or changing them).
 
Top Bottom