Treating Resolution - Rectangular or Triangular Distribution

[blueicecube2]

Can I re affirm my understanding before asking funnily out loud :

resolution is treated as triangular distribution in measurement uncertainty calculation only if the unit under test is an analog unit because you can actually see the in between of the the two smallest division of the scale

whereas

resolution is treated as rectangular in digital display because you can't split the final digit display into any smaller divisions.

Spare me, thank you.

[ScottBP]

I've always been told that if it is a digital instrument, the resolution is always treated as a step distribution, i.e. divide the resolution by 2 times the square root of 3. Example, if the resolution is 3 digits past the decimal, the calculation would be 0.001/(2*sqrt(3)) ≈ 0.0002887.

I've also been told that if it is an analog scale, and you can "eyeball" the reading in between increments, it's still treated as a step distribution, but you can use half a digit. Example, if the smallest increment of a test gauge is 0.1 PSI and you can clearly see that the needle is halfway between increments, then it would be 0.05 psi, and 0.05/(2*sqrt(3)) ≈ 0.01443. Some people may go even further and break it down into quarter increments, but I think that is overkill- go get a better gauge if the accuracy of the reading matters that much.

Hope this helps...

[blueicecube2]

thank you so much. can you please advise where do you think this guideline came from, was it from a training or a reference document of some sort?

appreciate it.

[ScottBP]

I printed this and taped it to the wall over my desk:
http://www.hn-metrology.com/distrib.htm

A lot of good papers on the site as well.

[blueicecube2]

awesome. thank you so much :-)

[stefanhg]

Uncertainty due to the finite resolution measurments

1. Digital display, instruments with an analogue-to digital converter (ADC)
- Rectangular distribution
- Distribution range a = ±0.5 digit
- Standard uncertainty u = a/(SQRT(3))
- Example: Resolution = 0.1, a = 0.05, u = 0.0289

2. Digital display, instruments with direct-gating counter
- Rectangular distribution
- Distribution range a = ±1 digit
- Standard uncertainty u = a/(SQRT(3))
- Example: Resolution = 0.1, a = 0.1, u = 0.0577

3. Analog display
- Normal distribution
- Distribution range a = ± 0.5 scale interval
- Standard deviation u = 0.25 scale division
(Distribution range 4u with 95% confidence level is ± 0.5 scale interval)
- Example: Resolution = 0.1, a = 0.05, u = 0.025

4. Analog display
- Rectangular distribution
- Distribution range a = ±0.5 scale interval
- Standard uncertainty u = a/(SQRT(3))
- Example: Resolution = 0.1, a = 0.05, u = 0.0289

Sources:
- UKAS M3003 The Expression of Uncertainty and Confidence in Measurement
- EA-4/02 Expression of the Uncertainty of Measurement in Calibration

[Pezikon]

A2LA document P103d, pages 8 and 9 go into detail about using UUT resolution as an uncertainty contributor. It covers digital resolution in the same manner as everyone here has been pointing out. However, concerning the resolution of an analog scale, P103d states:

"Many labs have a rule of dividing an analog scale into no more than four segments (i.e., estimation to no better than one-fourth of a scale division) although using magnification it may be possible to estimate even smaller divisions between adjacent scale markings. In such cases, the standard uncertainty due to limited resolution is that fraction of a scale division which can be distinguished."

P103d also gives the following example:

"Using a magnifying glass, a laboratory can accurately interpolate between the scale marking on a liquid-in-glass thermometer to within one-tenth of a scale division. In this case, the standard uncertainty due to limited resolution is one-tenth of a scale division."

Digital resolution standard uncertainty: (UUT resolution)/sqrt(12)
Analog resolution standard uncertainty: (smallest discernible fraction of a division).

No math. No divisor. Just the smallest discernible fraction of a scale division.