Does pitch/increment/resolution of a ruled scale apply to measurement uncertainty as line item?

I have a guy at work that says that the resolution of a lined scale (such as a stage micrometer or steel ruler) should be included as a line item in the uncertainty calculation (because the resolution of the device under test is to be included). While this makes sense for measuring instruments with variable displays, I don't think it does for fixed increment linear scales. He is going to dig up documents that are supposed to back up his position, so I am hoping someone can point me to a specific document (and where in the doc!) that says a linear scale resolution does not apply.

Thanks!

Help- this is still an issue; any help here?

In the absence of any qualified inputs yet, let me play devil's advocate at the moment.

" While this makes sense for measuring instruments with variable displays, I don't think it does for fixed increment linear scales. "

How did you arrive at this conclusion? From my point of view, resolution is resolution, regardless of whether it's analog, digital, or a cubit.

And while we're here, I'll plug a song I like -

I guess the main difference is that we are considering the calibration of high quality scales (chrome on glass typically). It is a line standard not a ruler.
Uncertainty calc. Example of why it does not make sense;

(uncertainty line items)
Resolution (of scale) 1mm
Repeatability (of measuring device) 0.001mm
Cal ref std 0.001mm
Meas. Equip cal tol. 0.002mm

K=2 uncertainty from this comes out to be 1.2mm

I am thinking only the resolution of the measuring device should be applied. I need some standard documentation to show this is so.
Help!

" He is going to dig up documents that are supposed to back up his position, so I am hoping someone can point me to a specific document (and where in the doc!) that says a linear scale resolution does not apply. "

At that point in the game, as usual, the influences to consider are

Industry standards
Regulatory requirements
Organizational procedures

I don't know what team you play for or in what league, but unless there are related requirements in the above three then it's just your preferences vs his.

You don't find a 1.2mm uncertainty to be ludicous for a precison scale/line standard (1mm pitch) calibration?

Oh, we're talking about what I find ludicrous ...

Well, you did use the words " makes sense ".

For purposes of self preservation I tend to find the world to be ludicrous, so I'm afraid I'm not always the best technical consultant.

But sometimes the area of uncertainty exceeds the information you wish to know. This is what drives us to use different tools, or re-frame the method, or the criteria.

I'd also hoped to draw in a real professional opinion - what is that rule about the fastest way to get the correct answer on the internet is to post the incorrect answer?

I had not heard of that particular internet rule; but now that you mention it- very true!

I may have found some evidence to help my case (but nothing as clear cut as "here is the standards doc, and it states such in such at paragraph # xxx".
What I found was a round robin test of a 100mm scale sent to many national metrology centers (doc at: https://www.bipm.org/utils/common/pdf/final_reports/L/K7/EUROMET.L-K7.2006.pdf )

This doc has wonderful descriptions of the machine used, the methodology and the detailed uncertainty stack. Some of the machine are quite similar to our setup (optical microscope to camera to screen). None of them mention the pitch of the scale read!

greif said:

I have a guy at work that says that the resolution of a lined scale (such as a stage micrometer or steel ruler) should be included as a line item in the uncertainty calculation (because the resolution of the device under test is to be included). While this makes sense for measuring instruments with variable displays, I don't think it does for fixed increment linear scales. He is going to dig up documents that are supposed to back up his position, so I am hoping someone can point me to a specific document (and where in the doc!) that says a linear scale resolution does not apply.

Thanks!

Click to expand...

If you are using the scale to make a measurement then the resolution of the measurement must be included as a contributor. For instance, if you are using an 8 1/2 digit multimeter to make a measurement but only report out to 0.001 V on your results it is the resolution of the measurement, not the meter, that gets reported.

If you are only reporting your results to the cardinal points on the scale, that is the resolution. If you are interpolating between lines, that is the resolution. Whatever assumptions you make regarding rounding up or down need to be included. The pitch of the gradations should be considered as they might be "significant" in some measurements, and significant is the threshold for reporting uncertainty contributors.

I note your words in italics (" then the resolution of the measurement ") and hope that this wording might be a quote of a standard document.
Is it?
This makes my point nicely; we are measuring the scale, and that measurement has an uncertainty. Using the scale to measure has it's own uncertainty.