Tolerance, decimal places, and measuring equipment capability

[heat_fan1]

I know the issue of measurement reporting and decimal places has been discussed many times, but I haven't seen it discussed in the spirit of measurement equipment capability. So, here are some facts:

• I receive and inspect material from my suppliers.
• I have many hundreds of parts with drawings that specify features in mm to, typically, 2-3 decimal places. They refer to Y 14.5-1994, and I don't reference any other standards.
• I use 2 CMMs for inspection, and their accuracy is to 1-3 microns. However, I have not validatd all programs, yet. By default, they report 5 decimal places and shortening that is done by rounding.

Because the systems report beyond their capability, I have been rejecting parts for tenths or hundredths of microns out, which are basically random numbers.

Obviously this is silly, but I'm trying to figure out the best way to address this. My thought is to implement rounding of inspections to 2 decimal places or 1 more than the number in the specification, whichever is greater. Any thoughts? Are there any standards I haven't seen that may help?

 

[Stijloor]

A Quick Bump!

Can someone help?

Thank you very much!!

 

[Miner]

Donald Wheeler does an excellent job of addressing this issue.

Quality Digest has a brief over view in: How Measurement Error Affects the Four Ways We Use Data; Understanding Probable Error and the Intraclass Correlation Coefficient makes it possible to quantify measurement uncertainty


The specific guidance is: "Therefore, the first thing we learn is that knowledge of the Probable Error will help you to record the proper number of digits. In general, you will want to have a measurement increment that is approximately the same size as the Probable Error. Useful guidelines are:
? Your measurement increment should not be larger than 2 Probable Errors.
? Your measurement increment should not be smaller than 0.2 Probable Error."