Here is what ASME Y14.5M-1994 says...

1.6.1(a) Where the dimensions is less hthan one millimeter, a zero precedres the decimal point.

1.6.1(b) Where the dimension is a whole number, neither the decimal point nor a zero is shown.

1.6.2(a) A zero is not used before the decimal point for values of less than one inch.

1.6.2(b) A dimension is expressed to the sam number of decimal places as its tolerance.

Now heres my question.

If my engineers put say 1.240-1.260" on a drawing, and my inspectors come in and measure that dimension but measure to 1.26005, it's technically a nonconforming part. According to the standard they should not even be measuring to that accuracy, am I correct?

This question is raised because of a class I attended with Pratt and Whitney, and they require absolute measurments. Basically what they have said is if our drawing is labeled as .000" then we want to see an inspection report with .XXX" and not .XXXX".

Am I getting this right or am I off base?

Here is what ASME Y14.5M-1994 says...

1.6.1(a) Where the dimensions is less hthan one millimeter, a zero precedres the decimal point.

1.6.1(b) Where the dimension is a whole number, neither the decimal point nor a zero is shown.

1.6.2(a) A zero is not used before the decimal point for values of less than one inch.

1.6.2(b) A dimension is expressed to the sam number of decimal places as its tolerance.

Now heres my question.

If my engineers put say 1.240-1.260" on a drawing, and my inspectors come in and measure that dimension but measure to 1.26005, it's technically a nonconforming part. According to the standard they should not even be measuring to that accuracy, am I correct?

This question is raised because of a class I attended with Pratt and Whitney, and they require absolute measurments. Basically what they have said is if our drawing is labeled as .000" then we want to see an inspection report with .XXX" and not .XXXX".

Am I getting this right or am I off base?It's nonconforming. And it's not accuracy, but resolution. If you were to round-off the measurement, it would be 1.2601.

It's nonconforming. And it's not accuracy, but resolution. If you were to round-off the measurement, it would be 1.2601.

But according to Pratt it's absolute, it is what it is, no rounding up where does it say to round up or round down in the standard? I am told that if they have .000" on thier drawing then they want .XXX" on the inspection report and not .XXXX". It's incorrect format regardless of the dimension being right or wrong.

See Paragraph 2.4 Interpretation of Limits.

It states "All limits are absolute. Dimensional limits, regardless of the number of decimal places, are used as if they were continued with zeros."

After some examples, it continues with " To determine conformance within limits, the measued value is compared directly with the specified value and any deviation outside the specified limiting values signifies nonconformance with the limits"

If your parts are right on the edge of going out of tolerance a more accurate instrument may be needed to make the final determination of conformance.

But according to Pratt it's absolute, it is what it is, no rounding up where does it say to round up or round down in the standard? I am told that if they have .000" on thier drawing then they want .XXX" on the inspection report and not .XXXX". It's incorrect format regardless of the dimension being right or wrong.

Two points:

Unless ASME Y14.5M is called out in the contract or on the drawing, it's irrelevant. A standard is only a standard when two or more parties explicitly agree to it.
1.2601 is greater than the upper limit of 1.260, which means that it is out of tolerance. You need to discuss with your customer how these things should be handled. The explicit specification says that 1.260 may not be exceeded. If your customer allows you to round to the third decimal place, they need to tell you that. If they allow for truncation beyond the third decimal place, they need to tell you that as well.

But according to Pratt it's absolute, it is what it is, no rounding up where does it say to round up or round down in the standard? I am told that if they have .000" on thier drawing then they want .XXX" on the inspection report and not .XXXX". It's incorrect format regardless of the dimension being right or wrong.I stated "if" to for emphasis. I know it's an absolute. All digits after the 3rd decimal must be "0". If they want 3 decimal places on the report, they are closing their eyes.

Lets keep sight on what is required.
The print is designed to let production know what to build and QA folks what to verify. IF there are other contractual items then they should be clearly called out.
Since we are not dealing with model based design (where there is no limit to the resolution and is truly a absolute) the key here is the resolution that is defnied on the print established by design( I will leave the commentary on the ability of some design engineers to correctly perform this function to another thread) They call out X.XXXX you measure/report to X.XXXX

For why you would measure to the next level of resolution a simple example is to take your everyday digital mic and try and get the last digit to show anything other than a 0 or 5. For us old school inspectors the same applys to "splitting the lines" and picking which one it is closer to. This is supporting that you measure 1 level past the signifigant digit and round to ensure that you avoid the 1:1 error.

:2cents: