+0/-.001 Tolerance question - Feature size is 1.249 +0/-.001 - Actually measures 1.2493 (.0003 OHL

[avodroctrebor]

I need help. The feature size is 1.249 +0/-.001. the feature actually measures 1.2493 (.0003 OHL). the feature was written up on an NCM and sent to the engineer and the disposition was: "Diameter measures 1.2493 (.0003 oversize). Higher precision of measurement than required by drawing (UAI)." The Engineer said that this feature is in tolerance because we used a higher precision of measurement than the drawing required. Feedback Please. Is there a standard somewhere out there that I can reference to either dispute or validate the engineers disposition?

Thank you

 

[Ninja]

Howdy, and welcome to the Cove.

If you do a search for the term "tolerance rounding" without the quote marks...you'll get some (a lot of) helpful reading.
This topic has been discussed here at length in the past, and it will be faster for you than waiting for answers to this thread.

HTH

 

[Cthames]

There are different schools of thought on this. Some will round if there is an extra digit beyond the specification. So, they might read 1.2493 as 1.249 and accept the part as in spec. Others will keep all digits from the measurement, and since 1.2493 > 1.249, would determine the part is out of spec. There is no clear cut rule on how to do this.

Here is an FDA document (not sure what industry you are in) that discusses rounding measurements on page 10. https://www.fda.gov/media/73535/download

 

[Mike S.]

I'd say your engineer is wrong unless the drawing specifically says the gage precision you used is wrong (I doubt it).

You want to use M&TE that is at least 4 times more accurate than your tolerance (i.e. in this case accurate to +/- .000125") whenever possible. I hope you did so. Such a gage is gonna read to the 4th decimal place, and it will be a valid digit.

If you were measuring thickness and used micrometers capable of +/- .0001" accuracy and they read 1.2493, it is a valid reject.

Tell the Engineer you'll ask the customer for their interpretation and live with their decision.

 

[avodroctrebor]

Howdy, and welcome to the Cove.

If you do a search for the term "tolerance rounding" without the quote marks...you'll get some (a lot of) helpful reading.
This topic has been discussed here at length in the past, and it will be faster for you than waiting for answers to this thread.

HTH

Awesome! Thank you

 

[Ninja]

you'll ask the customer for their interpretation and live with their decision.

Winner, winner, chicken dinner !

 

[Enghabashy]

If you select high precision measuring equipment ' i.e ; scale value as o.ooooo1 !!" therefore you should accept the resulted reading without any exclusion ; the result is approaching also the out of tolerance area ; the status as I see that : rejected according the Specs. , anyhow you should take corrective action because the sample approaching or near the out of tolerance & there's a risk for finding more when the lot or batch 100% inspected , anyhow MSA manual & their rules could be more judge of this status

 

[Ninja]

There is "guidance" all over the place, but only the customer "requires" in this area.

FWIW, we had approval from a customer (Tier I auto) to only write down the number of decimals that the spec limits had (which brings rounding straight into the picture) and Accept/Reject based on the recorded data.
With that direct blanket customer approval, we would have rounded in the OP's situation...and questioned/addressed the employee who wrote down "1.2493" in the first place for not following governing procedures...the "3" should never have been written in the first place, and the whole situation would have been avoided.
(Without customer approval, we would have called that thing out of spec).

 

[Jim Wynne]

FWIW, we had approval from a customer (Tier I auto) to only write down the number of decimals that the spec limits had (which brings rounding straight into the picture) and Accept/Reject based on the recorded data.
With that direct blanket customer approval, we would have rounded in the OP's situation...and questioned/addressed the employee who wrote down "1.2493" in the first place for not following governing procedures...the "3" should never have been written in the first place, and the whole situation would have been avoided.
(Without customer approval, we would have called that thing out of spec).

What you're describing is truncation, not rounding. If the reading were 1.2495, according to the customer dispensation you cite, you would still report 1.249, truncating everything past the third decimal place. If you were rounding, the reported value would be 1.250.

In any event, if the customer allows for this sort of thing, the permission had better be in writing from a person known to have the authority to make such decisions. When the tolerance band is only .001 and the measurement is reported at the high (or low) limit, someone is going to ask questions sooner or later.

 

[Ninja]

With the example data given by both the OP and myself, you are absolutely right since rounding and truncation end up the same place...but if the value was instead 1.2487, it still would have been written as 1.249. And if it was 1.2495, it would have been written as 1.250 and would be deemed out of spec.
Pretty sure that's rounding, not truncation.

And totally yes...if you go there, in writing is about as mandatory as it gets.

FWIW, we would have questioned that spec and resolution match pretty thoroughly up front since it is so low res and so tight a tolerance band for the res. Having band and resolution down to a 'single tick of the gage' is just begging for issues...I would like to see another decimal on both, and have that decimal recorded.
How would you even attempt to run SPC on a single tick spec range?