+0/-.001 Tolerance question - Feature size is 1.249 +0/-.001 - Actually measures 1.2493 (.0003 OHL

#1
I need help. The feature size is 1.249 +0/-.001. the feature actually measures 1.2493 (.0003 OHL). the feature was written up on an NCM and sent to the engineer and the disposition was: "Diameter measures 1.2493 (.0003 oversize). Higher precision of measurement than required by drawing (UAI)." The Engineer said that this feature is in tolerance because we used a higher precision of measurement than the drawing required. Feedback Please. Is there a standard somewhere out there that I can reference to either dispute or validate the engineers disposition?

Thank you
 
Elsmar Forum Sponsor

Ninja

Looking for Reality
Trusted Information Resource
#2
Howdy, and welcome to the Cove.

If you do a search for the term "tolerance rounding" without the quote marks...you'll get some (a lot of) helpful reading.
This topic has been discussed here at length in the past, and it will be faster for you than waiting for answers to this thread.

HTH
 

Cthames

Involved In Discussions
#3
There are different schools of thought on this. Some will round if there is an extra digit beyond the specification. So, they might read 1.2493 as 1.249 and accept the part as in spec. Others will keep all digits from the measurement, and since 1.2493 > 1.249, would determine the part is out of spec. There is no clear cut rule on how to do this.

Here is an FDA document (not sure what industry you are in) that discusses rounding measurements on page 10. https://www.fda.gov/media/73535/download
 

Mike S.

Happy to be Alive
Trusted Information Resource
#4
I'd say your engineer is wrong unless the drawing specifically says the gage precision you used is wrong (I doubt it).

You want to use M&TE that is at least 4 times more accurate than your tolerance (i.e. in this case accurate to +/- .000125") whenever possible. I hope you did so. Such a gage is gonna read to the 4th decimal place, and it will be a valid digit.

If you were measuring thickness and used micrometers capable of +/- .0001" accuracy and they read 1.2493, it is a valid reject.

Tell the Engineer you'll ask the customer for their interpretation and live with their decision. ;)
 
#5
Howdy, and welcome to the Cove.

If you do a search for the term "tolerance rounding" without the quote marks...you'll get some (a lot of) helpful reading.
This topic has been discussed here at length in the past, and it will be faster for you than waiting for answers to this thread.

HTH
Awesome! Thank you
 

Enghabashy

Starting to get Involved
#7
If you select high precision measuring equipment ' i.e ; scale value as o.ooooo1 !!" therefore you should accept the resulted reading without any exclusion ; the result is approaching also the out of tolerance area ; the status as I see that : rejected according the Specs. , anyhow you should take corrective action because the sample approaching or near the out of tolerance & there's a risk for finding more when the lot or batch 100% inspected , anyhow MSA manual & their rules could be more judge of this status
 

Ninja

Looking for Reality
Trusted Information Resource
#8
There is "guidance" all over the place, but only the customer "requires" in this area.

FWIW, we had approval from a customer (Tier I auto) to only write down the number of decimals that the spec limits had (which brings rounding straight into the picture) and Accept/Reject based on the recorded data.
With that direct blanket customer approval, we would have rounded in the OP's situation...and questioned/addressed the employee who wrote down "1.2493" in the first place for not following governing procedures...the "3" should never have been written in the first place, and the whole situation would have been avoided.
(Without customer approval, we would have called that thing out of spec).
 

Jim Wynne

Staff member
Admin
#9
FWIW, we had approval from a customer (Tier I auto) to only write down the number of decimals that the spec limits had (which brings rounding straight into the picture) and Accept/Reject based on the recorded data.
With that direct blanket customer approval, we would have rounded in the OP's situation...and questioned/addressed the employee who wrote down "1.2493" in the first place for not following governing procedures...the "3" should never have been written in the first place, and the whole situation would have been avoided.
(Without customer approval, we would have called that thing out of spec).
What you're describing is truncation, not rounding. If the reading were 1.2495, according to the customer dispensation you cite, you would still report 1.249, truncating everything past the third decimal place. If you were rounding, the reported value would be 1.250.

In any event, if the customer allows for this sort of thing, the permission had better be in writing from a person known to have the authority to make such decisions. When the tolerance band is only .001 and the measurement is reported at the high (or low) limit, someone is going to ask questions sooner or later.
 

Ninja

Looking for Reality
Trusted Information Resource
#10
With the example data given by both the OP and myself, you are absolutely right since rounding and truncation end up the same place...but if the value was instead 1.2487, it still would have been written as 1.249. And if it was 1.2495, it would have been written as 1.250 and would be deemed out of spec.
Pretty sure that's rounding, not truncation.

And totally yes...if you go there, in writing is about as mandatory as it gets.

FWIW, we would have questioned that spec and resolution match pretty thoroughly up front since it is so low res and so tight a tolerance band for the res. Having band and resolution down to a 'single tick of the gage' is just begging for issues...I would like to see another decimal on both, and have that decimal recorded.
How would you even attempt to run SPC on a single tick spec range?
 
Thread starter Similar threads Forum Replies Date
M Resolution question - Measuring a product on a CMM - the tolerance is .000/.001 Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
S Non-Normal Data - Measurement for "straightness" with a 0.001" max tolerance Capability, Accuracy and Stability - Processes, Machines, etc. 10
M Informational ANVISA – NOTA TÉCNICA 001/2019/GEMAT/GGTPS/ANVISA – Assuntos de Petições da Gerência de Tecnologia de Materiais de Uso em Saúde Medical Device and FDA Regulations and Standards News 0
A Domain of the component (EASA CM No.: CM-SWCEH-001 Issue 01 Revision 02) Point 9.3.3 (Usage domain aspects) EASA and JAA Aviation Standards and Requirements 0
A Microcontroller: Discussion about EASA CM - SWCEH - 001 EASA and JAA Aviation Standards and Requirements 1
A Boeing D1-4426 Code 001 - Supplier Compliance to Exhibit III AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 0
K Chromate Colour Issue - Yellow Chromate per AIPS02-05-001 Various Other Specifications, Standards, and related Requirements 4
T DITE(UK)-2011-IO-001 - Do you know anything about this code? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
J SR-001 Revision - Keep watching for the new revision of SR001 (It is expected soon) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
E Blueprint Notation Question - Print dimension like 4.312 g +.002 -.001 Quality Manager and Management Related Issues 14
Sidney Vianna IAQG Guidance PCAP 001 - Competence Management Guideline AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
E Source Inspection Minimum Qualifications - Complex PWAs and PWBs - SOW and J-Std-001 Career and Occupation Discussions 2
1 IPC/EIA J-STD 001 to 004 for Solder Specification Various Other Specifications, Standards, and related Requirements 9
L IPC/EIA J-STD-001 (soldering) revisions (Want to replace IPC-A-610) Various Other Specifications, Standards, and related Requirements 5
apestate Calibrating .001" resolution Dial Calipers - Calibration and safeguarding procedure General Measurement Device and Calibration Topics 6
K COPLANARITY: Composite profile tolerance on multiple surfaces- what does" lower dimensional reference frame tolerance" control? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
I Profile tolerance with a positional tolerance to a hole ? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
A % STUDY VARIATION VS. % TOLERANCE Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 31
L Validation without Tolerance Qualification and Validation (including 21 CFR Part 11) 0
A Need to calculate tolerance Intervals with a set of non-normal data and 3-Parameter Weibull distribution Using Minitab Software 0
Q Acceptable Tolerance - how to derive? Measurement Uncertainty (MU) 16
D Tolerance symbol Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
D Calibration tolerance question using Pipettes Medical Device and FDA Regulations and Standards News 3
M Determining a tolerance value for Measuring devices in-house inspection General Measurement Device and Calibration Topics 12
G 0.00005" tolerance on micrometer with 0.0005" resolution? General Measurement Device and Calibration Topics 4
N Drawing tolerance vs. Measurement device General Measurement Device and Calibration Topics 7
I Control Plan (Product/Process specification/ Tolerance) acceptance FMEA and Control Plans 27
J Mechanical inspection techniques of close tolerance parts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
Proud Liberal Small radius with tight tolerance on a torus General Measurement Device and Calibration Topics 4
L GRR for a tolerance that has changed Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 6
T How to justify this High %Tolerance Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
D Tolerance definition based on expected Cp/cpk Reliability Analysis - Predictions, Testing and Standards 14
N % Tolerance - Type 1 study on the gages, then a gage R&R (ANOVA) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
S Tolerance limits for micro balance General Measurement Device and Calibration Topics 1
J Determining SPC tolerance Statistical Analysis Tools, Techniques and SPC 21
Q % Study variation low, % tolerance high - GR&R Interpretation help Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
R Uncertainty in measurement larger than tolerance Measurement Uncertainty (MU) 2
WEAVER Electronic Weighing Scale Calibration Tolerance Manufacturing and Related Processes 1
I Determining Calibration Tolerance of a Measurement Device General Measurement Device and Calibration Topics 2
Ronen E Tolerance intervals (?) question - Flow Rate Probability Range Statistical Analysis Tools, Techniques and SPC 6
D Tolerance when verifying an analytical balance General Measurement Device and Calibration Topics 3
E Discussion between co-worker on tolerance and uncertainty and how to apply it. Thoughts? 17025 ISO 17025 related Discussions 1
C Thread Tolerance - Can I use a 3A Ring on a part that the drawing calls out for a 2A? Manufacturing and Related Processes 5
T GRR based on part tolerance or process variation. Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
A Excel OOT (Out of Tolerance) Formula Excel .xls Spreadsheet Templates and Tools 3
N Tolerance Interval plots in Minitab Statistical Analysis Tools, Techniques and SPC 0
S Question about a basic additive variance/tolerance example Quality Tools, Improvement and Analysis 2
G Assigning a calibration tolerance - An x-y coordinate machine - Uncertainty as my verification tolerance General Measurement Device and Calibration Topics 4
R Feeler Gauge Calibration - What tolerance is used for calibrating feeler gauges? General Measurement Device and Calibration Topics 4
S SPC (Statistical Process Control) for Unilateral Tolerance - Questions Statistical Analysis Tools, Techniques and SPC 6

Similar threads

Top Bottom