# Accuracy Equation - Linear gage with a stated accuracy of (1.5+L/50) ?m L= mm

C

#### Curtis317

We have a linear gage with a stated accuracy of (1.5+L/50) ?m L= mm. How would you set up calibration tolerances over a range of 0-200 mm?

Would you insert the gage block dimension for L and use that for the tolerance of the measurement for the gage block?

Just wanting confirmation or maybe another way to look at the issue.

Thanks

#### dgriffith

##### Quite Involved in Discussions
Re: Accuracy Equation - Linear gage with a stated accuracy of (1.5+L/50) µm L= mm

We have a linear gage with a stated accuracy of (1.5+L/50) µm L= mm. How would you set up calibration tolerances over a range of 0-200 mm?

Would you insert the gage block dimension for L and use that for the tolerance of the measurement for the gage block?

Just wanting confirmation or maybe another way to look at the issue.

Thanks
Question: are you calibrating the linear gage with the gage block, or the gage block with the linear gage?
Sorry, don't do much in the dimensional areas.

#### Jen Kirley

##### Quality and Auditing Expert
Staff member
Hi Curtis,

Are you asking for how to decide on a calibration tolerance in order to control measurement uncertainty?

Calibration tolerances are generally based on classes as identified in ANSI/AMSE B89.1.5, where measuring devices are calibrated to accuracy levels based on customer requirements and fitness for use. I am not affiliated with BuyGage.

Does this help?

P

#### Pezikon

Re: Accuracy Equation - Linear gage with a stated accuracy of (1.5+L/50) µm L= mm

Code:
``````    Test Point    Tolerance
10%        20 mm   ± 1.9 µm
20%        40 mm   ± 2.3 µm
30%        60 mm   ± 2.7 µm
40%        80 mm   ± 3.1 µm
50%       100 mm   ± 3.5 µm
60%       120 mm   ± 3.9 µm
70%       140 mm   ± 4.3 µm
80%       160 mm   ± 4.7 µm
90%       180 mm   ± 5.1 µm
100%      200 mm   ± 5.5 µm``````
In (1.5+L/50), L is the testpoint. It is nominal. It is the size of the standard gage block you are using to test the linear gage.

#### dgriffith

##### Quite Involved in Discussions
Re: Accuracy Equation - Linear gage with a stated accuracy of (1.5+L/50) µm L= mm

Is the formula you gave correct? By the order of operations, at 50% scale [(1.5+L/50) µm L= mm], 100mm / 50 = 2mm. 2mm = 2,000um. 2000 + 1.5 = 2001.5um, not 3.5um.
Any thoughts? Am I seeing this incorrectly?
Are you using high grade blocks?

In any event, if you are using blocks to calibrate the linear gage, then the tolerance of the block (or assigned value) should be between 4 and 10 times better than the tolerance of the linear gage at each test point.

Using the tolerances you listed above, for 100mm, a UUT tolerance of 3.5um should have gage blocks with a tolerance of 0.875um or better.
I think that's equivalent to GGG grade 3 or B89 grade AS1.

Last edited:
P

#### Pezikon

Re: Accuracy Equation - Linear gage with a stated accuracy of (1.5+L/50) µm L= mm

Is the formula you gave correct? By the order of operations, at 50% scale [(1.5+L/50) µm L= mm], 100mm / 50 = 2mm. 2mm = 2,000um. 2000 + 1.5 = 2001.5um, not 3.5um.
Any thoughts? Am I seeing this incorrectly?
If you google "mitutoyo linear gage" you will inevitably find the spec sheet for their "linear gages". Accuracy is published as: (1.5+L/50)µm (L=mm). They sport a resolution of 0.5µm or 1µm and ranges such as 10 or 50 mm. It would be unreasonable to presume the accuracy is anywhere near 2mm with a resolution and range like that. With this accuracy equation, you forget about the units once the input value (L) has been properly scaled; proper scaling is denoted by: "L=mm". And whatever the result, put µm on the end; hence "(...)µm". I can understand the confusion since specifications are typically not formatted like that.

There's bound to be even more confusion when people realize that the calculated specification has greater precision than the resolution of the device/meter. That is to say, how is one supposed to discern 4.7µm with a meter resolution of 1µm? The meter will read 4 or 5µm. The mitutoyo specification also states that quantizing error is not included in the accuracy spec. If it were, this wouldn't be an issue. The answer is to simply round the calculated accuracy up to be consistent with the resolution of the meter.

SAT (System Accuracy Test) temperature according to AMS2750E AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
IEC 60601-2-10 - Accuracy of Pulse Parameters - Required Measurement Uncertainty IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
Analog Pressure Gauge with no EN837 accuracy class information General Measurement Device and Calibration Topics 5
IEC 60601-1 Essential Performance - Is the signal accuracy specification an essential requirement? IEC 60601 - Medical Electrical Equipment Safety Standards Series 4
Warning about Deltronic Gage Pins - The "Certification of Accuracy" document General Measurement Device and Calibration Topics 13
Posting Measuring Equipment Accuracy for User Information General Measurement Device and Calibration Topics 4
Gagemaker MT-3000 accuracy for calibration Calibration and Metrology Software and Hardware 2
CMM Accuracy Formula General Measurement Device and Calibration Topics 3
Calibration Accuracy In Relation To Reporting General Measurement Device and Calibration Topics 1
Acceptable calibration accuracy of a 60" linear measuring device General Measurement Device and Calibration Topics 16
Calibration certificates avowing accuracy beyond the uncertainty limits? Measurement Uncertainty (MU) 7
Jewelry vs. Normal Laboratory Balances - Accuracy and calibration General Measurement Device and Calibration Topics 2
Tape Measure, the right device for 2 or 3 decimals accuracy on prints General Measurement Device and Calibration Topics 8
Equipment for Oven Calibration, Uniformity and System Accuracy Tests AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
Opinions on the "Best" CMM for High Accuracy Machining General Measurement Device and Calibration Topics 9
AMS 2750E System Accuracy Test (SAT) Frequency AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
A Process to Validate the Accuracy of Test Reports (AS9100) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 38
D AMS 2750E - Digital Instrument Calibration Accuracy Requirements General Measurement Device and Calibration Topics 1
Soldering Station Accuracy out of specification 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
Help trying to explain accuracy procedure General Measurement Device and Calibration Topics 1
D Long length measurement of 18" up to 100" with an accuracy of .005 min General Measurement Device and Calibration Topics 8
Cry for help - FDA calibration requirement for accuracy & precision General Measurement Device and Calibration Topics 2
W Dimensional Measurement of 10% accuracy and 25% uncertainty standard General Measurement Device and Calibration Topics 6
UUT Accuracy as part of Uncertainty Budget Measurement Uncertainty (MU) 6
A Uncertainty Measurement - What this accuracy specification "1% + 10^5": means? General Measurement Device and Calibration Topics 2
L Inspection Equipment Accuracy - Measuring the diameters of small holes Measurement Uncertainty (MU) 5
E How does Non-Linear UUT affect TAR (Test Accuracy Ratio) Calculation? Measurement Uncertainty (MU) 3
G Digital Autoclave Accuracy Requirements ISO 13485:2016 - Medical Device Quality Management Systems 2
S AMS 2750E Heat Treat Furnace System Accuracy Test SAT Report question AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
A Accuracy for electrosurgical analyzer General Measurement Device and Calibration Topics 2
A Calculating Accuracy for a Multimeter General Measurement Device and Calibration Topics 1
Verification of weighing scale 150kg capacity 0.05g accuracy General Measurement Device and Calibration Topics 9
A Accuracy Calculation of Leakage Current for an Electrical Safety Analyzer General Measurement Device and Calibration Topics 5
Laser Precision Linear Measurement - Inherrent Accuracy of 10 -6 - Nice PDF Paper General Measurement Device and Calibration Topics 1
S Stated Accuracy of Portable CMM Arm and Real World Use Inspection, Prints (Drawings), Testing, Sampling and Related Topics 9
A Intrinsic Uncertainty and Accuracy of a Medical Electrical Safety Analyzer General Measurement Device and Calibration Topics 1
V CMM plane construction method types and accuracy differences for measuring Pos Tol Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
J Weighing Balance Accuracy for Plastic Moulded Parts variation study Manufacturing and Related Processes 3
Best X-Y Microscope Stage for Submicron Accuracy Capability, Accuracy and Stability - Processes, Machines, etc. 2
K Wilson Hardness Tester Model 525 Accuracy/Repeatability issues help General Measurement Device and Calibration Topics 4
Measurement Uncertainty and Equipment Accuracy Measurement Uncertainty (MU) 2
C Basic Internal Calibration Procedure to check the Accuracy of a Balance Calibration Frequency (Interval) 6
C Accuracy Problem on the Calibrator - Two Labs with different Results General Measurement Device and Calibration Topics 7
R How do I extend Correction factor of Calibration cert to compute instrument accuracy Measurement Uncertainty (MU) 1
P Measurement Equipment - 10:1, 4:1 - Is it resolution or accuracy General Measurement Device and Calibration Topics 1
Accuracy of a Measurement Device Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
C Simpson 260 Series 8 Equipment Accuracy General Measurement Device and Calibration Topics 5
Accuracy and Precision of Test Equipment required? General Measurement Device and Calibration Topics 2
K What does Data Logger Accuracy = ? 0.25% of Full scale ? 1 Digit Mean? General Measurement Device and Calibration Topics 4
PPAP-PSW Accuracy Concern - Laser Cut Holes In Formed Fabricated Steel Part APQP and PPAP 15