How many measurement data decimal positions to report and be in conformance

F

Fralem

#1
Does anybody know if there is a standard on how many positions to report after the decimal?

Here is the situation: We have a technical drawing that states a dimension of X.XXX'' +/- 0.005'' and we have conflicting opinions on the subject as to how many places after the decimal must be reported back on the inspection report. Some say that the drawing says three so report three, others say that an extra position should be reported. So in the example above should we report 3 or 4 positions after the decimal.

That then leads me to another question: Either of the possibilities above what then happens if the reported position is the fourth (X.XXXX'') and the fifth or sixth position when I run the inspection routine would make the part out of specs. Is the part no good or are these positions to far out to matter as the technical drawing states 3 places?

Please enlighten me:)

Frank
 
Elsmar Forum Sponsor

Statistical Steven

Statistician
Leader
Super Moderator
#2
Re: How far must the mearsurements be reported and be in conformance

Does anybody know if there is a standard on how many positions to report after the decimal?

Here is the situation: We have a technical drawing that states a dimension of X.XXX'' +/- 0.005'' and we have conflicting opinions on the subject as to how many places after the decimal must be reported back on the inspection report. Some say that the drawing says three so report three, others say that an extra position should be reported. So in the example above should we report 3 or 4 positions after the decimal.

That then leads me to another question: Either of the possibilities above what then happens if the reported position is the fourth (X.XXXX'') and the fifth or sixth position when I run the inspection routine would make the part out of specs. Is the part no good or are these positions to far out to matter as the technical drawing states 3 places?

Please enlighten me:)

Frank
Frank you measure to the 4th decimal place and report on the record to the 3rd decimal place.

For example:

If you measure 5.4568 you report 5.457 which is compared to the specification.

If you measure 5.4562 you report 5.456 which is compared to the specification.
 

Kingsld1

Involved In Discussions
#3
If you are working to ANSI Y14.5 then technically any amount above or below the stated limit is out of tolerance.

Personally, I would prefer to have parts that were not pushing the edge of the tolerance, even though my samples were just barely in tolerance. Keeps the customer from finding that one sample part that is just out of spec. (Been on both sides of that)
 

Jim Wynne

Leader
Admin
#4
Does anybody know if there is a standard on how many positions to report after the decimal?

Here is the situation: We have a technical drawing that states a dimension of X.XXX'' +/- 0.005'' and we have conflicting opinions on the subject as to how many places after the decimal must be reported back on the inspection report. Some say that the drawing says three so report three, others say that an extra position should be reported. So in the example above should we report 3 or 4 positions after the decimal.

That then leads me to another question: Either of the possibilities above what then happens if the reported position is the fourth (X.XXXX'') and the fifth or sixth position when I run the inspection routine would make the part out of specs. Is the part no good or are these positions to far out to matter as the technical drawing states 3 places?

Please enlighten me:)

Frank
This question comes up surprisingly often. The question has a simple answer, though. If your upper tolerance limit is (for example) 1.005 and your measurement is 1.0051 (or 1.0050000...1, you have exceeded the upper tolerance limit, which is an expression of what size the thing must be in order to be considered acceptable.

In actual practice, how you go about dealing with this situation depends on factors not in evidence here, and the approach suggested by Statistical Steven is often used. Be assured, though, that there is no general industry standard that it's OK to exceed a maximum value or fall below a minimum.
 

pkost

Trusted Information Resource
#5
In actual practice, how you go about dealing with this situation depends on factors not in evidence here, and the approach suggested by Statistical Steven is often used. Be assured, though, that there is no general industry standard that it's OK to exceed a maximum value or fall below a minimum.
As long as you have documented how you make and record the measurement and justify why it is effective.
 
#6
As long as you have documented how you make and record the measurement and justify why it is effective.
???? - are you suggesting that there's a documented work instruction (or similar) on how to make a measurement? What makes you suggest such a thing is necessary? If you'd share that with us so we can better understand.
 
#7
If you are working to ANSI Y14.5 then technically any amount above or below the stated limit is out of tolerance.

Personally, I would prefer to have parts that were not pushing the edge of the tolerance, even though my samples were just barely in tolerance. Keeps the customer from finding that one sample part that is just out of spec. (Been on both sides of that)
Good point about the parts 'pushing the limit' - plus, no-one has mentioned the impact on the observed measurement the device's resolution can have. The resolution should be 4:1 (as a minimum) or 10:1 (better) to allow this determination to be helpful. I think Steven came close, but not how you measure and report to those digits.
 

pkost

Trusted Information Resource
#8
???? - are you suggesting that there's a documented work instruction (or similar) on how to make a measurement? What makes you suggest such a thing is necessary? If you'd share that with us so we can better understand.

My mistake - it appears that I misread the quote.

I do however believe it is important to specify the accuracy required in measurement especially when extreme accuracy is required. It leads to less argument or confusion!
 

Jim Wynne

Leader
Admin
#9
My mistake - it appears that I misread the quote.

I do however believe it is important to specify the accuracy required in measurement especially when extreme accuracy is required. It leads to less argument or confusion!
The question isn't about accuracy/precision per se, although it's something to consider when near the spec limits. The question, essentially, is whether 1.0051 is greater than 1.005, assuming accurate measurement.
 

Statistical Steven

Statistician
Leader
Super Moderator
#10
Good point about the parts 'pushing the limit' - plus, no-one has mentioned the impact on the observed measurement the device's resolution can have. The resolution should be 4:1 (as a minimum) or 10:1 (better) to allow this determination to be helpful. I think Steven came close, but not how you measure and report to those digits.
Andy...that is my point. The measurement system has to have at least 10:1 resolution, so why report out more digits than required by the specification? I take a hard line stance on signficant figures especially in the FDA regulated industry where reporting out to more or less significant figures is an audit finding.
 
Thread starter Similar threads Forum Replies Date
M How many parts do I choose for an MSA (Measurement System Analysis)? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
S Gage R&R Studies - 1200 operators, 2000 measurement tapes and many characteristics Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
Z Determining how many measurement devices need gage R&R? MSA Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 13
R How many process maps are adequate for a company for IATF 16949? Process Maps, Process Mapping and Turtle Diagrams 6
V How many hrs does it take to create a Technical File (TD)? EU Medical Device Regulations 4
G How many Design FMEAs do you have? FMEA and Control Plans 4
V Literature search over how many years? EU Medical Device Regulations 8
S How many tester quantity we need on the line based on the cycle time and peak volume Manufacturing and Related Processes 3
M R&R Studies - how many? per product? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
J How many majors is one company allowed ? IATF16949 Manufacturing and Related Processes 16
M Summative evaluation: how many sample per participant? IEC 62366 - Medical Device Usability Engineering 5
J How many hours for a NB to review Technical Documentation? EU Medical Device Regulations 6
T Too many NC's/ Deviations and CAPAs galore Nonconformance and Corrective Action 10
B How many nonconformances have you received on average during IATF audits? General Auditing Discussions 27
G Too many customer complaints Customer Complaints 16
I How many gauges in a Go/No Go Gauge? General Measurement Device and Calibration Topics 7
BeaBea How Many Processes should be created for each Department? Process Maps, Process Mapping and Turtle Diagrams 5
Watchcat How Many Notified Bodies are "Enough" Already? EU Medical Device Regulations 23
S How many mandays required for the below mentioned standards IEC 27001 - Information Security Management Systems (ISMS) 3
N MDR review process by notified body - How many steps exist in the review process EU Medical Device Regulations 0
F How many signatures do we need on calibration certificates? ISO 17025 related Discussions 8
M Informational From RAPS – Operating System Vulnerabilities in Many Medical Devices, Germany’s BfArM and Device Firms Warn Medical Device and FDA Regulations and Standards News 0
R Validation of Medical Device Hardware containing Software - How many to Validate ISO 13485:2016 - Medical Device Quality Management Systems 1
K How Many Samples to send for Chemistry Testing Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 3
N MDSAP Participants 2019 - How many companies are participating YTD? ISO 13485:2016 - Medical Device Quality Management Systems 20
S How many of you are using Robotic process automation for calibration lab management? ISO 17025 related Discussions 0
G Measuring customer parts on a CMM - How many decimals to report to the customer? ISO 17025 related Discussions 28
J Supplier choice limitation - Many of them dont have PSCR, VDA or even ISO 9001 VDA Standards - Germany's Automotive Standards 0
O Clean room performance qualification (pq) - How many times can I do each test? Manufacturing and Related Processes 8
G How many samples you require to test during PQ for validation of a equipment? ISO 13485:2016 - Medical Device Quality Management Systems 1
D IATF 16949: Do we have too many processes? Are they all the same? Registrars and Notified Bodies 7
D QMS Performance Based on how many NCRs were generated month to month Nonconformance and Corrective Action 8
S How many QMS processes and procedures? ISO 13485:2016 - Medical Device Quality Management Systems 2
L How many of you are Certified to IATF 16949 IATF 16949 - Automotive Quality Systems Standard 12
G How many Uncertainty Measurements for Micrometers and Calipers? Measurement Uncertainty (MU) 3
R How many Corrective Action Request can be Issued by DCMA for a single issue? Nonconformance and Corrective Action 2
A Training Program Help - Old docs, new docs, so many docs... ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
M How many Pieces to Test - Sterility Testing USP EU Medical Device Regulations 2
P Are we an outlier for so many variations of Change Management? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
M SPC with many data points from one dimension Statistical Analysis Tools, Techniques and SPC 16
S How many times do you have the term 'Requirements' in your Quality Manual? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
L How many working hours to required to implement ISO 14001 ? ISO 14001:2015 Specific Discussions 6
B ISO/TS16949 Internal Auditing - How many auditors? Internal Auditing 4
R Training Production Employees When We Have SOOO Many SOPs Training - Internal, External, Online and Distance Learning 9
P Scope Extension Audit - How many days? Do I audit all AS9100 requirements? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
J How can I manage Occurence Rates in FMEA efficiently when I have many processes? FMEA and Control Plans 9
V How many times the quality agreement has been invoked? ISO 13485:2016 - Medical Device Quality Management Systems 3
C How to document many document control systems with one Document Control Procedure? Document Control Systems, Procedures, Forms and Templates 6
A How many laboratories are accredited to ISO 17025 in the USA? ISO 17025 related Discussions 5
R Is it possible to randomly select samples from 1 column to many column Using Minitab Software 2

Similar threads

Top Bottom