Measurement Error When Measuring a Defect - Scratches

S

statdoug

#1
Normal evaluation of measurement systems usually involves comparison to process variation or to tolerance. What can one use when what you are measuring is a defect rather than a characteristic of a product or process.

As an example, a scratch that is less than 0.200mm (in length) is acceptable. A scratch that is greater than 0.200mm but less than 0.500mm is a minor defect, greater than 0.500mm but less than 1.000mm is a major defect, and greater than 1.000mm is a critical.

The distribution of scratch (lengths) is exponential, with most items having none or microscopic scratches.
 
Last edited by a moderator:
Elsmar Forum Sponsor

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#2
I don't understand this question. Evaluating attributes (scratches) is a visual inspection, but it could use tools to measure scratch length or depth. In this way inspecting a scratch (an attribute) could be considered a variable inspection - though variable is typically size, volume etc.

Scratches can arise from handling or process problems, but in my view that doesn't affect the inspection step.

Your methods (a kind of combination of variable and attribute inspection) are not unusual. When doing nondestructive testing on welds, dye penetrant testing may reveal pits as dots of color on the powdery absorbent spray coating. A weld with such spots isn't defective (we were drilled to call them discontinuities) unless the spots are of a size, number and/or pattern that is compared to a defined requirement and judged on that basis.

Also, a small scratch on an automotive part may not be determined to be a defect unless it's in a place that would be visible to the customer.

And so I would not get caught up in terminology. You are inspecting for fitness for use, whatever that means to you and the customer.

My statistic-smart colleagues here can chime in with the statistical treatment of such an inspection process.
 
S

statdoug

#3
The measurements are for the purpose of categorizing the defect. Why they are categorized as they are is out of my realm of control. However, procedural documents specify that all measurements used in the classification of defects will have comprehensive MSA analysis. The actual defects in question are not scratches, but they will work for an example. My feeling is that the question comes down to one of probabilities of misclasifying the defect.

So again, the question is, rather than tolerance or process variation, What can/should one use to compare measurement error (R&R) to, when what you are measuring is a defect rather than a characteristic of a product or process.
 
Last edited by a moderator:

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#4
Okay, I think I understand you better now.

I think it is interesting to consider the requirement that "all measurements used in the classification of defects will have comprehensive MSA analysis." That is certain to be a challenge when inspecting articles with acceptability depending on a flaw (for lack of a better word) exceeding parameters.

Where is this requirement from?

I don't have a good answer for you, so I did some looking around and found this page from bexcellence. I am not affiliated with bexcellence.

I hope this helps!
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#5
The method you are looking for is "attribute" MSA with multiple raters, multiple categories. There are several different statistical analyses that can be used, although a KAPPA analysis is probably sufficient given teh types of defects you described. These analyses will essentially provide you with the probability of misclassifying the defects.

A great 'starter' article on this is "When Quality is a Matter of Taste, Use Reliability Indexes..." by David Futrell. It was published in Quality Progress.

A spreadsheet I’ve posted to do the math for KAPPA is here:
Kappa test class example.xls


A document that explains other statistical methods is here:

Determining Repeatability without Gage R&R

in the attached document "Measurement Systems Verification and Validation"
 
S

statdoug

#6
The categorization of defects determines actions taken on the process. none of the product that is seen by human eyes or handled in any way, goes back in to the process, even if it seems perfect. A mistake in the classification can have serious repercusions both in terms of effect of product on the customer, and in terms of cost of actions taken, depending on the type of error (over-estimation or under-estimation of the criticality).
 
S

statdoug

#7
Hi Bev,
Thanks for the info. I am going to have to read through it more carefully tomorrow. I am not sure I understand what the colors were about, but hopefully the measurement document will give a more complete explanation. It looks at first look-through like we are going by agreement of the apparaisers on the classification (discrete classes). Doesn't that loose a great deal of information available with the (relatively) continuous data that we can get from the actual measurements. Shouldn't we be able to evaluate the variation in the measurements to determine a distribution of the repeatability error, as well as evaluating reproducibility, linearity, and SEMI?

The problem that we have, I think, is determining an apriori decision rule on the acceptability of the system in the absence of a useable specification or process variation. I am trying to avoid presenting an opportunity for an arbitrary decision of +/- 0.00Xmm, in favor of at least a broadly accepted standard, but preferably a standard that has some rational justification behind it.
 
#8
Normal evaluation of measurement systems usually involves comparison to process variation or to tolerance. What can one use when what you are measuring is a defect rather than a characteristic of a product or process.

As an example, a scratch that is less than 0.200mm (in length) is acceptable. A scratch that is greater than 0.200mm but less than 0.500mm is a minor defect, greater than 0.500mm but less than 1.000mm is a major defect, and greater than 1.000mm is a critical.

The distribution of scratch (lengths) is exponential, with most items having none or microscopic scratches.
I think you are going about this in a manner which is causing you a problem. Scratches, if they are cosmetic, shouldn't be measured by variable. It's far better to treat them as an attribute and set inspection criteria and a method for viewing them.

I've done a lot of this type of thing in a previous life and doing it the way you are, will drive you nuts - eventually!:bonk:
 

somashekar

Staff member
Super Moderator
#9
I think you are going about this in a manner which is causing you a problem. Scratches, if they are cosmetic, shouldn't be measured by variable. It's far better to treat them as an attribute and set inspection criteria and a method for viewing them.

I've done a lot of this type of thing in a previous life and doing it the way you are, will drive you nuts - eventually!:bonk:
This should be the way.
You determine the lighting condition, the viewing angle, the time to see and the clear definitions for point defect, line defect and area defect. Tune this to surfaces based on requirement like always viewed areas, often viewed areas, seldom viewed areas, and inner areas from the customer perspective. Validate this process across several visual inspectors as well as marketing personnel. (please include the finishing process operator and supervisor) It takes time, but is worth it when you realize that it starts giving you consistant results.
 
Last edited:
S

statdoug

#10
Quickly, as I am off to work, In this application, the "scratches" are not just cosmetic, and the inspection method is out of my control. My "charter" is to set up and test a measurement system.
 
Thread starter Similar threads Forum Replies Date
M Measurement Error - How to determine what is acceptable? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
Marc Definition MPEE - Maximum Permissible Error for Length Measurement Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 0
B Measurement Error on pure Graphite Coating on Tubes - To Error or Not to Error Using Minitab Software 7
S How to define Permissible Error for Measurement and Test Equipment? Measurement Uncertainty (MU) 3
bobdoering Dr. Wheeler on "How Measurement Error Affects the Four Ways We Use Data" Measurement Uncertainty (MU) 0
P Measurement Error Requirements - 99% or 99.73% spread to represent the 'full' spread? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
I Gage Error vs. Dimensional Tolerance - Accuracy needed for measurement equipment Measurement Uncertainty (MU) 10
M Maximum Permissible Error for Monitoring and Measurement Devices General Measurement Device and Calibration Topics 19
Z Definition Measurement Error - Precise definition of Measurement Error Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 1
M Measurement Uncertainty in Optical Microscopy Measurement Uncertainty (MU) 1
B ISO 6508 and portable hardness measurement instruments General Measurement Device and Calibration Topics 0
C How to Establish the Calibration & Measurement Capability (CMC)? ISO 17025 related Discussions 1
Ron Rompen Measurement of residual magnetism General Measurement Device and Calibration Topics 2
M Surgical angle measurement guide device with an application software Medical Device and FDA Regulations and Standards News 1
M Disabling measurement data during fault conditions IEC 60601 - Medical Electrical Equipment Safety Standards Series 5
J Most versatile measurement device General Measurement Device and Calibration Topics 2
P Surface scratch measurement General Measurement Device and Calibration Topics 1
N Drawing tolerance vs. Measurement device General Measurement Device and Calibration Topics 7
Ron Rompen MSA on automated measurement system - Multiple Step Vision System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
P Thermo-Hygrometer - Measurement of 10DegC @10%RH ISO 17025 related Discussions 4
hogheavenfarm GDT Flatness measurement question Inspection, Prints (Drawings), Testing, Sampling and Related Topics 10
K Software Validation for Measurement Tools used in Process Validation ISO 13485:2016 - Medical Device Quality Management Systems 2
K Best Measurement Systems Demos in California? General Measurement Device and Calibration Topics 1
K Process Mapping - Inputs/Outputs/Detail Activities/Control points/Measurement? Process Maps, Process Mapping and Turtle Diagrams 4
B IEC 60601-2-10 - Accuracy of Pulse Parameters - Required Measurement Uncertainty IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
N Program or application for standard time measurement in industry from video surveillance? Human Factors and Ergonomics in Engineering 1
M Defining frequency of measurement tools callibration Calibration and Metrology Software and Hardware 3
T How to “validate/qualify” a new measurement equipment? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
Bev D Verification and Validation of Measurement Systems Misc. Quality Assurance and Business Systems Related Topics 0
F How do you add accuracies for tools with different measurement devices? General Measurement Device and Calibration Topics 1
M Viable count Measurement ISO Standard Other Medical Device Related Standards 1
J Gauge R&R on a torque measurement Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
W Air Quality Measurement Hardware and Software General Measurement Device and Calibration Topics 11
O Dimension Measurement Tool Recommendation General Measurement Device and Calibration Topics 21
G Does pitch/increment/resolution of a ruled scale apply to measurement uncertainty as line item? Measurement Uncertainty (MU) 10
Marco Bernardi Plane measurement with a 3D touch machine like a CMM. Calibration and Metrology Software and Hardware 3
Marco Bernardi Diameters measurement with 3D touch machine like a CMM Calibration and Metrology Software and Hardware 4
R Uncertainty in measurement larger than tolerance Measurement Uncertainty (MU) 2
R Measurement Uncertainties Budget for Thermometer Measurement Uncertainty (MU) 6
O Performance Measurement ISO 9001: 2015 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
O Monitoring performance - How do I determine performance measurement basis within my organization? Misc. Quality Assurance and Business Systems Related Topics 4
G What information to put on measurement Dimensional Results APQP and PPAP 7
D O Ring capability and measurement - What is the automotive 'norm' for capability studies on O Rings? General Measurement Device and Calibration Topics 3
F Electrical Metrology (Source/Generation and Measurement) General Measurement Device and Calibration Topics 1
G Gauge R&R on multiple dimensions using 3D measurement system Capability, Accuracy and Stability - Processes, Machines, etc. 6
F Measurement Audit and ILC for ISO 17025 Clause 7.7.2 - Comparison with results of other laboratories ISO 17025 related Discussions 0
Emran.mi Measurement system analysis - Can you help me about implementation MSA for CMM device Manufacturing and Related Processes 2
K ISO 17025:2017 clause 7.6.2 - Performing calibration of its own equipment shall evaluate the measurement uncertainty ISO 17025 related Discussions 6
Q How should I analyze measurement correlation between me and customer? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 12
I Determining Calibration Tolerance of a Measurement Device General Measurement Device and Calibration Topics 2

Similar threads

Top Bottom