Is there a good way to compare two technicians measurements against each other?

The situation I have is this: I have one technician with whom management is not convinced in her measurement techniques. Right now, as a QE, mgmt. is making me go back in behind her and measure the same part again. The big problems here are that we're measuring an already machined die cast transmission main case and we're using a Romer arm to lay it out. I really have many other things I could be doing rather than going in behind a technician and remeasuring.

Is there some mathematical way to compare her measurements vs. my measurements, and maybe against the drawing specifications? Thanks in advance.

Is there a good way to compare two technicians measurements against each other?

The situation I have is this: I have one technician with whom management is not convinced in her measurement techniques. Right now, as a QE, mgmt. is making me go back in behind her and measure the same part again. The big problems here are that we're measuring an already machined die cast transmission main case and we're using a Romer arm to lay it out. I really have many other things I could be doing rather than going in behind a technician and remeasuring.

Is there some mathematical way to compare her measurements vs. my measurements, and maybe against the drawing specifications? Thanks in advance.


Hello NCYankee,

Welcome to The Cove Forums! :bigwave: :bigwave:

This appears more like a competency issue. Is there evidence that this person is indeed competent to perform the required tasks?

In addition, you may want to perform a Gage R&R. Search this Forum for applicable examples.

Stijloor.

Is there a good way to compare two technicians measurements against each other?

The situation I have is this: I have one technician with whom management is not convinced in her measurement techniques. Right now, as a QE, mgmt. is making me go back in behind her and measure the same part again. The big problems here are that we're measuring an already machined die cast transmission main case and we're using a Romer arm to lay it out. I really have many other things I could be doing rather than going in behind a technician and remeasuring.

Is there some mathematical way to compare her measurements vs. my measurements, and maybe against the drawing specifications? Thanks in advance.

Are you familiar with the general concept of Measurement System Analysis (MSA) and GR&R (Gauge Repeatability and Reproducibility)? It's what you're looking for. Do a search of the Cove--there's lots of information on the subject here. You may also want to avail yourself of a copy of the AIAG MSA manual (https://www.aiag.org/source/Orders/index.cfm?section=orders&task=3&CATEGORY=CORETOOLS&PRODUCT_TYPE=SALES&SKU=MSA%2D3&DESCRIPTION=&FindSpec=msa&CFTOKEN=56419417&continue=1&SEARCH_TYPE=FIND&StartRow=1&PageNum=1), which details the generally-accepted procedures for conducting measurement system studies.

Thank you both, Stijloor & Mr. Wynne. I am familiar with MSAs and GR&R. If I have to do it using one of the previously mentioned methods, I will. However, I was hoping there was something I could use with the data we already have in hand: Same piece, same dimensions checked, but only two persons, not the three people usually done by MSA & GR&R.

However, I was hoping there was something I could use with the data we already have in hand: Same piece, same dimensions checked, but only two persons, not the three people usually done by MSA & GR&R.

Hi NCYankee,

By all means - use the data you have on hand. Most of my customers who request a GR&R only require 2 operators.

Thanks CarolX. I'm presently searching for the GR&R & MSA stuff as previously suggested.

Thank you both, Stijloor & Mr. Wynne. I am familiar with MSAs and GR&R. If I have to do it using one of the previously mentioned methods, I will. However, I was hoping there was something I could use with the data we already have in hand: Same piece, same dimensions checked, but only two persons, not the three people usually done by MSA & GR&R.

You can do GR&R for two operators, and use existing data, so long as you can compare the results for both operators measuring the same part each time. If you don't want to do a new study (which I recommend), you can simply chart the data you already have to compare the measurement results of the two operators. This will probably just tell you what you already know, however. You really should compare the operators' results measuring the same parts, however.

We are measuring the same parts. Thanks again Mr. Wynne.

We are measuring the same parts. Thanks again Mr. Wynne.
You're welcome. I just want to be clear about what's meant by "same parts." I mean that you have some number of parts x, and operator A measures x1, and then operator B measures the same feature on x1, and so on. Each operator measures the identical part and feature.

Hello NCYankee,

Welcome to The Cove Forums! :bigwave: :bigwave:

This appears more like a competency issue. Is there evidence that this person is indeed competent to perform the required tasks?

In addition, you may want to perform a Gage R&R. Search this Forum for applicable examples.

Stijloor.

ahem - until there is data showing that 1. this operator's results are statistically & practically different from the other(s) and 2. that her measurements are the least accurate you - and the OP's management have no basis for assuming there is anything wrong with her.

this requires an MSA

You may not like the idea of more work, but I'd really suggest you also ought to do a check of the other technician also. You'd basically be trying to compare if the difference between your check and the questioinable techinician is significantly different from the difference between your check and the "good" technician.

You may also want to throw some known bad parts into this "test" to see if both can detect bad parts.

It's possible you could also look at past data from the two technicians, and check using p-chart control charts or binomial tests to see if their failure rates for "bad parts" are statisically the same or different. If we assume all parts have an equal chance of being bad, then both inspectors should have the same failure rate for bad parts, within statistical uncertainty levels.

if you have been redoing her measurements how often have you foudn a different result? how often has it changed the disposition of the material as pass/fail???

You may also want to throw some known bad parts into this "test" to see if both can detect bad parts.


This isn't necessary unless the measurements produce only attributes results. The expectation for a variables study (standard GR&R) is use of parts that reflect the range of variation in the process.

The quick comparison technique we may use is to study the average and standard deviation of the 2 sets of data with a same sample provided to 2 technician for few trials.

We might have the desire location (target) that we need to meet. Compare the 2 set of the sample average, are they significant difference and meet the target? (may use the 2 sample T test if the data is normal dist)

We may look into the data spread as well, are the spread significant difference and which is widen? (may use the F test if the data is normal dist)

We may know who is bias and precise in the quick way.
The average that far from the the desire target has more bias, however if the standard deviation is high mean he/she do not precise during test or measure a same unit in difference trials.

Most of the case is the method of measuring will give us major variable. We may standardize the method of data collection or measuring so that appraisal to appraisal variation is less. Document it properly so that everyone can follow the only method or technique to measure.

ahem - until there is data showing that 1. this operator's results are statistically & practically different from the other(s) and 2. that her measurements are the least accurate you - and the OP's management have no basis for assuming there is anything wrong with her.

this requires an MSA

A measuring device in the hands of an incompetent operator will have a negative impact on the MSA results. If a skills assessment shows that there is a competency issue, you don't need to perform an MSA to demonstrate that "something is wrong with her." In any statistical process (SPC, MSA, etc.) one is expected to first minimize unnecessary variation. Once all known sources of variation have been eliminated to the extent possible, further statistical analysis will demonstrate how "good" the system is.

Stijloor.

I might take a very simple approach. If you and the suspect technician measured the same part on the same instrument, then you can just do a paired t-test to see if there is a statistically difference. You will also need to determine a practical difference in case you find a statistical difference that is not meaningful.

A measuring device in the hands of an incompetent operator will have a negative impact on the MSA results. If a skills assessment shows that there is a competency issue, you don't need to perform an MSA to demonstrate that "something is wrong with her." In any statistical process (SPC, MSA, etc.) one is expected to first minimize unnecessary variation. Once all known sources of variation have been eliminated to the extent possible, further statistical analysis will demonstrate how "good" the system is.

Stijloor.

yes an "incompetant" operator will yield poor results. It will show - proof - in the data. on the other hand I inferred from the OP's choice of words that the idea that this operator is 'different' has no supporting data - just opinon. The OP stated that his repeat measurement "was a waste of his time". that led me to infer that he never found a different result - otherwise, I thought that if seh was making 'wrong enough' measurements then the OP would find different answers and his questions to us would be far different: he would have asked about making her better or performance management, instead he asks how to get the data to compare her performance to himself and/or the other technician.

I think we too often jump to "incompetant operators" based on no data but a lot of opinion...the OP's post appeared to me to be going in that direction. whatever happened to "innocent until proven guilty"? As a manager I would hate to claim that an employee was incompetant wihtout proof (and HR would kill me.) additionally without the baseline data how would we know that the training worked?