Automated and Manual Measurement System Comparison (Correlation)


Involved In Discussions

I have a production process where 100% of machines are tested for ‘airflow’ using an automated measurement system. And a small number of machines 5% are also re-checked manually to see if the 'automatic testing' and 'manual testing' yield the same results. In order to check the consistency of both the results, I plan to do a ‘simple correlation’ analysis. And if the ‘Coefficient of correlation’ is higher than 0.5 then I would conclude that both the ‘automatic’ and the ‘manual’ measurement systems are consistent. I do not want to compare one machine against the other but just want to compare the results of the two measurements systems.

The raw data is attached below and the coefficient of correlation is – 0.241.

Can I conclude that there is no correlation between the results of the two measurement systems? Are there any suggestions to use other tools than simple correlation analysis or other ways to conduct the same study?

automated_testing manual_testing

18,4 18,9
18,1 19,4
18,3 17,8
18,1 18,7
18,2 17,2
18 17,1
18,2 16,3
18,1 17,8
18,1 18,1
18,2 18,2
18,2 17,6
18,2 18,5
18,4 16,1
18,3 18,4
18,2 14,8
18,2 18,8
18,1 17,6
18,1 18,4
18 18,8
18,4 18,5
18,2 19
18,2 18,8
18,3 19,3
18,3 19
18,1 18,2
18,2 17,2
18,1 18,9
18,1 19,8
18,3 20


Bev D

Heretical Statistician
Super Moderator
I understand that you don't want to compare instruments to each just the measurement methods. This is a standard Measurement System Analysis.
I caution you that 'simple regression' and correlation coefficients are not appropriate for MSA's because this approach is used for two data series, one of which is dependent on the other. When comparing two measurements - or two measurement systems as in this case - the two measurements are independent of each other. Therefore the graphical display and the 'correlation coefficient are different. The graphical display is a chart with a 1:1 (45 degree) line intersecting the origin and not the best fit line. The coefficient of correlation is the "Intraclass correlation coefficient".

In your case, the agreement between the two methods is so bad that the coefficient is - regardless of the formula - is irrelevant. but the GRAPH is.

I have plotted the values you provided on a Youden plot. (see attached file) I'm not sure which method is represented in your two columns. A is the first column and B is the second column. The two methods are not equivalent.


  • Youden for Airflow Elsmar.pptx
    687.8 KB · Views: 206


Involved In Discussions
Thanks Bev D. I have one question how should i interpret the graphical display. How does the chart look like if there is a PERFECT match between the two measurement systems? Does it mean that all points will lie on the 45 degrees line and any deviation from this diagonal line indicates a mismatch between the two measurement system?

Bev D

Heretical Statistician
Super Moderator
A perfect match would have all of the dots on the 1:1 line. variation centered around the 1:1 line is simple measurement error. if the cloud of values is not centered on the 1:1 line there is a bias between the two instruments or measurements.

I have two papers posted at in the Resources tab under "Practical Quality Engineering Resources"

MSA Tools is a spreadsheet with various MSA graphs and calculations.
MSA Verification & Validation is a paper describing how the various tools work and when to use them
These two are at the bottom of the list...

Near the top of the list is "The Statistical Cracks in the Foundation of the Popular Gauge R&R Method".
Top Bottom