Automated and Manual Measurement System Comparison (Correlation)

tahirawan11

Involved In Discussions
Hi,

I have a production process where 100% of machines are tested for ‘airflow’ using an automated measurement system. And a small number of machines 5% are also re-checked manually to see if the 'automatic testing' and 'manual testing' yield the same results. In order to check the consistency of both the results, I plan to do a ‘simple correlation’ analysis. And if the ‘Coefficient of correlation’ is higher than 0.5 then I would conclude that both the ‘automatic’ and the ‘manual’ measurement systems are consistent. I do not want to compare one machine against the other but just want to compare the results of the two measurements systems.

The raw data is attached below and the coefficient of correlation is – 0.241.

Can I conclude that there is no correlation between the results of the two measurement systems? Are there any suggestions to use other tools than simple correlation analysis or other ways to conduct the same study?

automated_testing manual_testing

18,4 18,9
18,1 19,4
18,3 17,8
18,1 18,7
18,2 17,2
18 17,1
18,2 16,3
18,1 17,8
18,1 18,1
18,2 18,2
18,2 17,6
18,2 18,5
18,4 16,1
18,3 18,4
18,2 14,8
18,2 18,8
18,1 17,6
18,1 18,4
18 18,8
18,4 18,5
18,2 19
18,2 18,8
18,3 19,3
18,3 19
18,1 18,2
18,2 17,2
18,1 18,9
18,1 19,8
18,3 20

thanks

Bev D

Heretical Statistician
Staff member
Super Moderator
I understand that you don't want to compare instruments to each just the measurement methods. This is a standard Measurement System Analysis.
I caution you that 'simple regression' and correlation coefficients are not appropriate for MSA's because this approach is used for two data series, one of which is dependent on the other. When comparing two measurements - or two measurement systems as in this case - the two measurements are independent of each other. Therefore the graphical display and the 'correlation coefficient are different. The graphical display is a chart with a 1:1 (45 degree) line intersecting the origin and not the best fit line. The coefficient of correlation is the "Intraclass correlation coefficient".

In your case, the agreement between the two methods is so bad that the coefficient is - regardless of the formula - is irrelevant. but the GRAPH is.

I have plotted the values you provided on a Youden plot. (see attached file) I'm not sure which method is represented in your two columns. A is the first column and B is the second column. The two methods are not equivalent.

Attachments

• 687.8 KB Views: 173

tahirawan11

Involved In Discussions
Thanks Bev D. I have one question how should i interpret the graphical display. How does the chart look like if there is a PERFECT match between the two measurement systems? Does it mean that all points will lie on the 45 degrees line and any deviation from this diagonal line indicates a mismatch between the two measurement system?

Bev D

Heretical Statistician
Staff member
Super Moderator
A perfect match would have all of the dots on the 1:1 line. variation centered around the 1:1 line is simple measurement error. if the cloud of values is not centered on the 1:1 line there is a bias between the two instruments or measurements.

I have two papers posted at www.qualityforumonline.com in the Resources tab under "Practical Quality Engineering Resources"

MSA Tools is a spreadsheet with various MSA graphs and calculations.
MSA Verification & Validation is a paper describing how the various tools work and when to use them
These two are at the bottom of the list...

Near the top of the list is "The Statistical Cracks in the Foundation of the Popular Gauge R&R Method".

Thread starter Similar threads Forum Replies Date
Sub-supplier change from manual to automated process - same specs - Report to FDA? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
MSA for a part Automated and part Manual Inspection Machine IATF 16949 - Automotive Quality Systems Standard 7
L Dry Lube Application (Ever lube 620) - Manual Application vs. Automated Spray Process Manufacturing and Related Processes 1
What do you think about automated QA testing For software app industry? Misc. Quality Assurance and Business Systems Related Topics 5
Gage RandR on automated equipment. IATF 16949 - Automotive Quality Systems Standard 7
MSA on automated measurement system - Multiple Step Vision System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
Images of Product for Automated Process - How long to keep? US Food and Drug Administration (FDA) 2
Correct MSA study for an automated camera system which makes attribute inspection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
Applicability of Means of Protection, working voltage in an Automated External Defibrillator IEC 60601 - Medical Electrical Equipment Safety Standards Series 0
Tests IEC 60601-1 - Grounding Resistance, Leakage Current and Dielectric Rigidity in an automated way IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
GR&R on Automated Inspection Cell Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 15
What are you looking for in an automated quality management system? Quality Assurance and Compliance Software Tools and Solutions 8
F Automated Platform (IVD) Classification/Registration (New to RA) EU Medical Device Regulations 3
Is Automated Internal Auditing Possible? Internal Auditing 13
A Automated Measurement Machine MSA - Small Parts Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
B How to do an MSA on an Automated Torque / Angle System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
A Automated calibration of Digital Multimeter without communication port Calibration and Metrology Software and Hardware 2
Bench Marking Automated Data Collection Benchmarking 4
How to perform Gauge R & R on Automated Test & Measurement Equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 8
Looking for recommendations for an automated vision measuring system Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
M Automated Revision Updates - Notifications of new revisions for ISO's to subscribers Document Control Systems, Procedures, Forms and Templates 2
Tools Six Sigma for my Automated Machine Six Sigma 8
C Gage R&R on In-Process Automated Inspection Equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 13
21 CFR 820.30 - Automated with Computer Software - Applicable? US Food and Drug Administration (FDA) 5
S How to calibrate AOI (Automated Optical Inspection) for IOPQ Qualification and Validation (including 21 CFR Part 11) 3
P GR&R on automated test system which produces hundreds of results Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
C Multiple Variation Gage R&R for Automated Testers Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
J Classes for Hydraulic Calibration - Dead Weight Tester or Automated Controller General Measurement Device and Calibration Topics 1
Q Automated Test Software and System Validation 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
L Automated Calculators like EXCEL sheets for different processes Quality Assurance and Compliance Software Tools and Solutions 2
E MSA (Measurement System Analysis) Study on an Automated Process Quality Tools, Improvement and Analysis 4
Gage R&R on Automated Tool that supports Pass/Fail and some Parametric Testing Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
G Automated Production Line Quandary... FMEA and Control Plans 1
R Use of Automated Liquid Handling Robot for a PCR based Assay CE Marking (Conformité Européene) / CB Scheme 2
D Automated Vision Systems - Cost, Value and Payback of Implementation Statistical Analysis Tools, Techniques and SPC 2
I Gage R&R for Automated Measurement Equipment Systems Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
A Classification of Automated Gait Training Device for Rehabilitation EU Medical Device Regulations 5
R GAMP (Good Automated Manufacturing Practice) simplified please. Quality Manager and Management Related Issues 3
G Is Agilent?s automated network analyzer verification ISO/IEC-17025 compliant? ISO 17025 related Discussions 4
Gage Repeatability for an Automated Leak Tester Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 10
Automated System's Signature on Electronic Record - PART11 Qualification and Validation (including 21 CFR Part 11) 4
C Multiple Fixture Automated Vision System GR&R (Gage R&R) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
M GR&R (Gage R&R) for Automated Electrical Testing Equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
B Automated paint process control on conveyerized lines - Million dollar question Capability, Accuracy and Stability - Processes, Machines, etc. 14
Q Validating the AOI process (Automated Optical Inspection) Reliability Analysis - Predictions, Testing and Standards 9
S Automated regression testing software Software Quality Assurance 4
MSA (Measurement System Analysis) on automated machine Capability, Accuracy and Stability - Processes, Machines, etc. 1
B Automated Testing - R&R study - calculation Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
S Attribute Gage R&R - Automated Test Equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
R Attribute Gage R&R analysis - Automated test system Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5