Gage R&R study - Ordinal data

Tahirawan77

Starting to get Involved
#1
Hi,

I would like to perform a Gage R&R study where the data type is 'Ordinal' in nature. The defect can be categorized as No defect, Low, Medium and High

The process in question is drilling of holes in composite material. After the drilling process there is risk of making a 'splinter' at the edge of hole. The size of the splinter can vary from 'No splinter' to Low, Medium and High splinter. I cannot / do not want to measure the splinter size as this is not measured in the production due to time constraints. Instead each hole is inspected visually by QC inspector and rated as one of the above category. Low Splinter will not be reworked, any Medium splinter will be reworked where as High will be rejected.

I would like to plan the study as Attribute Gauge R&R study with 30 holes where holes with 'No Splinter' is marked as 'Good' and all other form of splintering can be categorized as 'Fail'. But by doing so I will miss the ability of the operators to identify different defect types (medium defect identified as High and vice versa).

I would like to know if there is any other way how I can perform the study where the data type is 'Ordinal' or any other suggestions to improve the Gage R&R study

Thanks
 
Elsmar Forum Sponsor

John Predmore

Trusted Information Resource
#3
The Kappa statistic is used to assess agreement of multiple reviewers producing categorical ratings. Cohen's Kappa is for two mutually exclusive categories, and Fleiss' Kappa for three or more categories. Repeatability of a reviewer is not the same metric as inter-reviewer agreement, but Kappa analysis may give you some useful information. If you are comparing reviewers to a known correct categorization result, you could also use Chi-square analysis.
 

Miner

Forum Moderator
Leader
Admin
#5
The Kappa statistic is used to assess agreement of multiple reviewers producing categorical ratings. Cohen's Kappa is for two mutually exclusive categories, and Fleiss' Kappa for three or more categories. Repeatability of a reviewer is not the same metric as inter-reviewer agreement, but Kappa analysis may give you some useful information. If you are comparing reviewers to a known correct categorization result, you could also use Chi-square analysis.
The various Kappa statistics are used for nominal (unordered) categories. For ordinal (ordered) data, perform an attribute agreement analysis then use Kendall's coefficient of concordance. Kappa only considers whether the nominal categories were matched or unmatched. Kendall's takes into consideration how close on the ordinal scale you were.
 
Last edited:

Tahirawan77

Starting to get Involved
#6
The Kappa statistic is used to assess agreement of multiple reviewers producing categorical ratings. Cohen's Kappa is for two mutually exclusive categories, and Fleiss' Kappa for three or more categories. Repeatability of a reviewer is not the same metric as inter-reviewer agreement, but Kappa analysis may give you some useful information. If you are comparing reviewers to a known correct categorization result, you could also use Chi-square analysis.
Thanks John for your reply but just to make sure i got it correct. Since i have four categories in my study, so i need to perform Fleiss Kappa?
My plan is to first give the samples to a 'specialist' and ask him to rate the samples and then give it to three operators to perform MSA. As your link points that there is no formal way of interpreting the results but i plan to use a score of 0.8 or higher for passing the MSA?
 
Last edited:

Miner

Forum Moderator
Leader
Admin
#7
Thanks John for your reply but just to make sure i got it correct. Since i have four categories in my study, so i need to perform Fleiss Kappa?
My plan is to first give the samples to a 'specialist' and ask him to rate the samples and then give it to three operators to perform MSA. As your link points that there is no formal way of interpreting the results but i plan to use a score of 0.8 or higher for passing the MSA?
No. Use Kendall's coefficient of concordance for ordinal data. Kappa is for nominal data. The results of the specialist will be your "known standard."

A score >/= 0.7 is good; >/= 0.9 is excellent
 
Last edited:
Thread starter Similar threads Forum Replies Date
J Ordinal Data Gage Study - Kappa scores vs. Kendell?s Coefficient Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
M Type 1 Gage r study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
T Attributes SPC study - Attributive control (Go gage) Statistical Analysis Tools, Techniques and SPC 5
S Capability or Gage R&R Study for Leak Tester? Reliability Analysis - Predictions, Testing and Standards 15
N % Tolerance - Type 1 study on the gages, then a gage R&R (ANOVA) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
M Definitive answer on Type 1 vs Type 2 vs Type 3 Gage Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 0
D Gage R&R Study on Load Cells - Large chemical blending tanks IATF 16949 - Automotive Quality Systems Standard 1
B Minitab Type 1 Gage Study on True Position Question Measurement Uncertainty (MU) 1
A Gage type 1 study on CMM Capability, Accuracy and Stability - Processes, Machines, etc. 2
D Do I need part variation while doing Destructive Variable Gage R&R MSA study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 19
A Does an attribute study (Go/NoGo pin gage) need two studies (1 for each pin) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
B Attribute Gage R&R study according to "Charbonneau" Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
O Capability or Gage R&R Study for Leak Tester Reliability Analysis - Predictions, Testing and Standards 11
B Linearity study on a non-linear gage General Measurement Device and Calibration Topics 7
M Repeatability issue in Gage R&R study for inner diameter of a bushing Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 16
T Gage R&R Study- Measuring a flexible part (compressed part) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
V Minitab - Attribute Gage Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
J Gage R&R Study (Crossed) - How to determine Process Tolerance Using Minitab Software 4
O Gage Study training explanation for Gage Users Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
K MINITAB 16 - Attribute gage study (Analytic method) and Gage R&R Study (Expanded) Using Minitab Software 2
N When and which gages require a Gage R&R study? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
J Excel spreadsheet for Gage R&R study with 9 appraisers Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
C P/T % of a gage R&R to 0 MMC Tolerance study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
P What is a Type 3 Gage Study? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
D Reducing Within Part Variation during a Gage R&R Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 26
B "Historical std deviation" (Standard Deviation) in Gage R&R study Using Minitab Software 6
B Performing a Gage R&R Study with Minitab 13 Using Minitab Software 5
Z How many parts to measure in a Gage R&R study Quality Tools, Improvement and Analysis 5
G Ford - Type 1 Gage Study MSA (Measurement Systems Analysis) acceptable? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
M Sample Selection for a Gage R&R Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 6
G What is the right sequence for a Gage R&R study? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
H Type 1 Gage Study Interpretation Question Using Minitab Software 7
S Minitab: Type 1 Gage Study vs. Gage R&R Study Using Minitab Software 5
S Does a CMM (Coordinate Measuring Machine) require Gage R & R study? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
D AIAG 4th Edition Capability Study and Gage R&R Requirements Document Control Systems, Procedures, Forms and Templates 8
L Gage Linearity and Bias Study Analysis - Minitab help needed Using Minitab Software 4
B Can the Variation on Data be too small in a Variables Data Gage R&R Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
7 Gage R&R Study (Nested) - Feedback and Comments appreciated Quality Tools, Improvement and Analysis 1
D Quality Digest article - "An Honest Gage R&R Study" - Where is he getting the d2*? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
N Repeatabilty + Reproducibility (Gage R&R) Study - Inter-Laboratory Study (ILS) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
J Gage R&R Study by Attribute (ACCEPT-REJECT-INVESTIGATE) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
H GRR (Gage R&R) Study for Digital Temperature Gauge Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
D Small Variation in Thickness Measurements in Gage R&R Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 23
S Gage R&R (GRR) Study - Please comment on the results Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
M Type 1 Gage Study - What's your opinion on this gage? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
B Minitab 15 Gage R&R Study Nested understanding Using Minitab Software 1
B Study Variation and Discrimination Ratio in Gage R&R ANOVA Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
M Gage R&R Short Study vs. Long Study vs. Other Studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 25
T Gage R & R Study "Best Practices" - More than three operators? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 8
Miner Intro to MSA of Continuous Data – Part 7: R&R using Wheeler’s Honest Gage Study Imported Legacy Blogs 17

Similar threads

Top Bottom