Attribute MSA Study using Kappa - Why Kappa value is low

C

Chua Kwan Teng

#1
Hi all

I have a question pertaining to Attribute MSA Study using Kappa

In attribute study; a "1" represent an acceptable decision and a "0" represent an unacceptable decision.

I found that if there is a lot of "1" (say almost 90% )between Appraisers and very little "0"; the calculated kappa value is very low. Probability less than 10%.

Can any one please help to explain why Kappa value is low when there is a alot of "1" decision and very little "o" decision.

Thanks in advance

KT:read:
 
Elsmar Forum Sponsor

antoine.dias

Quite Involved in Discussions
#2
Re: Attribute MSA Study using Kappa

Do you mean the study in the MSA manual?
In this case the kappa value is a lot higher than 10%.

In case you have made your own study it would be better to attach your data or your study here so people in the cove can evaluate.

Best regards,

Antoine
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#3
yes, please attach your data. without it, I can't tell what is happening to you...
 
W

waja4803

#4
Hi, :bigwave:

From my understanding, Kappa is to determine on the agreement level of decisions between appraisers. It has nothing / little to do with the "1" and "0" decision. When the is many mismatch decision between appraisers, your kappa tends to be low. Kappa close to 1 showing there is very strong agreement between appraisers (BUT not necessary right decision).

Please right me if I'm wrong.:rolleyes:
 
Y

Yew Jin

#5
Hi guys,

When we get kappa = 1, it means it is perfect agreement. When we get kappa = 0, the agreement is the same as would be expected by chance. The higher the value of kappa, the stronger the agreement. Negative values occur when agreement is weaker than expected by chance, but this rarely happens. Depending on the application, kappa less than 0.7 indicates that the measurement system needs improvement. kappa values greater than 0.9 are considered excellent.:biglaugh:
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#6
Actually Kappa can be used for
1. ability to detect truth for each 'inspector'
2. within inspector repeatability and
3. between inspector reproducibility (or what some would term "agreement")

I use it for all 3 reasons on a regular basis.

For 1. you need a set of known standards, acceptable, marginal nad rejectable. you have the inspector inspect the set under normal inspection conditions. Then calculate Kappa with the known state as one of the "judges results" This test can be modified based on what you actually want to measure: if you are only testing the inspector's inate ability to make the correct call (given the appropriate training), you can use a sample set with any proportion of good marginal and rejectable parts. If you are conserned with their ability to make the correct call under typical inspection conditions (your real world) you need to have a sample set that is proportional to the current occurence rates of good, marginal and rejectable parts adn you must have the inspector inspect the set at the speed at whihc they woudl normlaly have and wiht the same lighting, magnification, etc.

2. You can - and should - then test the same inpsector with the same set of samples again (presented in a different order) some time later - a couple of days or weeks later. Then calcualte the Kappa wiht the judges resutls being the first time and the second time: this gives you repeatability of the inspector.

3. repeat the above 2 across inspectors and use one set of resutls from each inspector (typically their first results to minimize learning bias) and you have reproducibility....
 
W

waja4803

#7
:topic:

Hi, I am just starting to use the Attribute MSA method in my company.

According to MSA 3rd Edition Manual : Cross-Tab Method, page 128, the number of samples set to be 50, with 3 appraisers and 3 trials.

The question is, can we have smaller number of sample, operator and trial to get the same effectiveness of Attribute MSA? If yes, how small can we go?

There is some concerns from the users that the number of Sample (50) is hard to get and time consuming.:nope:

Very much appreciate for any feedback.:thanx:
 

Miner

Forum Moderator
Staff member
Admin
#8
Attribute data requires a much greater sample size than variable data. For attributes, 50 should be considered a minimum. Smaller sample sizes will greatly increase the uncertainty of the results.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#9
on the other hand it isn't necessary to have 3 trials...that's really overkill. In fact in the literature I've never seen 3 trials recommended. I've done - or managed - hundreds of Kappa studies, never done 3 trials and had great success with the method.

The sample size that is needed is dependent on what you are trying to do - review the existing threads here on Kappa for more info.

I've successfully used sample sizes ranging from 8 to 500...depends on your intent.
 
P

PaddlerCelia

#10
Attribute Studies used to be carried out using only 20 parts. However if you need to meet the TS16949 standard you will need to follow the MSA 3rd edition Manual and use 50 parts.

You should choose your parts carefully so that about 50% are non-conforming. The study where Kappa values are low may have too few non-conforming parts.

Unofficially, our consultant will say that the move to 50 parts has not been beneficial because the attribute studies become hard work and costly so not as many are carried out.
Regards
Celia
 
Thread starter Similar threads Forum Replies Date
T Correct MSA study for an automated camera system which makes attribute inspection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
V MSA Attribute Data Analysis Study IATF 16949 - Automotive Quality Systems Standard 1
Manix MSA - How to Approach a Gauge R&R Go/NoGo Attribute Study Statistical Analysis Tools, Techniques and SPC 2
Rameshwar25 Acceptance Criteria for Attribute MSA Study Effectiveness Summary Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
V MSA (Measurement System Analysis) Doubt on Analytic Method (Attribute Study) IATF 16949 - Automotive Quality Systems Standard 8
S Separate attribute MSA study required for each Go/No Go gauge? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
A MSA - Is an attribute study required for visual inspection? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
O Attribute MSA Study Requirements? Marginally good & marginally bad with good & bad Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
J How to do attribute data GR&R (Gage R&R)? Do a Kappa study or follow the MSA book? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
A GR&R (Gage R&R) - MSA - Section 6 Attribute Gage Study - Short Method Requirements Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
M Attribute MSA-CrossTab Study Example in MSA Manual:How to compute the Expected Count? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
J AIAG's MSA Manual 3rd Edition Attribute Gage Study - Calculating the UCI and LCI Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 10
S MSA for attribute relation gage Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
I Overwhelmed with attribute MSA requirement for visual inspection IATF 16949 - Automotive Quality Systems Standard 8
H Need MSA 4th ed. compliant attribute MSA template General Measurement Device and Calibration Topics 4
J MSA on Part Specific Attribute Fixtures Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
F MSA - Measurement System Analysis Attribute Data Crosstab sheet APQP and PPAP 1
S Destructive Attribute MSA for Simple Water Leak Check in Trailers Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
J Attribute MSA - Number of samples Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
E MSA of Attribute Gauges Signal Detection Method Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
C MSA-4 Attribute Kappa Template - Verification Check Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
D Requiring a "bad" part for attribute MSA studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
D Attribute Agreement Analysis - MSA - False Negative/Positive IATF 16949 - Automotive Quality Systems Standard 3
N How to make a MSA for an Attribute Gauge if the Pp=Ppk=1.33 Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
J Attribute Data MSA for more than 3 Operators Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
J MSA for Attribute Visual Inspection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
G MSA Gage R&R and Attribute Forms Attached Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
N Designing an attribute MSA for a new piece of metrology equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
C Attribute Measurement System Analysis (MSA) Template Document Control Systems, Procedures, Forms and Templates 0
J The appropriate Attribute MSA for Automatic Visual Inspection Machine Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
M Attribute Gage MSA - What is the width of the gray area of ​​doubt? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
S Attribute MSA Visual Inspection Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
B How many Samples are required in Attribute Data MSA Studies? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
C MSA on Attribute Data - Can use Different Parts with Different Cosmetic Issues? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
S Cross Tabulation in Attribute MSA - Help Needed Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
S What is the Reference Value when we do Attribute MSA Studies? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 17
S Sample Size for Attribute MSA (Measurement System Analysis) in 4th Edition Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
S Choosing Parts for Attribute MSA (Measurement System Analysis) in 4th Edition, pg 132 Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
B Attribute MSA for Visual Inspection via Camera Automatic Inspection Tool Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
automoto Attribute MSA (Measurement System Analysis) - .xls Form attached Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 27
V Questions on Variable and Attribute MSA Interpretation Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
O How many bad parts needed for Attribute MSA? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
B Attribute MSA of Automatic Inspection Equipment Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
I Attribute MSA for 3 factors Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
R Destructive Attribute MSA (Measurement System Analysis) Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
P MSA Attribute Studies - Calculate Cross Tabs, Effectiveness, Miss Rate, False Alarm Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 10
W Attribute MSA (Measurement Systems Analysis) for Low Volumes Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 7
C AIAG Attribute MSA - Analytical Method - Page 135 of MSA manual 3rd edition Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 19
L MSA 3rd Edition - Attribute Data - Page 134 Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 6
R MSA Attribute gage repeatability and reproducibility IATF 16949 - Automotive Quality Systems Standard 1

Similar threads

Top Bottom