{"id":4472,"date":"2015-02-08T03:18:12","date_gmt":"2015-02-08T08:18:12","guid":{"rendered":"http:\/\/www.thejuliagroup.com\/blog\/?p=4472"},"modified":"2015-02-08T03:18:55","modified_gmt":"2015-02-08T08:18:55","slug":"computing-kappa-is-a-piece-of-cake","status":"publish","type":"post","link":"https:\/\/www.thejuliagroup.com\/blog\/computing-kappa-is-a-piece-of-cake\/","title":{"rendered":"Computing Kappa is a Piece of Cake"},"content":{"rendered":"

\"baby<\/a><\/p>\n

Kappa is a useful measure of agreement between two raters. Say you have two radiologists looking at X-rays, rating them as normal or abnormal and you want to get a quantitative measure of how well they agree. Kappa is your go-to coefficient.<\/p>\n

How do you compute it? Well, personally, I use SAS because this is the year 2015 and we have computers.<\/p>\n

Let’s take this table, where 100 X rays were rated by two different raters as an example:<\/p>\n

Rating by \u00a0 Physician 1<\/p>\n

————-Abnormal \u00a0 | \u00a0Normal<\/p>\n

Physician 2
\n————————————–<\/p>\n

Abnormal \u00a0 \u00a0 \u00a0 \u00a040 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 20<\/p>\n

Normal \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 10 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a030<\/p>\n

 <\/p>\n

So ….. the first physician rated 60 X-rays as Abnormal. Of those 60, the second physician rated 40 abnormal and 20 normal, and so on.
\nIf you received the data as a SAS data set like this, with an abnormal rating = 1 and normal = 0, then life is easy and you can just do the PROC FREQ.<\/p>\n

 <\/p>\n

Rater1 \u00a0 \u00a0Rater2<\/p>\n

1 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a01
\n1 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 1<\/p>\n

and so for 50 lines.<\/p>\n

 <\/p>\n

However, I very often get not an actual data set but a table like the one above. In this case, it is \u00a0still relatively simple to code<\/p>\n

DATA compk ;<\/p>\n

INPUT rater1 rater2 nums ;<\/p>\n

DATALINES ;<\/p>\n

1 1 40
\n1 0 20
\n0 1 10
\n0 0 30
\n;<\/p>\n

 <\/p>\n

So, there were 40 x-rays coded as abnormal by both rater1 and rater2. \u00a0When rater1 = 1 (abnormal) and rater2 = 0 (normal), there were 20, \u00a0and so on.<\/p>\n

The next part is easy<\/p>\n

PROC FREQ DATA = compk ;<\/p>\n

TABLES rater1*rater2\/ AGREE ;<\/p>\n

WEIGHT nums ;<\/p>\n

 <\/p>\n

That’s it. \u00a0The WEIGHT statement is necessary in this case because I did not have 100 individual records, I just had a table, so the WEIGHT variable gives the number in each category.<\/p>\n

This will work fine for a 2 x 2 table. If you have a table that is more than 2 x 2, at the end, you can add the statement<\/p>\n

TEST WTKAP ;<\/p>\n

This will give you the weighted Kappa coefficient. If you include this with a 2 x2 table nothing happens because the weighted kappa coefficient and the simple Kappa coefficient are the same\u00a0in this case.<\/p>\n

See, I told you it was simple.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Kappa is a useful measure of agreement between two raters. Say you have two radiologists looking at X-rays, rating them as normal or abnormal and you want to get a quantitative measure of how well they agree. Kappa is your go-to coefficient. How do you compute it? Well, personally, I use SAS because this is…<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[9,11],"tags":[],"class_list":["post-4472","post","type-post","status-publish","format-standard","hentry","category-software","category-statistics"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts\/4472","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/comments?post=4472"}],"version-history":[{"count":2,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts\/4472\/revisions"}],"predecessor-version":[{"id":4475,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts\/4472\/revisions\/4475"}],"wp:attachment":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/media?parent=4472"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/categories?post=4472"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/tags?post=4472"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}