{"id":1283,"date":"2011-04-19T02:12:40","date_gmt":"2011-04-19T07:12:40","guid":{"rendered":"http:\/\/www.thejuliagroup.com\/blog\/?p=1283"},"modified":"2011-04-19T02:17:04","modified_gmt":"2011-04-19T07:17:04","slug":"scoring-open-data-with-vvalue-the-best-function-id-never-heard-of","status":"publish","type":"post","link":"https:\/\/www.thejuliagroup.com\/blog\/scoring-open-data-with-vvalue-the-best-function-id-never-heard-of\/","title":{"rendered":"Scoring open data with VVALUE : The best function I’d never heard of"},"content":{"rendered":"

\"\"<\/a>Anyone who tells you they know all of SAS is like that creepy guy at the fraternity party who swears that his father is the Duke of Canada, that is, they have a perception of themselves that is not in screaming distance of contact with reality.<\/em><\/p>\n

Even though I have been using SAS for decades, I am still discovering new functions, tricks, tips and procedures all of the time. In fact, the one I came across today was so helpful, I even cross-posted this on SAScommunity.org <\/a><\/em><\/p>\n

My problem and how I solved it<\/strong><\/p>\n

There are currently over 370,000 datasets on the data.gov site, not to mention the numerous others available from the National Center for Education Statistics and many other open data sources. One problem users of these data often encounter is that the formats used by the creator are not necessarily those desired by the end user. For example, many files have user-defined formats for each individual item such as:<\/p>\n

value S9FMT
\n1 = “A”
\n2 = “B”
\n3 = “C”
\n4 = “D*”
\n6 = “NOT REACHED”
\n8 = “NOT ADMIN.”
\n9 = “OMITTED”
\n;
\nvalue S10FMT
\n1 = “A”
\n2 = “B”
\n3 = “C”
\n4 = “D”
\n5 = “E*”
\n6 = “NOT REACHED”
\n8 = “NOT ADMIN.”
\n9 = “OMITTED”
\n;<\/p>\n

A common desire would be to have the items that were not administered to the student have a missing value and the rest scored as either correct or incorrect. At first thought, using the PUT function to get the formatted value might seem like a good idea, but that would require specifying the format. Since many of these datasets include several hundred variables and several hundred different formats, that’s not going to work.<\/p>\n

Here is one solution using the 2007 dataset for eighth grade students from the Trends in International Mathematics and Science Study (TIMSS) :<\/p>\n

%include \"C:\\Users\\me\\Documents\\TIMSS\\Tformats.sas\" ;<\/code><\/p>\n

data scored ;
\nset in.G8_ACHIEVE07 ;
\nattrib Fval length = $9. ;
\narray rec{*}\u00a0 M022043 — BSSREA05 ;
\ndo i = 1 to dim(rec) ;
\nFVAL = vvalue(rec{i}) ;
\nif FVAL\u00a0 = “NOT ADMIN” then rec{i} = . ;
\nelse if index(FVAL,”*”) > 0 or FVAL = “CORRECT R” then rec{i} = 1\u00a0 ;
\nelse rec{i} = 0 ;
\nend ;<\/p>\n

The %INCLUDE statement includes the formats defined by the organization that created the dataset.
\n(TIMSS, like many of the original data sources, includes a folder of SAS files along with the data downloaded that read in text data, create formats and labels, merge files and perform other useful functions. )<\/p>\n

The VVALUE function returns the formatted value of the variable.
\nIn the program above, it is necessary to first recode the items that were not administered to have a missing value, and then score the students who were administered an item as having been correct or incorrect. In this particular example, all of the formatted values for correct responses either had an “*” next to the correct multiple choice value or the words “CORRECT RESPONSE” . Of course, this statement would need to be modified depending on the formatted values of your particular dataset.<\/p>\n","protected":false},"excerpt":{"rendered":"

Anyone who tells you they know all of SAS is like that creepy guy at the fraternity party who swears that his father is the Duke of Canada, that is, they have a perception of themselves that is not in screaming distance of contact with reality. Even though I have been using SAS for decades,…<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[15,9],"tags":[],"class_list":["post-1283","post","type-post","status-publish","format-standard","hentry","category-open-data","category-software"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts\/1283","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/comments?post=1283"}],"version-history":[{"count":4,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts\/1283\/revisions"}],"predecessor-version":[{"id":1288,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts\/1283\/revisions\/1288"}],"wp:attachment":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/media?parent=1283"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/categories?post=1283"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/tags?post=1283"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}