A few years ago, when I was at USC, I tried to get a desktop version of Enterprise Miner to run on a virtual machine on my Mac and that never happened, although I did get it working on a Windows machine I had at home.

Last week, I wrote about my failed attempt to get SAS Enterprise Miner from SAS On-demand to run on a Mac running Windows under boot camp.

Since then, I have successfully installed Enterprise Miner and started it using a Windows native machine.

Sadly, the same cannot be said for my Macs. Using boot camp on two different Macs, one running Windows 7 and another with Windows 8.1 I have had the same problems.

Be aware that if you are going to run Enterprise Miner on any operating system you are going to need at least some idea of what a C: prompt is and feel comfortable poking around things like .dll files.

You might think that this can be assumed and goes without saying if you are teaching, or even taking, a course in data mining. You would be wrong. Nothing can be assumed or goes without saying. Trust me on this.

I am not going to assume that you checked your configuration and the appropriate Java Runtime Environment is installed. If that is not the case,or you are not sure, go here and take care of that now. (See how this not assuming thing works?)

If that is taken care of, regardless of operating system, you will probably have a problem on Java security blocking the application from starting.  For me, changing Java security setting to medium fixed that on all 3 machines. I tried several other things that did NOT fix it. To find your Java security settings, you can go to the control panel (in Windows 8, search for control panel first) and then search for Java with control panel. Click on Java, then the security tab to find the slider to move to medium.

At this point, the Windows machine worked, even though I had to click on several boxes where Java asked me was I ***SURE*** I wanted to do this.

With the Mac though, after I click on Start SAS On Demand Software, Enterprise Miner – it downloads a main.jnlp file which when I open  it,  I eventually get a message an error exists in the user services configuration. You can see screenshots here The same exact problems occurred with both Mac computers running boot camp.

The ever-helpful Rebecca Ottesen said that two of her students using Macs last semester had the same problem and sent me an email directing me to this site.

So, I did a PROC OPTIONS in SAS, which I had loaded on my desktop and verified that the .dll file was located where expected

— and this led me to thinking, wait a minute, my students aren’t going to have SAS loaded on their computers so what are THEY going to do to troubleshoot.

That was kind of a moot point, though, because …

When I got to step 3 and type in the command as directed in the exact directory directed.

C:\Program Files (x86)\Java\jre1.6.0_24>java -fullversion

I get the error message ‘java’ is not recognized as an internal or external command, operable program or batch file.

Now, there could be any number of other things to try but the fact is, I have other things to do and the course is not for a few months. I will keep plugging away and keep you abreast here. If I do decide to go with Enterprise Miner in the fall, I am sure these posts will be helpful references for students.

I do want to advise anyone who is thinking about using the on-demand version of Enterprise Miner to be aware that you are definitely going to have at least a few problems with getting it installed, for example, the security thing, and if you have any students using boot camp, they are going to most likely hate you.

New semester coming up when I will be teaching data mining.  Because I never do anything at the last minute, I’m registering my course and testing the SAS on-demand for Enterprise Miner now.

I have learned from experience not to ignore it when the instructions say to check your configuration. You should find how to do that here.

http://support.sas.com/ondemand/emconfig.html

The first step is to open a command window and see if you have the appropriate Java Runtime Environment installed (JRE). Haven’t had to do anything from a C prompt in a while. On Windows 7, go to the start window at the bottom left of the screen and type in Command in the search box. Command prompt should pop right up.

I followed the instructions and it seemed my JRE was hunky-dory but the first time I started SAS On-Demand with Enterprise Miner it told me my Java was out of date. I went ahead and downloaded the latest version and installed it.

Your mileage may vary but total time getting up and running was about 2 minutes – but don’t get excited yet.

I clicked on start SAS Enterprise Miner and I got this message

Security warning from Java

I clicked RUN anyway but it was blocked from running.

So … I went into the control panel, typed in Java to search the control panel and in the Java security settings added the SAS on-demand login site as an exception. Still no luck.

Next, I went and changed the Java security settings from high to medium. A lot of people would not feel comfortable doing this, but I at least wanted to get Enterprise Miner to work. I could always set it back later.

At this point, I actually got Enterprise Miner to sort of start. That is, there were a couple of screens of security warnings I had to accept and then I got this error message.

error1

After this message, I got another saying the components failed to load.

failed to load

Perhaps, I thought, I should not have updated Java. So, I went back to the configuration instructions, checked to make sure I had a 32-bit JRE even though I have a 64-bit computer (check).

I went and downloaded the recommended version of the JRE from the Oracle site which required me to create an Oracle ID.

After that was downloaded and installed, I went to the Java control panel and disabled the later version so only one version of Java was installed …

and I got the same error!

 

I checked the SAS documentation and it recommended clearing the Java cache. I did that. I got further that time with lots of messages about downloading the application and verifying the application, but just when I was getting excited, it came up and asked me if I wanted to run with an older version of Java. I picked to continue with the older version. After some more security warnings, I got the same error messages as before.

So … I went through the whole circle again, cleared the cache, started again, selected the newer version of Java – and still the same messages.

It’s past midnight on Memorial Day weekend, so I’m not going to bother calling SAS technical support. I have a laptop running Windows 8, so I’m going to try installing it on that tomorrow and see if I have any better luck.

One thing is pretty clear – my students in the fall better have a lot more familiarity with computers than just pointing and clicking or they are going to have a really hard time – and I still haven’t gotten Enterprise Miner to run!

P.S. At one time, years ago, I had gotten SAS Enterprise Miner to run with version 1.6.0_18    so I tried that and it also failed to start with the same error messages.

I’m running on boot camp on a Mac.  I have a Windows laptop, so I’ll try it on that tomorrow also and see what happens.

I am suspecting this is going to be a disaster, but I’m hoping to be mistaken.

During the time since I started this series of posts on a little thing I knocked out one evening to illustrate long division, I’ve probably done a dozen other somewhat interesting pieces of code – I am sad that Java has co-opted the use of the word codelet because it is such a nice term for a bit of programming that is more than a function but not a real application. Anybody has a good word, let me know. While we’re on the subject of words, what exactly is the difference in Dreamweaver between an extension and a widget?

Anyway …. our games include hundreds of bits like this, where if a student misses a problem, he or she gets routed to a page to pick an option to study.

So … here is the rest of the story. Yes, it could have been done more beautifully, and when I go back and revise it, I think I will change the answer button instead of having two buttons to have one that is changed after the first onClick.

The DOCTYPE (html5) and title are pretty obvious.

All of our web pages have a container ID that is set in the style sheet. That makes all of the content fall within a defined window size, regardless of the screen size.

The w class is just so the background is white in the spot where the problem is. The Invisible Developer wanted some type of background and he liked the specky one.

You might wonder why something like w is a class instead of an ID if it is only used once. In fact, I simplified this example for the blog. Actually the w class is in an external style sheet so their could be pages with more than one element using this same style.

As a commenter on an earlier post pointed out (thank you!) it would really be better practice to give these more descriptive names like white_back because in the future I’ll probably be looking at this page and wondering what the hell ‘w’ was supposed to do. Of course, I can look in the style sheet, but it still is better to name things something descriptive.

You can see that the input field for the second digit of the answer is hidden, as is the button for getting another problem.

The forms have an ANSWER button because we found that students in this age group (9- 12 years) often type something by accident or as their first impulse. This forces them to think, at least for a second, whether or not they really meant that and gives them a chance to change  their mind. We added this at the request of several teachers after our first year of beta testing.

The table width is set at 40% and since the container width is defined, the table will always be the same size.
The q class (again, should be renamed and shame on me), has a border at the bottom of the cell. That is used to give the top part of the division problem and used again when each digit of the quotient is found and multiplied by the divisor. The product is then put in a cell with a line underneath.

The first input field is where the first digit of the quotient will be entered. Onclick this will be hidden and the correct answer shown in the element yans1. If the student had entered an incorrect answer, they’ll also get a message telling him or her it is an incorrect answer. All of this is handled by the javascript.

For the remaining rows of the table – the left cell is underneath the divisor, so it will remain empty. The right cell will have each step in the division problem entered, as the student enters the first digit and then the second.  Again, this is handled by the javascript, all I need to do is make sure the id values for each cell match what is in the script.

Once the problem is finished, the div with the id fin will be shown, as will the button for trying another problem. The student now can select one of three choices:

Get another problem (button3), go back and select another option for studying division, or take a quiz to go back to the game. Five correct answers and he or she can go back to playing Spirit Lake.

<!DOCTYPE html>

<html>
<head>
<title>Practice division</title>

</head>

<body >
<div id=”container”>
<div class=”w”>

<h3>PRACTICE LONG DIVISION</h3>
<p></p>
<h3 id=”hd1″> Enter the FIRST digit in the answer</h3>
<h3 id=”hd2″ class=”hidden”> Enter the SECOND digit in the answer</h3>
<input type=”button” class =”hidden” value=”ANOTHER PROBLEM” size=”5″ name=”button3″ id=”button3″ onclick=”window.location.reload()”>
<p></p>

<p></p>
<form name=”formx” id=”formx” >

<input type=”button” value=”ANSWER” name=”button1″ id=”button1″ size=”5″ onClick=”checkProb(1)”>
<input type=”button” value=”ANSWER” size=”5″ name=”button2″ id=”button2″ class=”hidden” onClick=”checkProb(2)”>
<table width=”40%” border=”0″ cellpadding=”0″ >
<tr>
<td width=”20%” >&nbsp;</td>
<td width=”20%” class=”q” ><input type=”text” name=”ans1″ id=”ans1″ size=”3″><scan id=”yans1″ class=”hidden”></scan>
<input type=”text” name=”ans2″ id=”ans2″ size=”3″ class=”hidden”><scan id=”yans2″ class=”hidden”></scan></td>
</tr>
<tr >
<td ></td>

<td></td>
</tr>
<tr>
<td id=”c” ></td>

<td id= “divide”>&nbsp;</td>
</tr>
<tr>
<td ></td>

<td id= “d”class=”d” >&nbsp;</td>
</tr>
<tr>
<td ></td>

<td id= “e” >&nbsp;</td>
</tr>
<tr>
<td ></td>

<td id= “f” class=”d”>&nbsp;</td>
</tr>
</table>

</form>
<div id=”fin” class=”hidden”>

<p></p>
<a href=”../learndividelong.html”><img src=”../scenephotos/arrowhead_point_left.gif” width=”130″ height=”70″ alt=”back arrow” />
Go back to study more</a>
<img src=”../scenephotos/smalls/handblue.jpg” alt=”blue hand” /> <img src=”../scenephotos/smalls/handyellow.jpg” alt=”yellow hand” />
<a href=”../quizzes/dividelongerquiz.html”>Take a quiz to go back to the game<img src=”../scenephotos/arrowhead_point_right.gif” alt=”next arrow” /></a></div></td>
<p></p>
</div>
</div>
</body>
</html>

 

Yesterday I posted the code to get the problem, now here is where we check it. As I said yesterday, may way of programming is to knock out something that works and then go back and make it work better, like a first draft for a journal article. So, this is my first draft.

Keep in mind the point here is NOT a quiz but for them to review and see how long division works. So, if they get the wrong answer, they get an alert message that this is the wrong answer, but then the correct answer is shown. This happens for both the first and second digit of the quotient.  There are two digits in the quotient in these problems. We are trying to show students that when you do long division, you find the first digit, multiply that by the divisor, write the product below the dividend and subtract. Then, you do the same thing again for the next digit.

Also, I showed using alert here, but we actually use a function we wrote in our game because there are problems with using multiple alert boxes in the same page with Unity. The alert is included here for generalizability. This post is the second half of the javascript. You also need a bit of css and html that I’ll put up next.

You can see the final product here.

This is one of hundreds of applets we have written that are just auxiliary to the main game. You get sent here to study if you miss one of the math challenges in Spirit Lake: The Game.

Here is what this code does in order …

When they type in an answer, it is one digit at a time. The function checkProb, if it is the 1st digit,  hides the input box and answer button for the first digit and shows the correct answer. It also shows the input box and button to answer the second digit. The correct first digit is shown.

The product of the divisor and that first digit is computed, set to a value for a new variable d1, and that is shown.

The result is subtract from the dividend, and that result, e1 is shown but with a space included so the digits are lined up correctly.

If their answer is wrong, a message is shown telling them it is wrong and what the correct answer is. Actually, that message comes up first so once they click OK they can see the correct answer, product, etc.

Then, they enter the second digit and all of the steps execute again. After they have done a complete problem, the instructions on how to complete the problem are hidden and two new options are shown, to either get a new problem or go back to the game.

function checkProb(num){
this.num = num ;
if (this.num == 1)
{
var theirs = document.formx.ans1.value ;

$(“#ans1″).hide() ;
$(“#hd1″).hide() ;
$(“#button1″).hide() ;
$(“#hd2″).show() ;
$(“#ans2″).show() ;
$(“#button2″).show() ;
document.getElementById(“yans1″).innerHTML = rightans1 ;
$(“#yans1″).show() ;
var d1 = rightans1*divisor *10 ;
var e1 = dividend – d1 ;
document.getElementById(“d”).innerHTML = d1 ;
$(“#d”).show() ;
document.getElementById(“e”).innerHTML = ‘&nbsp;’ +e1 ;
if (theirs != rightans1){
alert(“Sorry,the correct answer is ” + rightans1) ;
}

}
else if (this.num ==2)
{
var theirs = document.formx.ans2.value ;
var d2 = rightans2*divisor ;
document.getElementById(“yans2″).innerHTML = rightans2 ;
document.getElementById(“f”).innerHTML = d2 ;
$(“#f”).show() ;
$(“#ans2″).hide() ;
$(“#yans2″).show() ;
$(“#fin2″).hide() ;
$(“#fin”).show() ;
$(“h3″).hide() ;
$(“#button3″).show() ;
$(“#button2″).hide() ;
if (theirs != rightans2){
alert(“Sorry,the correct answer is ” + rightans2) ;
}

}
}
</script>

You’d think it would be easy to find source code for a simple applet to demonstrate long division with a one-digit divisor and two-digit quotient. I wanted it to show the steps in long division, with the product for the first digit in the quotient shown, then that subtracted from the dividend, and the next digit in the quotient shown.

I had one in Flash I wanted to replace and I found all kinds of applets but none with the source code, so, here, as a public service, is what I did this evening while drinking beer.

You can see the end product here 

In case you are dying to know, here is how I write a program, regardless of the language:

  1. Get something to work
  2. Clean it up to make it better

To me, trying to get your code perfect on the first try is like expecting your first draft of an article to be perfect. I find it much easier to dash something off and then go back and rewrite. I know not everyone does it that way but it works for me.

I’m going to use jquery so let’s start with that

<script type=”text/javascript” src=”../javascript/jquery-1.11.0.min.js”></script>
<script type=”text/javascript” src=”../javascript/jquery-ui.min.js”></script>

<script type=”text/javascript”>

<! — First you need a random number function –>

function randnum(min,max)   {
var num=Math.round(Math.random()*(max-min))+min;
return num;
}

// Set up to get a new problem when the window loads. Create variables ;

window.onload = getProb ;
var rightans1 ;
var rightans2 ;
var dividend ;
var quotient ;
var divisor ;

 

function getProb()
{

quotient=randnum(10,100);
divisor=randnum(1,9) ;
document.getElementById(“ans1″).value = “” ;
document.getElementById(“ans2″).value = “” ;
document.getElementById(“yans1″).innerHTML = “” ;
document.getElementById(“yans2″).innerHTML = “” ;
dividend = quotient *divisor ;
divisor=dividend / quotient;
var w = quotient + “” ;
rightans1 = w.substring(0,1) ;
rightans2 = w.substr(1,1) ;
document.getElementById(“c”).innerHTML = divisor ;
document.getElementById(“divide”).innerHTML = dividend ;

}

Function above creates a problem with a quotient between 10 and 100 ;

The divisor will be between 1 and 9  ;

The quotient will be between 10 and 100 ;

I set the values in the form all to empty so that when a student reloads the page and gets a new problem the answer from the previous problem is not still there.

I made the dividend equal the quotient times the divisor to make sure that the divisor went into the dividend evenly with no remainder. I made a  local string variable, w, and then created two variables that were the first and second characters of that variable.

The final two lines in this statement write the problem to the page.

 

— Checking the problem is step 2.  Since I don’t like horrendously long blog posts, I’ll put that up tomorrow.

Getting ready to teach a data mining course at the end of the year, I started looking through data sets I have on my desktop. Not sure what I will end up using. My first lesson, no matter what, is going to be on data quality.

The very first thing I did was a series of PROC FREQs. Then, I thought maybe that was a mistake. Perhaps I ought to start off with SAS Enterprise Guide or Enterprise Miner.  Here is how I did the first peek at data quality with Base SAS. I’m going to do the same thing with Enterprise Guide tomorrow and see if it would be easier. After that, I’ll try Enterprise Miner. I know I downloaded the SAS On Demand version a while back and haven’t done much with it lately.

(There is a new SAS for the Web offering but from what I have seen (admittedly, a while back), it requires you to set up a virtual machine with VMware and I did not have the time to do it nor could I find my Windows 8 or Windows 7 install disk. Must clean office.)

The first thing I did was pull out a data set with a couple of thousand student quiz records. Yes, I know in data mining we will get to data sets in the millions but this is the first exercise of the first class.

I did not expect to have 2,000 quiz records because we only have around 1,200 beta testers and about 200 of those are teachers who I would expect would get all of the in-game problems correct so never be routed to a quiz. I also know from observation that some of the students never made it to the part of the game where they could do the quizzes. The first challenge page requires students to be able to read simple words and subtract two-digit numbers.

I did a super-simple PROC

proc freq data = in.realquiz ;
tables username*quiztype ;

and found that a couple of the users had supposedly taken the same quiz 40 or more times. One students showed having taken the quiz 70 times and another 91 times. While that is theoretically possible, I was suspicious because after those three, the highest number was 7.

I went into the data set and looked at those particular records and the time stamp showed them coming in tenths of a second apart. Clearly, the student was not answering 5-7 questions in less than a tenth of a second.

We tracked these down to a particular school that was having issues with the firewall. It appeared that when the program couldn’t connect to the server, it tried again and again. When there was a connection, all of those records went through at once.

LESSONS LEARNED

  1. Always look at the outliers. Don’t just toss them out. They can tell you things. In this case, taking a closer look at that PHP code is on my list of program fixes. If it happens at one school it can happen at others.
  2. Time stamps are your friend.  I try to include them whenever I can. Yes, it might take up a bit of time and space but there is nothing like it for detecting duplicate records – and fraud.
  3. Just because data has supposedly been cleaned up, never, never assume that it is problem-free.

 

At the moment, we are interested in knowing the most common failure point in the games. Do we need to add in more teaching and problems earlier? The games are designed to teach and test students in mathematics at the fourth and fifth grade levels. The teachers we work with often tell us that their average student is below grade level. So here was my next series of steps.

proc sort data = in.realquiz ;
by username quiztype ;

data test ;
set in.realquiz ;
by username quiztype ;
if first.quiztype ;

*** These first steps sort the dataset by username and type of quiz and then only retain the first instance of each. So, if a student actually did take the same quiz seven times, I am only interested in the fact that beginning the game, he or she could not do multiples of 3, not that it took seven tries to get there.

proc freq data = test ;
tables quiztype*pass / out=quizfreq ;
run;

*** This step shows both the quizzes students took and the result.

This is the point at which I began to become concerned, not about data quality but by what the data was beginning to reveal.

Table of quizzes by passing

Visually impaired – click here for HTML files of tables instead of png

Over half of the students failed and the quizzes they were failing seemed to be at the lower levels – around third-grade math.

LESSON LEARNED

You can get some very valuable information from some very simple statistics. A lot more about that, tomorrow, though, since I have to get back to work ….

I can imagine the type of person served by an expensive, intensive programming bootcamp – someone with money (or, at least, good credit) and several weeks of free time. That has never described me in my life. The last time I had six weeks free was in the summer after tenth grade, before I started working full-time and at age 14, I had neither money nor credit.

It doesn’t require a pile of money and uninterrupted summer’s worth of time to keep up or catch up on technology. If you fall behind, you have no one to blame but yourself.

My whole life, I have been interested in learning more about everything. (Well, except about literary and film criticism because, well, it sucks. Just try reading any of it and you’ll see I am right.) That’s included lots of graduate coursework. For years, I took one class a year – in something – microbiology, matrix algebra – just to learn something new. Now, I try to teach a course a year. Last year it was biostatistics. This year,  I think I will teach both biostatistics and data mining.  I always learn something new when trying to come up with good examples and activities for classes, I have to keep up on the latest software and operating systems. It isn’t just free, but they pay me – not a lot, which is why I only teach once or twice a year.

Someone recently tweeted,

I hope to never learn the meaning of the word “webinar”.

Webinars aren’t all bad (just most of them!). However, I was on one this morning Yakov Fain did on building HTML5 applications, hosted by O’Reilly Media that was definitely worth an hour of my time. It was free, by the way. Now, I’m sure it was just a way for them to sell books – which worked, since I bought one – but it is also a way to get a lecture by experts on a topic. I probably get 80 invitations to webinars for each one I attend. It’s not the most exciting format so I don’t recommend signing up unless it’s a subject you are really interested in learning.

Virtual conference – this is  a first for me to attend, so I will tell you how it works out. I signed up for one on health analytics sponsored by SAS. It looked interesting and it was free. There was a virtual conference I was interested in a while back, on javascript, but it was several hundred dollars for what appeared to me the equivalent of watching youtube videos. Maybe I missed out on something amazing. I’ll never know.

jennifer

Youtube – You are mistaken if you’ve only thought of it as videos for cute kittens and teenage rock star wanna-bes. (Are they actually called rock stars any more? Are they all rap star wanna-bes?) I actually watch youtube videos on jQuery and javascript on the TV while riding the exercise bike. This habit has caused The Perfect Jennifer to wonder aloud more than once
Exactly how is that you people don’t die of boredom around here?
Sadly, the public library hasn’t been a very good resource for me for programming resources. The books they have tend to be far out of date. It makes me sad because I love libraries and have cards for both the Santa Monica and Los Angeles libraries as well as a couple of university ones.

If your university or company offers you an account on the Safari library, I would jump on that because you get unlimited access to all of their books, videos and courses. The individual price for $43 a month seems a little much to me. If I didn’t have a free license, I’d just buy the ebooks I needed. We already have a LOT of technical books, though. If you don’t, maybe it’s worth it.

Just for questions, answers and randomly poking around stackoverflow.com is awesome.

It reminds me of when I was first learning SAS over 25 years ago. I was on the SAS-L mailing list and would just read every day what the really smart people were talking about.

I have to get back to work but there are lots more resources out there, both that I didn’t have time to list and others that I’m sure I don’t know about. Have a favorite?  Please share in the comments. I’m always looking for new places to learn cool stuff.

snow

 

Since there is a blizzard and I’m inside analyzing data, I thought it was a good time for another random SAS tip.

The default output type for SAS 9.4 is HTML, which is nice for presentation and sharing, but sometimes I would like a plain text output, especially if I’m going to be doing something like copying and pasting the output into my program.

You can easily toggle between plain text and HTML output by doing this

Go to the TOOLS menu, select OPTIONS and then PREFERENCES.

In the Preferences window, click on the RESULTS tab.

window with listing preference

 

Check the box next to Create listing and uncheck the box next to Create HTML . Click OK.

Now your results will be in plain text output instead of HTML. To switch back, just go through the same steps, uncheck the listing box and check the HTML box.

Speaking of copying your output into your program ….

I wrote previously about a problem where  I needed to do, among other things, a PROC CONTENTS with the variables in the order they occur in the dataset, not alphabetical order (use varnum) .

 

If you just copy and paste the output of a typical CONTENTS procedure and all you want is the variable names,  you will have a lot of stuff about data type, length and label that you need to delete. Also, my life experience has been that the keystrokes you make (including the backspace key), the more likely you are to make mistakes. When it comes to the keyboard, less is more.

What I really want is the variables output in the order they appear and nothing more. The SHORT option does this for me.

proc contents data= in.sl_pretest varnum short;

and produces this output

Which_choice_is_the_same_as_the What_is_five_time_six__ Fred_walked_for_one_hour__How_ma How_is_nine_thousand__thirty_sev There_are_124_students_making_th
Valerie_has_225_pennies__She_div Joe_did_this_division_problem Which_sign_goes_in_the_space_to

What possible good is that mess? Well, I copy and paste it under the RENAME keyword and then hit the spacebar between each variable name and type = q1   , or whatever number,  like so

RENAME

Which_choice_is_the_same_as_the = q1

What_is_five_time_six__  = q2

Fred_walked_for_one_hour__How_ma = q3

 

As I mentioned in the previous post, I could not do this using an array statement because the data were of mixed type, character and numeric, and SAS does not accept data of mixed types. I also mentioned how to get around that so if you are interested, go back and read that post.

This week I had one of those pain-in-the-ass problems. I had a test with 24 items but they were of mixed types. That is, for some the answer was multiple choice and for others it was numbers.

The data was received as an Excel file.

Now, I could have opened it with SAS Enterprise Guide and specified data types for each variable, but the problem is, I am going to get this particular data set over and over, so I want code I can write once and run every time.

As if that wasn’t bad enough, the variables all had names like:

which_choice_is_the_same_as_the_

I wanted to rename these all to something sane like q1, q2 etc.

The first step was an option I don’t think I’ve ever used before, oddly enough.

 

proc contents data= annoying  varnum ;

 

Normally, SAS gives you the variables in a data set in alphabetic order when you do a PROC CONTENTS. The varnum option lists the variables in the order they appear on the data set. This was immensely helpful because it spared me going through the data trying to figure which was the first question, which was the second, and so on.

I just copied the variables in order after a RENAME statement and tacked on an =q1, q2, etc. like so

Data better ;

set annoying ;

rename

which_choice_is_the_same_as_the_ = q1
what_is_five_time_six__ = q2

etc.

proc contents data= better ;

I could have combined this with the previous step , but the fact is that unless the data set is really gigantic, the time that needs to be preserved is not computer processing time but my time, and this way was quicker because I didn’t have to write out those ridiculous variable names and worry about the program failing because I used  _ in the name instead of __ .

SAS does have a function to detect variable type, but that wouldn’t really have helped me because I still need to write all of these variables into a single array of item1 – item24 for later use, and you cannot have mixed type arrays. So, I did this

 

data mo_better ;
set better ;
array qs{*} q2 – q6 q10 q12 q14 -q16 q19- q21 ;
array itemN {*} $12 item2 – item6 item10 item12 item14 – item16 item19 – item21 ;
array qsA {*} $12 q1 q7-q9 q11 q13 q17 q18 q22- q24 ;
array itemA {*} $12 item1 item7 – item9 item11 item13 item17 item18 item22- item24 ;
do i = 1 to dim(qs) ;
 itemN{i} = put(qs{i},12.) ;
end ;
do j = 1 to dim(qsa) ;
 itemA{j} = qsa{j} ;
end ;
drop i j q1 – q24 ;

I have 4 arrays. The first consists of the numeric variable type questions. I couldn’t use _numeric_ to create an array of all numeric variables because there were others in the data set that were NOT test questions but were numeric and I did not want them in my array. I had to actually list each variable individually or in a range like q14-q16.

The next array is the one I am going to recode the variables into as character variables. Notice that character arrays need a $ and a length. The next two arrays are the character variables and the variables I’m going to copy them into. I could have just renamed the character variables in a RENAME statement and then changed the length in an ATTRIB statement  but  it would have taken more typing.

The DIM function is the dimension of the array, so it is going to loop through from 1 to however many variables in the array because I didn’t feel like counting them.

The PUT function is going to put this numeric variable into a new character variable with the specified length. It changes the variable to character.

The next loop just puts all the character variables into other character variables with the names item1, item7, etc. Now I have variables that are all the same type and length, named item1 – item24 and I can do things with them like compare each student’s response to each variable to the answer key, score it right or wrong and sum up the scored items, like this (1ANSWER is the first username)

 

Data in.pre_scored ;
set mo_better ;
by username ;
Array scored {24} sc1 – sc24 ;
Array items {24} $12 item1- item24 ;
Array ans{24} $12 ans1 – ans24 ;
if _n_ = 1 then do i = 1 to 24 ;
ans{i} = items{i} ;
end ;
else do i = 1 to 24 ;
if ans{i} = items{i} then scored{i} = 1 ;
else scored{i} = 0 ;
end ;
Retain ans1 – ans24 ;
total = sum(of sc1-sc24) ;

Since this is part of a two-year grant and I am going to receive these same test data sets many times, I am now finished with reading in and scoring the data for the next two years. After this, I just need to import the excel file and click run. I am happy.

Also curious, because I noted that this year’s pre-test scores are 1.5 standard deviations higher than the previous year.  I suspect this is because we have many more fifth-graders in this sample. So … with the scoring done automatically, I can now go on to interesting stuff.

If you want to check out the game these results came from, you can read about it here

 

I read a blog post where the author said the women who dropped out of programming “should have been discouraged” because it’s not for everyone and many women try to use smiles and flattery to get men to do their work for them.

I actually have had the experience the author cites, but with both men and women. It’s true there are some people in the tech field who are very introverted or socially inept. They are willing to help you with your technical problems if you will just stop by and have a cup of coffee and chat with them.

I’m not that person. I have a husband and four daughters. Interestingly,The Invisible Developer, who is so introverted as to be never seen in public is also not that person. He has me and aforementioned four daughters. That is enough for him.

Clearly, people who want you to do their work for them are annoying, however, I haven’t found them to be limited to one gender at all.  Lately, I’ve been wondering whether they are like that in SOME  cases because they don’t believe they can learn to do it themselves. I don’t know the answer to that.

What I do know, though, is that over the years I have known many people to succeed in areas I would not have given them a chance. Two very fine physicians that I know didn’t attend the best high schools, have the grades as undergraduates and honestly,  I didn’t think they had a prayer of  getting into medical school, much less succeeding. Neither got accepted in medical school the first year that they tried. People I would not have given the chance of a prayer in hell of becoming elite athletes have often gone on to surprise me, including a couple who won Olympic medals.

Life discourages people enough. Don’t add to it!

That advice is particularly true for programming. The last couple of days have been discouraging. We had our next install almost ready and then I found some bugs in it. Then we thought it was done, and I found some more bugs in it.

Yes, he's a man and we work together to fix stuff

The Invisible Developer is upstairs fixing those and testing the latest version. I am downstairs fixing his code on the next game (so much for women wanting men to do their work for them, and he is definitely a man. I can point to fact of having collaborated in producing The Spoiled One as irrefutable proof of said manliness. Photograph attached.). Actually, he’s brilliant and totally capable of fixing it himself, but he was already working on the other game.

Everyone’s code, if it is the least bit complicated, is going to have bugs in it. Sometimes it can take you days to find them.

Some days we succeed in writing quizzes where students can drag and drop answers, video clips with sound and animation play in response to correct answers with dialogue in English and Dakota, and then the student is transported back to a 3-D virtual world to continuing playing.

Other days, nothing happens. Just nothing. There are no errors in our consoles, just a screen looking obstinately back at us refusing to do what it’s supposed to do.

Programming is discouraging some days on its own and the LAST thing you need those days is someone saying,

“Maybe you’re just not cut out for this. “

I was complaining about how today had just not been productive, that I wanted to have the latest fixes on Spirit Lake in the hands of the teachers today but it wasn’t saving the game state frequently enough. While The Invisible Developer worked on that I found that some of the quizzes in the next build of Fish Lake were telling the student the answer was wrong even when it was right.

The Spoiled One said,

“Don’t worry, Mom. You’ll figure it out. You have time. Life is long.”

You know what? She was right. We figured it all out today. People should be encouraged. I’m proud of that she has figured this out at not-quite-sixteen.

← Previous PageNext Page →