Oct

31

Someone sent me a video on self-defense a while back showing the various ways people respond when threatened.  It noted that we automatically think of ‘weapon-like’ objects for self-defense. We may not actually have a knife or club but we grab a letter-opener, baseball bat or broomstick. The sender noted that other objects can be equally effective – whacking someone on the head with your laptop could make a substantial impact – although probably not good if data conservation is high on your priority list. My favorite video seemed to be a villager in eastern Europe with his small child. When threatened by an intruder, the father picked up a goat (yes, a goat), swung it by its feet and beat off the intruder.

I am really not interested in whether or not this is an urban legend, the aerodynamics or physics of impact of goat horns to the head going at a given velocity, nor am I advocating goats as the new weapon of choice, so if you want to comment on any of those things or you are a member of PETA, don’t contact me.

My point is that we often limit ourselves to familiar ways of solving a problem when better ones are available. A prime example of this is people who all took the same classes on database design and learned about the infamous star schema, and the importance of having whole bunches of tables to expedite the speed of access. They don’t seem to have gotten the memo that their education is only relevant if one of the following is true:

  1. You work for Amazon, the Social Security Administration, Google or other site that handles hundreds of millions of records.
  2. You are living in the previous century.
  3. You live in the country where technology is approximately equivalent to that of the family in the goat-swinging video.

If none of the above are true, you are wasting everyone’s time and are an annoyance to me personally. One of the reasons that we did all of this with the tables and the look-up and the binders saying what was where is because it saved us oodles of computer time, at one point – that point was back in the 1990s, which in technology time is about equivalent to back when our ancestors where dancing with Australopithecus if you happen not to believe in evolution (which I am guessing by looking at your code).

For the umpteenth (def: umpteen – a number that comes before umpteen and one) time, I am working with a database that has about twenty different tables and around 50,000 records. There is not one unique id that fits all of these tables. This is the standard sort of thing where you have a customer id, product id, sales id and you merge them altogether, do table look-ups and so on to get, for example, the total purchases and dollar value of a given customer. Do you know what I did? I merged all twenty tables together to be one SAS dataset. The whole thing took about 85 MB, which can fit on even the cheapest flash drive. Rather than having to document what is in each of the twenty tables and where to look, I can just do a proc contents. I don’t have to spend any time worrying about which table or query has the data I need. It is all in one spot. This dataset with the 50,000 records and less than a hundred variables is not the dumbest example of  this sort of thing I have seen. More often than I care to recount I have seen this key table + student records table + student profile table etc. ad nauseum sort of design for datasets with 500 people or less.

Okay, there are some REALLY good programmers I know who are self-taught. However, there are also some people who learned one thing, read a book on Access, took a six-month course, whatever and for the rest of their lives pound every problem into the same round hole whether it makes any sense or not.

Listen, if you are managing the data for a small project or company, think about doing it with SAS. Just enter the damn data into one file. You can do your reports, graphs, statistics – and documentation is easy because it is all in one spot. You could also use SPSS. It even comes with a data entry tool (costs extra over the base package).

gargoyleMany, many years ago I remember doing data entry screens in SAS, if you need that sort of thing. It was either with FSP or SAS AF, both of which have kind of faded in my memory due to disuse, just like that course I took in college on Medieval Economic History to fill some graduation requirement.  Really, if you only have a few hundred records to enter, it’s probably not even worth it. Just open a SAS or SPSS dataset, create some variables and enter the @#% data.  The multiple tables with key fields solution is equivalent to bringing in sword-wielding mummies back from the dead to defend your family when you could have accomplished the same task just by swinging a goat.

Happy Halloween.

Oct

11

So you want to install SAS for Linux….

Step 1: Read the documentation for some hints on how to download and install the download  manager, and  the SAS software depot (if you are the SAS administrator).

Step 2 : Throw away the documentation. It will just confuse you after this point and it doesn’t help anyway.

I had installed SAS on a Ubuntu operating system once before, on a virtual machine running on Virtual Box running on an older computer in my office that has Windows XP. I couldn’t remember how I did it except that I ended up literally throwing away the documentation and just using Google to figure it out.  Then I deleted the virtual machine later when I needed space on that computer.

This weekend I installed SAS on the computer in the living room that runs Linux. My husband received the computer for $100 in some raffle or something. It was a bit of a dilemma at first, because our other six computers are Macs and Dennis equates Windows with the anti-Christ, but, after a few minutes thought, he installed Ubuntu and all has been well, barring a few arm-wrestling matches with the little one over whether she was going to get to use it for Club Penguin or he was going to be able to use it for programming.

The  good news is downloading the download manager and installing the SAS software depot was a piece of cake. I followed the instructions on the SAS install site and everything worked perfectly. It took a few hours to download, which was a bit surprising to me, since we just had FiOS installed on the promise of speeding things up, but for once no one tripped over a cord and disconnected the computer, spelled coke on the keyboard or any of those other things that always seem to happen when you are right in the middle of something.

I created the DVDs from the SAS software depot but they did not work. By “they did not work”, I mean that when I popped in the DVD and ran sudo /cdrom/setup.sh   instead of giving me three options, one of which was to install the software, it launched right into creating a new software depot on the computer where I already had a software depot from which I had created the DVDs in the first place. Sort of an annoying Escher-type loop.  FAIL !

If I was more of my usual self I would have been determined to figure this out, but, unfortunately, I only have about 10 or 11 productive hours when I am not in too much pain to work (this sucks) so I decided to go ahead and use the software depot which was installed and actually DID give me three options. So, here is what I did after having downloaded the software depot, and it did work.

LOG IN WITH ROOT ACCESS

1. cd to to the directory where your software depot is located, e.g.,

cd /usr/local/SAS/

2. Make sure your default shell is bash

(If your default shell is NOT bash, in my case it was dash, you will get a message that says:

/bin/sh: illegal option -p

Since the pain medication is eating about 40% of my brain as well as a good bit of my productive hours (this sucks), I passed the problem along to the Windows-hating Dennis when he got home from the lab. He emailed me the following from the upstairs bedroom  ….

In order to install SAS components without the “/bin/sh: illegal option -p” error which stops installation,  your /bin/sh file must be pointing to bash.

To check this, in the terminal execute the command:

ls -l /bin/sh

The line you get in return should end in something like:

sh -> /bin/bash

if it just ends in sh, or something like sh -> /bin/dash, you need to replace it with a symbolic link to bash. To do this, execute the two following lines:

sudo rm  /bin/sh
sudo ln  -s  /bin/bash  /bin/sh

3. Type  sudo ./setup.sh

give your password

Just click along with the next screens and SAS should install no problem.

(The SAS documentation says to just type ./setup.sh  but in my case that tells me I don’t have permission.)

Actually compared to the problems we had early on with SAS 9.2 for Windows, this was relatively pain free.

Running SAS

You need to create a work directory and assign it when you run SAS. I went to my home directory and created a directory named tmp.

mkdir tmp

Then, to invoke SAS, I did this

usr/local/SAS/SASFoundation/SAS92/9.2/sas -work tmp

and SAS started up just happy as could be.

Of course, at this point, three things occur to me.

  1. I had no data on this computer and no actual need for SAS on it at the moment, I just wanted to see if I could install it and get it to run.
  2. There must be something like a .login file where I can set the path and don’t need to specify the full path each time.
  3. I have yet to install SAS on Linux from DVDs. The last time I did it from an external disk drive that had the software depot on it.

Also, there was no Enterprise Guide and no Power and Sample Size application. Not sure if SAS does not make those for Linux or they just weren’t part of our license.

So, there you have it. SAS for Linux. Except for the little bash/ dash episode, it really wasn’t much trouble at all. We only have one person outside our department who wants to install it (the other three used the same external drive that I did), so I think I will just copy the software depot on to a flash drive and send it to him.

Maybe I’ll figure out the DVD thing when I feel better (this sucks) or much more likely I’ll give them to my wonderful assistant and have her figure it out while I write up my lectures for next week on calculating various effect sizes with SPSS, SAS Enterprise Guide for Data Management & Statistics and Basic Statistics using SPSS 14 (yes, unbelievably, we actually had a request for a class on that!)

Oct

10

Not being able to move much, I have spent a lot more time lately in the company of my 11-year-old daughter, and that is not a bad thing. She started Algebra this year and she commented to me,

“You know, when I read this stuff, like Y = 3x +5   and what is Y if x is 2, I get it now, but I know if I went back to myself as a kindergartener, or if I said it to a kindergartener now, they wouldn’t understand it.reading_julia Did you know that math is like a language? I feel like I have learned to speak Alien language or something.”

I stopped what I was doing, which was installing the SAS software depot on the Linux computer in the living room, and said,

“You’re right and it gets better and better from here on out. You got done with the boring part of math where you have to memorize your times tables and stuff and now it is the fun part. There’s lots of other languages you can learn and every time you learn one the next one gets easier.  Some day, you’ll be able to do things and say things in that language you can’t even imagine now.”

So, I feel like she made a major breakthough this week, all on her own. At some point, we will have to have the opposite talk, which is that no matter how good you are at math there is going to come a point where you just don’t get it and you have to struggle through. For years now, when I have looked back on having won the world judo championships in my twenties, it has struck me odd in retrospect that at the time it all seemed perfectly natural, being best in the world. Now, even older, when I look back at graduating from college at 19 or  my graduate courses in statistics and realize that two of my professors gave me an A+ in courses when the grading scale only went up to a 4.0 for an A,  that is all quite odd, too. One of them explained that I had not only had the highest grade, but was a standard deviation above the next student, so he felt he had to do something. At the time, it seemed like just what I did, though and I did not think of myself as particularly good at math or statistics, it was just something I happened to like and the university offered graduate courses in it, I lived a few miles away and had three babies, so it was something to do. In retrospect, I run into my old classmates and they all still remember me as “Oh, you were the statistics BRAIN!”

AND YET, there have been so many times when I read something not just once but two or three times before I got it.

There are times to this day when I attend a presentation on say, multiple imputation, and I have a general idea, but I don’t feel like I completely understand it, so I read an article on it, and then another, and it gels. There are other times when I feel completely at sea. The first time I read an article on LISREL (probably the first program that came out on structural equation modeling) and I understood about 10% of it. So, I got Leslie Hayduk’s book on structural equation modeling and read the whole thing and still didn’t understand half of it. I took a course from Keith Widaman, at UCR and then I felt like I understood a good bit, which I am now trying to remember as I get more SEM questions these days.

My point, and I do have one, is that no matter how good you are in math, there are always those bumps in the road. You just have to barrel through. To read the book over and over, read a different book, take a class, be determined to understand it, and you will.  You need to learn not to give up. Then, when you move from matrix algebra to Linux, you learn to keep trying, if

./Setup_Linux

doesn’t work, try

sudo ./Setup_Linux

Eventually, you’ll find yourself talking to the aliens as if you’ve known them all your life.

Blogroll

WP Themes