# Two Useful Resources for Teaching University Statistics Courses

June 16, 2013 | 1 Comment

The Mplus website has free videos on using Mplus. The introductory course (not very introductory, really, because the the main topics are advanced factor analysis and structural equation modeling) has some pretty good resources. While it is not professional quality – just a video of a class  – and much of it is beyond the first or second statistics course most students take, about 22 minutes into the first course it has  a very nice discussion of regression and an example with a dummy variable. Think I’ll use that part for my next class.

There is also a good introductory discussion of path analysis in the same video at about 27 minutes. The reason I’m giving the points in the video is that it is VERY long and much of it is kind of irrelevant – why Mplus came about – or too advanced for introductory graduate courses.

The Inter-university Consortium for Political and Social Research has a wealth of resources. The one I have used the most in the past is data that you can download. However, you can also select a study and perform simple cross-tabulations. For example, I used the 500 Family Study to create a table of the frequency with which a respondent attended religious services with a child by the family’s religion, controlling for relationship of the respondent. I was assuming mothers would attend services with their children more often than fathers. I had no idea whether there were differences among religions – well, it turns out there are pretty dramatic differences.

You can use ICPSR not only to statistically answer questions, but also to look at the statistical methods themselves. For example, I did a cross-tabulation of age and sex using the Kaiser-Permanente study of the oldest old.  Each analysis produces both a table and a graph and here is the graph this one produced.

Now, we know that women live longer than men, but in this study we see there isn’t a real trend of males going down as a percentage as the age moves from 65 years on the left to 94 years on the right. In fact, 100% of the subjects over 92 years old were male. What does this tell you? Well, hopefully, after some discussion it will tell your students that this was not a random sample. In fact, it looks more like they deliberately sampled to get close to equal numbers of males and females, possibly within an age range, such as 80 to 90. It also suggests that there probably weren’t too many people over 92 years old in the sample. That finding that 100% of this age group is male was probably based on  three or four people. Having generated some hypotheses, you can go back to the ICPSR site and test them, requesting percentages for the row and total lets you know that 0.1% of the sample was over 92 years of age. Speaking of hypotheses, you can also got chi-square values, z-values and confidence intervals for your various research questions.

I like this site because unlike a lot of others that have pre-packaged and usually boring questions, “Will males choose red M& M’s more often than females?”, ICPSR has hundreds of studies with hundreds of thousands of variables, allowing students to pose questions that really interest them and find an answer.

# First World Problems and What to Do about Them

June 12, 2013 | 1 Comment

I’m sitting by the pool at my sister’s house typing this on my iPad keyboard. I’m using the iPad for three reasons.

1. My laptop, which I seldom run in boot camp, is now installing umpteen updates before it re-starts in Mac OS
2. I’m supposed to not be typing so much because a combination of working 80 hours a week and doing judo for 42 years has not done good things for my elbow. Rather than following the doctor’s advice to take a couple of days off each week, I’m just switching between devices and desks hoping that makes it less repetitive motion, and swimming instead of doing judo.
3. My eyesight, even with glasses on top of nuclear-strength prescription contacts still isn’t good enough to make working on an iphone feasible.

This all reminded me that the only problems I have are first world problems. My laptop is slower than the desktop I have at home. Let’s see, I have a laptop, iPad and iPhone. If your biggest problem today is that the \$3,000 worth of electronics you carried with you is inferior to the electronics you left at home, you, my dear, have first world problems.

My great-grandfather was a farmer. He built his home himself, dug it into the ground, then got something that looked like concrete blocks that he stacked up. When his son got married, the two of them, my great-grandfather and great-uncle, built a house for him the same way. He worked at manual labor his whole life – this isn’ t some story the old people made up. I used to stay at my great-grandfather’s house when I was a kid. People around him had bad backs from carrying water from the well, bales of hay, stones for building or to get them out of the way of the plow (or both). If you’re pouting because the hotel pool is full of children, making it inconvenient for you to swim laps for exercise, you definitely have first world problems.

Think about it. I try to think about this every day. When I get irritated by some little thing – and really, the only irritants in my life are little things – I try to remember how insignificant my problems are and show my gratitude for life in some way.

No, this isn’t one of those new-age, feel good, be grateful every day posts. It’s like Ricky Gervais said on twitter,

With all these people saying to send prayers to Oklahoma after the tornado, I feel so stupid. All I did was send money so they could buy food and clothes.

Here is how I try to show my gratitude – either volunteer for some organization or send money. For those of you who believe charity begins at home, in Los Angeles, we have Didi Hirsch Mental Health Services, Homeboy Industries and Children of the Night .

If you’d like to help out and believe people should help themselves, you started a small business and worked your way up, etc., you can put your money into Kiva.org that provides small loans to people in poverty to start businesses.

In short, the answer to your problems my dear, is that you don’t have any real problems. Just realize that, help someone who has real problems and be grateful that you can.

That is another thing I have learned in (almost) 55 years.

# Being a Manager Means Doing What Needs to be Done

When I started dating my late husband, I was a 26-year-old engineer and he was 44-year-old manager. In addition to forever losing any possibility of criticizing age differences for any boyfriends our daughters might bring home, it also means that he had a view on life that I often did not appreciate until years later.

He had an excellent reputation as a manager, and I asked him,

“What is it that makes you so successful in your job? “

He answered in that slow, country drawl of his,

“Well, darlin’ the key to being a good manager is to see what needs to be done and then do it.”

I argued that was redundant, of course anyone who saw what needed to be done would do it. He just shook his head at my naiveté .

He was right.

Very often these days, I find myself facing tasks that I don’t want to do. I don’t want to criticize someone’s work, stay until 2 a.m. to work on a deadline, tell someone we won’t take a contract to work with them because it does not fit with our strategic plan (yes, unbelievably, we have one). I don’t want to tell anyone that their skills just don’t fit with what we need at the moment, no matter how nice and smart of a person they might be. I don’t want to throw out a design and start over because the first one did not meet our client’s needs.

These are the times I feel like whining,

Can’t someone else do it?

The answer is no. It’s my job to make decisions and follow through.

The fact is that many people see what needs to be done and, paradoxically, they DON’T do it. They put it off, they have meetings about it, avoid seeing the people involved.

When I catch myself thinking I will talk a decision over with other staff members, tackle a problem when I get back from a trip, I stop and ask myself whether I’m really just avoiding doing what I know needs to be done. More often than I’d like to admit, the answer is “Yes”. Once I face up to the facts, I just bite the bullet and have what my late husband used to call a

“come to Jesus meeting. Why, darlin’ that’s when we all sit down and I explain things to them and then they see the light.  And if they don’t, “

he said, taking a long drag on his Marlboro cigarette,

“well, then, I fire their sorry asses.”

So, there is another thing I learned in almost 55 years. When you see what needs to be done, do it.

# Jealousy is Toxic, Honesty is Good and other things I’ve learned in (almost) 55 years

Five more things I have learned in almost 55 years.

1. Jealousy is bad for you.
3. Much negative criticism stems from jealousy.
4. Yes, employers ARE right to turn you down because you are over-qualified.
5. Don’t take it personally but DO take it seriously.

I am on a roll today, no?

By this age, I’ve had the unenviable experience of many vicious comments, spoken behind my back, in email and on-line (although almost never to my face). I’m the first to admit that I live so far from perfect we don’t even have a passing acquaintance. Still, what’s it to you? I’m certainly not in a unique position here.

I’ve seen people go so far as to track the times their colleagues came into the office, when they left, how long they spent for lunch and question everything from their expense accounts to salaries to paid time off.  When I see someone repeatedly running down a co-worker or  supervisor. I wonder why.  The common thread I’ve noticed is jealousy – Suzy has a job, salary or reputation that they don’t.

Take it from me, running to upper management or the board with their “concerns” is not helping you. Your argument that you are “only concerned about what’s best for the organization” is not fooling anyone. If you left two hours early yesterday, then complained that Suzy came in an hour late today, it’s pretty evident that you only want to “get Suzy”. Do you really think that no one noticed you left early? Complaining about Suzy just makes you look like a hypocrite. It WILL get back to Suzy, and how likely do you think she is to lift a finger to help you in the future? Your co-workers are going to trust you less because they expect you’ll do the same to them.  When you leave your position, odds are good that supervisor you complained to (or about!) will subtly let others know that you are difficult to work with, making it harder for you to find a new position.

Even if you are not so crazy, it reflects poorly on you. For example, I do not do mornings and before I agree to work with anyone we have an understanding that if I’m in the office before 9 a.m., it had better be REALLY important. On the other hand, I’m working past 10 pm on a regular basis. I never miss a deadline. But I still don’t do mornings.

I’ve been in a few situations where someone has gone to the powers-that-be to “let them know” that I didn’t come in until 10 a.m., well, ever. In every case, since upper management had already agreed to those hours, it reflected badly on the person complaining. Their boss would be offended that Complainer was criticizing an arrangement that he or she had made with me, since it seemed to be questioning the boss’s judgement. It also was clear that Complainer hadn’t mentioned any concern to me – setting him/her up as a person who doesn’t  try to solve problems but just does running to the boss.

STOP!

Before you get to the point of  climbing under a desk and short-circuiting Suzy’s computer, ask yourself what this is all about, because, believe me, your colleagues are asking that question already.

Is it REALLY because you are so concerned about whether the company is over-paying for Suzy to have three meals a day on her per diem when she did not leave on her trip until 11 am, so she should rightfully only get breakfast? Really? That was the biggest problem that needed your attention at work today?

If your motive is to get Suzy fired so that you can get her job, it’s probably not going to work. Even if it did,  you might forget that you had booby-trapped her computer and die an agonizing fiery death your first day on the job.

You don’t want this, do you?

Even if Suzy gets fired, there is no guarantee you’re going to get her job. Why should you, really? Does she really suck totally at her work? If so, she’ll probably get laid off or fired eventually anyway. If she doesn’t have as much education or experience or, let’s face it, intelligence as you, is that sufficient reason to fire her and give the job to you? From your perspective that might seem fair, but that’s not the way  the world works, because if it did, we’d have a never-ending cycle of firing and promotions. There are actually several very good reasons to keep people in a job even if you can get someone “more qualified”. Yes, employers are correct to not hire people who are “over-qualified” and replace the less qualified people who are working for them. That is a post for another day, since I have to head to the airport in a minute.

Being honest about your motives is going to make you a better person with a better life in all areas. Here is a non-business story but I think it relates.

Many years ago, my daughter was competing in judo. She had a back and forth rivalry with another child. I could not stand this kid’s coach. About the same time, Ronda was going through a growth spurt and it was difficult for her to stay in the weight division, but she was also getting much, much better. When signing her up for the next tournament, someone made a comment that we (the coaches) were just using our children to get back at one another. Now, this comment came from one of the most odious people you’d ever meet, who made a hobby of saying hateful things to people – which didn’t make him any less right in this instance. I moved Ronda up a weight division over her complaints (“I can beat her, Mom!” “Yes, I’m sure you can, but you’re not in the same division any more.”) It was the right decision, even if the person was being snidely critical, he was correct.

Ronda went on to do well in judo and now mixed martial arts and I have no idea where  the other kid is now, nor her coach.

Whether it is on your coaching style, your research design or your code, people often do criticize you because they are just plain jealous and want to hurt you. That doesn’t mean they never have a point. If the criticism is correct, fix what you can and go on with life as they go to hating on other people.

Jealousy is bad for you but it may actually pay off for the people you are jealous and critical of if they have the right attitude.

Funny how that works.

# Success! With SAS Web Editor

So, I did get my data uploaded to SAS Web Editor using ftp. Little hint here, if you are using Filezilla and having trouble connecting, check to make sure that Filezilla is not using sftp. Sometimes in the box labeled Host it will be prefaced with sftp as in sftp://sascloudftp etc. Remove the s so it just says, for example, ftp://sascloudftp.sas.com . I wrote about that previously so I’m not repeating myself here.

2. Use PROC CPORT to export the formats

This is not always necessary but IF, as I mentioned yesterday, you have a problem reading the formats, this is your next step and thanks to Tom Edds from SAS for suggesting it. One thing PROC CPORT allows you to do is transfer data sets from older versions of SAS to newer ones.

Run this program on your desktop (assuming the data are located on your Windows machine).

FILENAME trans “C:\Users\AnnMaria\Documents\oldpeople\sasdata\trans” ;
LIBNAME in “h:\sasdata” ;
PROC CPORT LIBRARY= in FILE = trans MEMTYPE = CATALOG ;

The FILENAME statement specifies where you want your transport file written, the LIBNAME statement is where your format catalog is located.

So, this created a transport file to upload.

3. USE PROC CIMPORT to import the formats

Run this program on SAS Web Editor

FILENAME mydata “/courses/myuniv/dir/otherdir/trans” ;
LIBNAME test “~/Projects/oldpeople/sasdata/formats/ ” ;
PROC CIMPORT LIBRARY = test INFILE = mydata ;

The FILENAME statement refers to the file you created in the previous step.

The LIBNAME statement is where you want the formats written. This directory must already exist.

4. Now you’re ready to access your data using SAS Web Editor

LIBNAME saslib “~/Projects/oldpeople/sasdata/formats/ ” ;
OPTIONS FMTSEARCH = (saslib) ;

The first LIBNAME statement is the course directory where you uploaded your data .

The second LIBNAME statement is the folder in the Projects folder where you stored the formats. You could put them in a subdirectory of the same course directory where you uploaded your data. You could put them almost anywhere, really.

OPTIONS FMTSEARCH = tells SAS where to search for the formats it expects.

P.S. Thanks to Tom Edds from SAS for suggesting the cport/cimport .

# More adventures with SAS web editor

June 4, 2013 | 1 Comment

After dropping The Spoiled One off at the beach and setting a personal best for calls returned, I’m back on setting up data sets, assignments and more for the fall semester.
So ….. I tried using the UPLOAD option from the menu in the SAS Web Editor and that was a sad failure. Next, I tried using FTP and that worked with one small caveat.

First of all, to find the path you will need and other information for FTP, you can read about it on this post. Also, you will find a picture of a badger and learn, if you did not already know it, that the proper term for an aggregation of badgers is a “clan”.

It uploaded in next to no time and all was great except for the fact that it does NOT show up in my Projects folder. If you have been using SAS Web editor with the upload feature, it may be disconcerting to you (or your students) that now you appear to be working with data that is invisible.

However, if you do use the correct LIBNAME statement you’ll have no problem accessing your data.

So …. I got as far as a proc contents. Then, when I tried running some simple analyses I got this message

ERROR: File SASCAT.FORMATS.CATALOG was created for a different operating system.

This probably is not anything specific to SAS Web Editor. The original data is nine years old, God knows what it was created on. I downloaded it on to a 32-bit Windows OS running on Boot Camp on a Mac then uploaded it to SAS Web Editor. I’m not surprised there was some incompatibility.

My next EIGHT HOURS are spent either in meetings or driving to and from meetings – GRRR – but after all that I’m going to try the PROC FORMAT with a  CNTLOUT option, then try using PROC FORMAT with CNTLIN on the other end on SAS Web Editor. I think that will work. Stay tuned.>In the meantime, there are a couple of points to learn from this

1. As I’ve ranted on here before, don’t think that using open data for your class is going to be simple. Plan in advance. Give yourself time to identify any problems
2. Just mentioning the PROC FORMAT CNTLOUT = and even the use of a FMTSEARCH options gives me pause to think that perhaps you are in the best position to teach using SAS Web Editor if you are proficient using SAS yourself. This is just  a general rule on teaching, I think. There are universities that want to hire people as cheaply as possible and get someone who reads the textbook one chapter ahead of the students. I’m very happy that I don’t work anywhere like that. If that person is you, you’re probably going to be deeply frustrated if you try using SAS Web Editor. All isn’t lost, though. Repent! There are a lot of helpful resources for faculty, so start in the summer, solve all of the issues like these ahead of time and both you and your students will be better off.

I do realize there are schools that only tell people they are hired a few weeks before the semester starts. That sucks. I don’t know what to tell you to do in that case except try to find some place to work that does things in a less half-ass fashion. Good luck.

# Testing SAS On-Demand

With three children under age six while I was writing my dissertation, I learned never to leave things until the last minute because that is EXACTLY when two of them would come down with the chicken pox. That’s why I’m working on assignments, lectures and videos for fall semester now. I was asked to teach an on-line course and I thought SAS On-demand might be perfect for that. Here’s the scenario –

• Select a large, public data set and upload it to the server. This way, every student has access and there only needs to be one copy.
• Students could get some experience working with relatively big data and the issues that come with messy, real data. The example I selected was a longitudinal study with 14,000+ records and over 1,200 variables.
• This will be great, thought I, with  this number of variables, there is certain to be something to interest everyone in the class.
• Also, since it is one data set, I can take care of some of the issues that new graduate students might not be familiar with – a format catalog, for example – so that we can concentrate more on the statistics.
• With only one data set, I only need to go through one codebook.
• Since it is online, I don’t have to worry about scheduling computer labs, whether it is compatible with students’ operating systems (Mac, Windows XP – yes, people still use that).
• Plus, it’s free, so students don’t have to buy yet one more thing, and I know how they hate that.

The dataset really isn’t that big – about 70 MB – but it’s been a while and it STILL has not uploaded. By contrast, my .5MB format file uploaded in nothing flat. So, I think I will go to bed and see if it has uploaded by morning. This is not a good sign. If it takes this long to upload, I wonder how long it is going to take to run any analyses.

If this doesn’t work … which is starting to look sadly probable – I’m on to Plan B which is to winnow the data set down to maybe a couple of hundred variables, making the data set one-sixth it’s current size, and start again.

This kind of defeats the purpose of having them work with sort-of big data, so I am sad.

P.S. Now it’s morning and the file never did upload. On to Plan B as soon as I drop The Spoiled One at the beach. Yes, she could walk to Santa Monica Beach but

“We don’t want to go to that beach. We want to go to a beach in Malibu.”

To get the true spirit, you must say that in as whiny as tone as possible – which wouldn’t be so bad except that they don’t want to do go to the closest beach in Malibu, which is a couple of miles from here. No, we need to pass several perfectly good beaches to get to the cool beach. So, I’m off to make a 25-mile round trip to the middle of Malibu and back. Did you know Malibu is 27 miles long? Why did God make so many really rich people? It just seems unnecessary.

# Never let go of anything you need – & other travel advice

I travel A LOT for business. This occurred to me today as I realized I knew where to find an outlet in the one restaurant in the Grand Forks International Airport. (It is international because the have a couple of flights two and from Canada, which is about 80 miles due north and it is a restaurant because it serves wine. I’m not complaining. I have wine and power, which is more than I can say in a lot of airports.)

If you travel, there are basically two kinds of problems you can have:

• Not having the stuff you need.
• Not having the time you need.

So here, from massive experience, are a few things I have learned in (almost) 55 years.

If you NEED to have it when you get there, never let it out of your sight.

I mean this most literally, both the word “need” and the phrase “never let it out of your sight”.

What is is that, if you arrived at your destination and found you did not have it, your first response would be:

I am so screwed!

For example, I am legally blind without my contacts or glasses. So I need an EXTRA pair of contacts or a pair of glasses in case the pair I am wearing gets lost or ruined. I cannot see, drive or work without them and they are at the strength that I guarantee you that no one is going to have them in stock.

Any data on my laptop has been stripped of personally identifying information, but still, there is a lot of work on there, plus I need it to work wherever I land. I can’t always count on being able to connect to the Internet and dropbox.com has turned out to be a great disappointment in terms of reliability.

If I bring a projector with me, it’s because I’m not sure that where I’m going will have one and I need to do a presentation. Again, I’m not letting go of it.

I can probably buy clothes wherever I’m going and if not, I can wash what I’m wearing in the hotel and dry it with the blow dryer. I have pulled clothes out of my carry-on and replaced them with a projector when I was required to check my luggage plane-side because I was flying in a six-seat puddle-jumper to some place. Good thing I did, too, because somehow my bag didn’t make it on that plane. The only time I let go of anything I MUST have is when I go through security and I’m legally required to do so. My list:

• Cash
• Credit card
• Debit Card
• Identification (drivers license or passport)
• Laptop
• Spare contacts or glasses
• Projector (depending on location and purpose of visit)
• Cell phone

If you decide what you absolutely must have and keep hold of that small subset of stuff, any problems you have may be inconvenient but nothing will rise to the level of a catastrophe.

Give yourself enough time and then add time.

I’m continually amazed by people who leave for the airport in time “if everything goes as planned”. Nothing ever goes as planned. I usually check my flight date and times the week before I leave and then forget (I travel a lot and it’s easy to get the times mixed up). I check again the night before and again a few hours before. I absolutely positively refuse to fly early in the morning because I know myself and I know damn well I am going to sleep until the last possible minute. If I can’t fly after 10 am then I’m coming in the night before on a red-eye. If your business requires me to take a 6 a.m. flight then we’re not doing business, sorry. (I think that should be another post – don’t work for an asshole.)

If you are going to LAX assume that there will be traffic any time of the day or night. If you’re flying Southwest out of terminal 1 in LAX – well, just don’t, but if some fate forces you to do so, give yourself an extra hour. Unless you KNOW that something will not be an obstacle – for example, it is a safe bet you won’t run into a traffic jam heading to the Grand Forks airport on Saturday afternoon – assume it will be and plan accordingly.

So what if everything DOES happen to go right and you get there an hour and a half early? If that’s the worst that could happen, find something to do – read a magazine, watch CNN, call your mother. I would say always have work with you that you can do – that’s what I do but I’m not sure it’s a good thing and is perhaps something that I should UNlearn after (almost) 55 years, but that’s a subject for another day.

« go back