If that sounds like total marketing b.s. to you, I can’t blame you — download this app and supermodels will throw themselves at you with thousand dollar bills in their teeth.
Well, no super models are provided but the SAS Web Editor really *will* write your code for you in the latest version soon to be released. They have these things called templates which are code snippets.
So, if you want to do a PROC SQL, you can select that option and the code will be written. You’ll still have to select the variables you want, of course.
There are other code helper options and that would be the big selling point for me if they were actually selling it instead of giving SAS Web Editor free to universities. Often, I end up doing that myself – creating models for students to follow to ease them into programming.
If you are at SAS Global Forum you can go to meet up in room 2001 at 6 pm and find out for yourself. You can also say what else it is you would like to see in the next release.
Personally, I’d like to see the option to email output added because my students often email me their homework. Think how easy it would be if they could run SAS, get the output and click an email button to send it to their professor. Well it would be easy for *me* and for the students, not so sure how easy it would be for the SAS people to get to work,but it’s not about them, it’s all about ME.
Trying this live blogging from SAS Global Forum again.
The title kind of says it PROC QUANTLIFE new procedure in SAS 9.3
Why DO we need a new procedure for survival analysis?
Survival analysis used to analyze time-to-event data
already had procs lifetes, lifereg & phreg
Lifereg is fine if you have IID errors – but what I’d you don’t . Enter quantile regression, possibly wearing a cape #Sasgf13 #noCape
Qy(tau) is the tau-th quantile of a random variable Y eg Qy(25) is 25th percentile
Quantile regression – can have same slope & different intercept for each value given for tau
Quantile regression, option 2 can have different slopes for each value of tau #Sasgf13
Cumulative distribution function is the inverse of the quantile function #Sasgf13
QUANTLIFE example shows covariate that has negative effect for those with short life but positive effect for those with longer life #Sasgf13
Interested in survival analysis when covariates have non-linear relationship to time to event? Check the QUANTLIFE procedure paper #Sasgf13
Here is my first attempt at live blogging on my new iPad and my first effort with the WordPress app.
Peter Eberhardt doing a Hands-On workshop at SAS Global Forum. It’s standing room only well it would be standing room only if they let you stand. It’s actually full & they are going to turn people away. If the turn you away, don’t pout, there are plenty of other awesome sessions.
The workshop is on doing the perfect pivot table with SAS
Starts by explaining what a PivotTable is & how do one in Excel
To do in SAS only need base SAS
Download the TableEditor tagset from the SAS website.
Link in Eberhard’s paper
TableEditor tagset not included in the SAS distribution
It’s a file with a PROC TEMPLATE
According to Eberhardt you don’t *have* to know anything other than it exists, but you can modify it if you want, for example to change the color scheme
Start your program with
ODS TAGSETS.TABLEEDITOR FILE= “filename.html”
options( auto_excel = “yes” pivotrow =”some_name” pivotcol = “othername”
Creates an html file
auto_excel = yes —
Back to that later, I guess. Looking at the tableeditor.sas file.
Lots of proc template code.
Okay, back to our ODS statement
button__text = “”
auto_excel = “yes”
pivotrow = “some_name”
pivotcol = “other_name”
pivotdata = “other_name2”
pivotpage = “some_other_name”
Don’t worry about the ActiveX warning that comes up. You also get told there may be data in the other worksheets and you might be deleting it. Just say it’s okay.
Summary variable – sum statistics is selected by default
pivotdata_stats = “sum, average”
if you want a different column for each statistic, you can match them up like this,
pivotdata = “some_name, othername”
pivotdata_stats = “sum,average”
The auto_excel = yes puts a button on your html output file which will automatically start Excel when you click on it
Not sure the pivot table tagset is something I would use personally but it is kind of cool
Peter does not know if it would work with openoffice / libreoffice if you clicked on the auto_excel button. Well, that sounds like a challenge (-:
Creating multiple pivot tables – I remember working at a company years ago where we did a zillion of these by hand , well automatically more or less using Excel, but we did do them one at a time. We wrote some clunky VBA macros.
pivot_series = “yes” <— gives you the multiple sheets pivotrow = “Name1 | Name1 | Name1” pivotcol = “Something | Something| Something” pivotdata = “Variable | Variable | Variable ” pivotdata_stats = “sum | average | minimum” This will give you three pivot tables, one each with the sum, average and minimum of the variable over whatever the something column and Name1 rows are ============== Eberhardt says list of options for table editor tagset is in his paper ==== Big advantage of using tableeditor tagset is you can do the code once, automate the task and have it run every day, week or whatever. Again, I don’t know if I would use this personally on the projects I’m working on now but I certainly can see the usefulness of it ===== Bottom line – if you have SAS & you need to do a lot of pivot tables in Excel particularly if you have to do the same ones repeatedly, just as the data change, get Eberhardt’s paper in the SAS Global Forum proceedings. You can also creat pivot charts
Okay, well, I lied (sort of). I have no idea what will happen at the SAS Academic Reception tomorrow except that it will be from 2-4 pm and I will be there. Also, there is a panel where people (including me) will be answering questions from the audience about how they use SAS in teaching.
Also, there is usually free food, so if you are at SAS Global Forum, you should come, especially if you are a graduate student because
a> You may learn something useful to your future academic career,
b> If you are a graduate student, you are probably broke and hey, free food.
My notes on things I want to mention ….
SAS web editor
Can run on any device (almost) – windows, pc , ipad
tip: can use your phone as a personal hotspot if you only have the wifi enabled ipad.
can upload SAS files into the project directory with just the upload
can also upload excel files (I think you need to code the PROC IMPORT )
Much faster than the enterprise guide version & doesn’t require Windows
Can run enterprise guide on the mac but need to use bootcamp, not a virtual machine (unless it has changed since the last time I tried it)
Advantage of Enterprise Guide – process flow gives visual representation of logic of project.
MAJOR (dis)advantage is no need to learn programming
Tasks with specified roles for dependent variable, independent, nesting etc. is helpful in teaching statistics
for both the web editor and enterprise guide i extend class time by making movies of how to install (in the case of EG) and how to use the products for specific assignments. This gets around problems of slow wireless in the building or even uncertainty of wireless availability
Originally began making videos available on-line before and after class at the request of two students for whom English was a second language but since then many other students have downloaded the videos and now I just do it as a matter of course.
SAS has a lot of instructor material available. Unlike much documentation which is only read as a last resort, I HIGHLY recommend that you read theirs.
With all of these positives being said, I have two points to add, one plus and one minus
Using SAS whether it is in a wired desktop computer lab or on an iPad with a personal hotspot from your iPhone takes prep time and quite a bit of it. If I was teaching statistics along with three other completely different courses, I don’t think I would have time for it.
I strongly recommend if you are planning on using SAS with your class (& I recommend it) that you start in the summer. Decide what problems you want to do to and run the solutions (I had a teaching assistant for this). If using open data, which I also recommend, select the data sets you want, then clean up the data and create a code book (I had a teaching assistant for some of this as well)
It’s fabulous when your students have their own interests and want to do a project that is original research – but that usually means you are going to help them find, download, clean and upload the data.
The plus you may not considered is using SAS in teaching even when you are not teaching statistics, or when your students are not quite to the point of doing programming themselves. I get asked from time to time to speak at elementary and middle schools on what statisticians do, on statistics in general, and SAS is great for producing graphs and tables matched with the interests of the students in the classes. Sometimes, I have them send me questions ahead of time and show up with the answers
I’m always a bit bemused when people refer to me as a “SAS expert”. I don’t think of myself as an expert at anything except, perhaps, bricolage, a word I am indebted to sascommunity.org for even being aware of its existence.
Merriam-Webster defines it as:
: construction (as of a sculpture or a structure of ideas) achieved by using whatever comes to hand; also : something constructed in this way
Origin of BRICOLAGE
French, from bricoler to putter about
Often, I think there are probably much more elegant ways of doing things but mine gets done very quickly which my clients appreciate since they pay me by the hour.
One example of bricolage that comes to mind is my frequent use of PROC SUMMARY for output as input.
Take this recent problem.
A researcher had three measurements on each subject. The dependent variable was the mean of these measures. They were all taken at the same time, so this wasn’t a repeated measures type of design. All I needed was to get the mean. Data were entered like this.
ID Group Measure
01 ooo1 .47
01 ooo1 .46
01 ooo1 .46
02 o001 .49
02 ooo1 .48
I could have created a couple of variables using the LAG function, but really, I found this to be much quicker.
PROC SORT data = myfile ;
BY ID GROUP;
PROC SUMMARY DATA = myfile ;
BY ID GROUP;
VAR measure ;
OUTPUT OUT = myfilefix / AUTONAME ;
DATA myfilefix ;
SET myfilefix ;
WHERE _STAT_ = “MEAN” ;
DROP _STAT_ _FREQ_ _TYPE_ ;
See what I mean about bricolage? I’m sure a real expert would have used some DROP option on the dataset in the PROC SUMMARY or something and not needed the DATA step to only keep the variables for ID, and the mean of the measure variable, and some other option to only compute the mean statistics but since this took me all of five minutes and it was not a large data set to worry about sorting and time taken by extra steps, I didn’t bother. The reason for including the group variable in the sort and proc summary as well as the id variable, even though it is obvious that the same individual will always be in the same group, is simply so the group variable was carried along and saved in the file output by PROC SUMMARY. I’m sure a real SAS expert would have a less kluge-y way of doing that, also.
The researcher had individual subject data in one file and the group data in another file. For example, a record of all students and a record of data for the classroom. We want to merge these two files. In this case, however, the student file had a student id variable and the class file had a classroom id. Fortunately, there were 10 students selected from each class, so that students with id numbers 1-10 were in class 1, id numbers 11 – 20 were in class number 2, and so on.
I could have done a PROC FORMAT or maybe a DO -loop. What I did was this”
class = INT((student – 1)/10) + 1 ;
If I use the SAS function INT take the integer part of 1/10 to 9/10 I get 0 and if I add 1 to that, I get class = 1.
What about student number 10? Well, he or she will end up with 10/10 = 1 , add a 1 and that student will be in group 2. Not correct.
If I subtract one from every student number, it works out and sorts them exactly correct. I’m sure a real expert would know some function that does exactly that, but hey, it was one line and gave me the exact results I wanted.
See what I mean? Bricolage.
I will concede that there are times when I am working with an enormous dataset when I need to do things as efficiently as possible, or when I am teaching and I need to show the exact “best” method .
There are other times, though, when I just slap something together and call it a day.
A friend of mine, also a consultant, who would definitely consider himself a SAS expert, said, disapprovingly,
“I would never do that.”
He went on to ask me if my clients were satisfied with the way I work.
I told him that sure, I have lots of the same clients for a decade or more. Often, with simple problems like the ones above, it takes me so little time to get the correct results, I just knock it out in a few minutes and I don’t even charge them, which makes them particularly satisfied. That’s the part where he REALLY gasped and said,
“I would NEVER do that!”
Several years ago, I was fortunate enough to attend a lecture by Albert Bandura, arguably the greatest living psychologist. To almost anyone who ever took an undergraduate psychology course, he is best remembered for his Bobo doll study, where he found that young children who observed aggressive behavior being reinforced (on a doll), were more likely to be aggressive themselves – that is, modeling and vicarious reinforcement contributed to aggression.
This talk wasn’t about that. It was about happiness.
Dr. Bandura was talking about one of his studies on happiness and I remember him saying that one of the strongest predictors of happiness in the subjects in his study was, “an absence of self-ruminative thoughts.”
I’ve always thought rumination was slightly disgusting. It’s what cows and other ruminants do, defined as “to chew again what has been slightly chewed and swallowed”. Turns out, not only is the physical kind of rumination a bit yucky but the mental kind is particularly bad for you.
We all do it sometimes – wonder what that cute guy in physics class thinks of me, did he notice I wore a particularly nice outfit today or that I got the answer right to that especially difficult question the professor asked, does he think maybe I’m too pushy calling out the answers in class, maybe that answer really wasn’t perfect even though the professor said it was correct. Maybe there was a more complete answer and I left out part of it. Would it have been better if I mentioned how it related to the section we just read on forces? And on and on.
I’m pretty happy most of the time and I attribute a lot of it to having missed home economics class on the day they brought all of the girls together and told them that they needed to evaluate themselves constantly and in parts. I have never, ever understood the women and girls I meet who say, “I think my nose is ugly. My butt is just too big. I’d be so much happier if I could just lose a few pounds. My hair looks awful like this.” And on and on.
Here is what I think when I get up in the morning and look in the mirror.
I look fine.
Then I brush my teeth and my hair and go on with my day. The world of full of people both prettier and uglier than me. I’m over it.
What Bandura said is that the happiest people were focused outward. Mihaly Csikszentmihalyi said something very similar, that we are happiest when we are absorbed in an activity. When I am happiest is when I’m working – designing a program, reading research, writing code. When I was young, what made me happiest was when I was doing judo, running track, or yes, writing code which we called programming back then because we weren’t cool enough to call ourselves software developers.
I went to a start-up event a couple of weeks ago, and the very best comment I heard from any of the speakers was this,
“Quit telling yourself that everyone knows more than you. First of all, they don’t and secondly it doesn’t matter. If you keep working at whatever it is you’re doing, you’ll get better.”
Quit thinking about you and start thinking about the task at hand. It’s probably the secret to success and it’s definitely the secret to happiness.
And that’s #18 of 55 things that I’ve learned in (almost) 55 years.
While I agree with much of what entrepreneur/ professor/ author Vivek Wadhwa writes, I think his latest article in the Wall Street Journal is a little off target. He says,
Why would any company pay a computer programmer with out-of-date skills a salary of say $150,000, when it can hire a fresh graduate — who has no skills — for around $60,000?
I’m heading up to the SAS Global Forum on Saturday with thousands of other people who will be learning about everything from data visualization to random forests to high performance computing. I haven’t been sitting around running one ANOVA after another on BMDP for the past thirty years. (For those of you who think SAS is old hat, you might be surprised. If they are successful in their moves into web-based products from Enterprise Guide to data mining, the people who thought SAS would be decimated by R might end up looking like the people who said Apple should sell its assets and give the money back to the shareholders. See, that’s the thing about us older people – we DO have experience and sometimes it’s applicable.)
Perhaps Los Angeles is different than Silicon Valley, where I hear people in their forties get Botox and other “work done” so they could appear younger to venture capitalists. The irony of LA being LESS youth-obsessed is certainly not lost on me.
This next line in the article made me laugh out loud, really.
Even if it spends a month training the younger worker, the company is still far ahead. The young understand new technologies better than the old do, and are like a clean slate: They will rapidly learn the latest coding methods and techniques, and they don’t carry any “technology baggage.”
I’m married to someone a few years older than me and he is brilliant. I guarantee you that he is worth triple any fresh graduate – and he was getting that, too, until he decided that he would rather work at home with me on 7 Generation Games doing programming with Unity 3D
Although Wadhwa makes the comment with which I wholeheartedly agree
To be writing code for a living when you’re 50, you will need to be a rock-star developer and be able to out-code the new kids on the block. Top developers are always in demand and companies will readily pay top dollars for them.
My gripe with him is that it is buried far down in the article after spending two-thirds of it equating older with obsolete. Again, my experience is undoubtedly not typical. A very large proportion of the people I know do fall into that rock star category. Truly, I often mention on twitter how blessed (and that is the most accurate term) I am to be surrounded by brilliant people in my daily life. I’m sure my experience is disproportionately filled with those rock stars and I am truly grateful for that. If his point is that it’s hard to retire in place with mediocre skills, well, that is true, but I think it has probably always been true in every field.
Where Wadhwa is correct is that
Prominent Silicon Valley investors often talk about youth being an advantage in entrepreneurship. If you look at their investment portfolios, all you see are engineers who are hardly old enough to shave. They rarely invest in people who are old.
Part of that may be due to the other alternatives older entrepreneurs have. We have investments from the past 28 years of income so we don’t need to give up equity in the company at this point. Also, I’m cool with not working with anyone who thinks they could hire someone without the skills our team has and teach them to be equivalent in a month. Good luck with that. Maybe that explains why such a large percentage of start-ups in your portfolio fail.
It has been pointed out to me that when other people take two days off in a row, it is called “The Weekend”. However, I don’t really have any hobbies except for judo, which I teach a couple of times a week. I like hiking in the mountains, but that only takes a couple of hours since we live less than a half hour from the Santa Monica mountains.
So, for example, on Wednesday I walked up to the graduate library at the top of the Pepperdine campus in Malibu, passing this deer on the way, but it didn’t take that long and there was still the rest of the day to work.
While I’m perfectly happy with my life, unfortunately, the last few weeks of trying to fix everything on the game has taken its toll and my elbow hurt ALL of the time. The Invisible Developer insisted that we take off somewhere and I leave my computer at home. So, we went wine-tasting in the Ojai valley for two days.
Here is the funny thing, though. Without my computer, I worked on the game a lot and made huge progress. I sat out in the backyard of the cottage and wrote (yes, with pen and paper) out several of the math challenges, several parts of the game, sketched out the logic for the program and even wrote a bit of the code by hand. Without twitter, Facebook, email or blogs to distract me, I got an amazing amount done. Since we were near The Spoiled One’s school, she came in the evening to relax and do her homework away from the dorms.
She thought it was a great idea and that we should do it every week. While that may be a bit much, and I’m pretty certain the wine cannot be deducted as a business expense, we’re planning another computer-less weekend this summer.
There is just one thing I haven’t figure out, though?
Why DO they knit sweaters for the trees in Ojai?
I haven’t been using Enterprise Guide much lately but tonight I could not get the SAS Web Editor to work and I had some graphs I needed to get done. It was convenient to do this with SAS Enterprise Guide because I wanted to do bar charts of the mean of one variable broken down by three other variables. There are a few things, though, that I always forget how to do with SAS Enterprise Guide graphs and I always have to look them up again, so I’m writing it all in one place for the next time I need to remember how to do this.
For example,let’s say I gave children one of two brands of juice. They received the juice either 25%, 50% or 75% of the school days and the other days I gave them either distilled or fluoridated water. At the end of some time, I wanted to measure the amount of Fluoride in the children’s system. (Don’t worry if this is possible or not, it’s just an example). SAS is handy for this because you can create a summary table using the SUMMARY TABLES task and then under results click to save the results to a data set. Now I have my data nicely set up to graph.
I want it to have one bar for each unique data value. I don’t want it to scale the X axis to fit the data and show bars at 30%, 60% or whatever. I want it show the EXACT value of 25%. (Note: If these images are too small to see you should be able to double-click to bring up the original image size.)
The first part is easy. Under the APPEARANCE tab in the left window pane, click BAR. Then, in the window that appears near the bottom, click on SPECIFY NUMBER OF BARS and click next to the first button ONE BAR FOR EACH UNIQUE VALUE.
Also, I have two different groups and I very, very much want the bar charts to have the same vertical axes to make it easy to compare them side by side.
SAS does NOT do this by default. Instead, it scales to fit your data. I don’t want that. I want both charts to have the same maximum and minimum.
The next step is to make sure the Y axes of each chart are the same. Under VERTICAL AXES click MAJOR TICKS. In the window that appears, click the button next to the last option, SPECIFY. On the right side of the window you can now give individual tick marks for the Y axis or you can give a scale, such as 0 TO .5 BY .05 .
Note: I did this with SAS Enterprise Guide 4.2 and it worked fine. I did it with SAS On-Demand for SAS Enterprise Guide 4.3 and it did NOT work. Not at all. Not even when I copied and pasted the code into the program editor and ran it. It just did not re-size the axes. I have a very old version of SAS Enterprise Guide/ SAS On-Demand. In fact, the only reason I was using it at all was because the SAS Web Editor wasn’t working. So, hopefully that is something fixed with the new release coming out in June.
Minor thing – I want to change the label on all of these to make my chart look better. Since it already says in the title it is the mean Fluoride level, I want to change the variable name from what the TABULATE procedure named it Fluoride_mean to just Fluoride. To give a variable a different label on the chart, right-click on the variable name once it has been dragged into the task role on the right window pane. Select PROPERTIES. A window will pop up that allows you to change the properties (duh). Type in whatever label you want.
After you have given it a label and closed the properties window, right-click on your variable again. This time select SHOW LABELS.
So, now I have my nicely labeled chart with the same Y axis for both graphs and a unique bar that matches exactly each of my categories.
Just another random thing I always forget. I happened to be doing this running Windows 7 using boot camp on a Mac. If you want to take a screen shot using a Mac some help pages tell you to use the F14 key. Just one problem – my keyboard only goes up to F12. Fn-shift-F11 will do a screen print when you are using boot camp.
Darling daughter # 2, also known as The Perfect Jennifer, commented that she had the least impressive career of all of my daughters.
I responded with true maternal tact,
“That’s a load of crap! You’re far more likely to make a difference in the world than any of them. Yes, it’s great that Ronda is making lots of money and has a flashy car, and we’re all proud of her that she won a couple of world titles – but how many people have you ever heard say their lives were changed by some fighter, or anybody, who they saw on TV? Maria is a terrific writer and it’s great that she’s been at ESPN for years, but almost never do you hear someone tell you about how important a sports writer was in their life. Almost EVERYBODY, though, can ten, twenty or even fifty years later mention a teacher who made a difference in their life. You hear all of the time about people who became writers, doctors, or yes, even statisticians, because that one teacher inspired them. YOU are in a position to change people’s lives and I know you do because I have seen your students and I have seen you teach.”
“That’s nice of you to say that.”
My brother, who after putting his own children through college, earned a Ph.D. in education and now teaches middle school mathematics was talking one day about his desire to have more of an impact on more students.
I told him,
“You never know. I’m sure if my first statistics professor remembered me at all – and I cannot imagine that he does – he would say I was one of the students he did not reach. It was a Monday, Wednesday, Friday class and I skipped every Friday to go to frat parties. Thirty-five years later, every time I teach about the Central Limit Theorem, I remember Dr. Spitznagel’s class, with all the very serious mathematics graduate students and one sometimes hungover 19-year-old senior in a halter top, who would be me. I’m sure Chris and Phyllis, the two math teachers who taught me Calculus and matrix algebra in high school despaired twice as much because I sometimes skipped class to smoke pot behind the school, but forty years later I still remember their names, how to find a derivative and the determinant of a matrix.”
“That’s nice of you to say.”
Although he did have to glare at me disapprovingly about the pot-smoking and drinking. He always was the perfect one in our family. I think Jennifer takes after him.
Both of them should know me better than to think I ever say anything just to be nice. It’s true. Teachers matter. Long after the latest hedge fund manager has retired to his tax shelter island with his fourth wife. Long after today’s greatest innovative disruptive start-up has collected $10 million in venture capital and then gone bankrupt. Long after we will have all forgotten David Brooks and his students at Yale who think that only the time you spend getting yourself higher SAT scores and more money is not time wasted. Long after we have forgotten all of them we will still remember the teachers who mattered.
The most interesting man in the world, or, at least, the most interesting man I ever met, is Dr. Carter Revard. I took an independent study from him in 1978 in the last semester of my senior year at Washington University in St. Louis when it turned out that I could not graduate without ever having taken an English course. Apparently, hating English is not a sufficient reason for them to waive the English requirement. Who knew? I was amazed by his breadth of knowledge. He commented that he tried to take a class on something every semester, since he was at the university and it was free.
While I never managed to do it every semester, I have tried to take one class a year. I took microbiology, matrix algebra, data mining, COBOL, cyberpsychology, Chicano Studies. Every now and then, I try to take a course in an area I know nothing about so I can relate better to my students who are taking multivariate statistics for the first time and thinking
I’m going to be teaching an online course this fall – for the first time since 1999. You can imagine how much online courses have changed since then! So, to try to better understand how my students will experience it, I decided to take a bunch of courses from code school. (It’s a site, www.codeschool.com )
So, even though Dr. Revard taught English which I absolutely hated, he made enough of an impression on me that I am still, 35 years after graduation, I’m trying to continue putting myself in my students’ place.
Teachers matter. Even to that kid who hates your subject and skips your class for frat parties.