It’s been a pleasure speaking to groups around North Dakota this past week, in part because I was asked a lot of intelligent questions, which really forced me to think about the answers.
One young woman asked how I maintained a positive attitude when times were difficult, when my husband died, when there is a seeming unending pile of work to do, when my children are heading in what I think is the wrong direction.
The answer is that I try every day to wake up grateful, and it really is pretty easy if you are realistic and honest about your situation.
Read any history book – and not ancient history, either – about people breaking the film of ice on the pan of water IN THEIR HOUSE, as they started their day, to wash clothes or make coffee.
Even today, people wake up sleeping under bridges, on the ground in refugee camps. I’ve lived in old houses where the wind blows through cracks in the winter.
North Dakota is cold and for almost the entire history of the world there wasn’t much anyone could do about it. Yes, people discovered fire, hunted, had deer skins, tipis. However, it was nothing like the last few days when I woke up every morning, warm and comfortable, in well-insulated houses, on soft mattresses under a pile of quilts.
Being at Minot State University reminded me of my own graduate school days at the University of Minnesota. They had tunnels under the campus, connecting buildings, for which I was extremely grateful because I was a broke, graduate student and I didn’t have enough money to buy a lot of warm clothes. Any time I had to go outside between buildings, I was SO cold.
These days, buying long underwear, gloves, warm coats, is something I don’t even give a thought. If I need it, I get it. Half the clothes I didn’t even buy – my daughters gave me sweaters and coats for Christmas or because they had more than they could use.
Maybe you think it’s silly to wake up grateful that I have warm clothes and a soft bed in a warm house, but I think it’s objective. There was a point in my life when I had neither. Most of the people who ever lived on this earth had nowhere near the level of comfort that I wake up to every day. If they (or me, decades ago), could be magically picked up and dropped into my life, their first thought on waking up would be,
“Oh my God, this is amazing!”
After laying in bed with that thought for a few minutes, I get out of bed.
Years ago, a friend of mine was in college and had an old, beat up car that leaked oil on to the street where it was parked, which, for some reason annoyed her elderly neighbor. When we returned from a trip overseas competing for the U.S., there was a notice on her car – the neighbor had reported the car as abandoned and we got home just in time to stop the city from towing it away. As a joke, the coach got her a bumper sticker that read, “This is not an abandoned vehicle.”
It’s almost two weeks since I last posted. Contrary to appearances, this is not an abandoned blog!
I just this minute – hurray, tap-dancing – submitted a grant I’ve been working on for the past two weeks.
While writing the grant this week, I’ve been in North Dakota, first giving a presentation on Using Native American Culture to Increase Math Performance. You can see a bit of it that was shown on the local TV station here.
After meeting lots of students at Minot State, we headed over to the Minot Job Corps and I met with students and faculty, talking about our games, starting a company and life in general.
On to New Town, on the Fort Berthold Reservation where I met with the staff and students from the Boys and Girls Club, again, gave demonstrations of our games, and threw a judo demonstration in along with it.
Along there somewhere, I finished the final report on our Dakota Math project that once again found significant improvement in performance of students who played our games, hired two more employees, signed another consulting contract, had way too many meetings and squashed a few bugs in the games.
Tomorrow, I head home to Santa Monica, for two weeks, until I head out to Fort Totten, ND. In the meantime, I’m back to blogging. Did you miss me?
The results are in! The chart below gladdens my little heart, somewhat.
One thing to note is the fact that the 95% confidence interval is comfortably above zero. Another point is that it looks like a pretty normal distribution.
What is it? It is the difference between pretest and post-test scores for 71 students at two small, rural schools who played Spirit Lake: The Game.
I selected these schools to analyze first, and held my breath. These were the schools we had worked with the most closely, who had implemented the games as we had recommended (play twice a week for 25-30 minutes). If it didn’t work here, it probably wasn’t going to work.
Two years ago, with a sample of 39 students from 4th and 5th grade from one school, we found a significant difference compared to the control group.
COULD WE DO IT AGAIN?
You probably don’t feel nervous reading that statement because you have not spent the last three years of your life developing games that you hope will improve children’s performance in math.
The answer, at least for the first group of data we have analyzed is – YES!
Scores improved 20% from pre-test to post-test. This was not as impressive as the improvement of 30% we had found in the first year, but this group also began with a substantially higher score. Two years ago, the average student scored 39% on the pre-test. This year, for 71 students with complete data, the average pre-test score was 47.9% , the post-test mean was 57.4%. I started this post saying my little heart was gladdened “somewhat” because I still want to see the students improve more.
There is a lot more analysis to do. For a start, there is analysis of data from schools who were not part of our study but who used the pretest and post-test – with them, we can’t really tell how the game was implemented but at least we can get psychometric data on the tests.
We have data on persistence – which we might be able to correlate with post-test data, but I doubt it, since I suspect students who didn’t finish the game probably didn’t take the post-test.
We have data on Fish Lake, which also looks promising.
Overall, it’s just a great day to be a statistician at 7 Generation Games.
Here is my baby, Spirit Lake. It can be yours for ten bucks. If you are rocking awesome at multiplication and division, including with word problems, but you’d like to help out a kid or a whole classroom, you can donate a copy.
Simple. You are merging by a variable that is a unique user identifier like username, social security number. Because the two different data sets have different lengths, they do not match. If you are computing the number of unique users you may overestimate by a huge amount. If you want the number of people who are in both data sets, you may vastly underestimate the amount of true matches.
As with anything in programming, there are many ways to do this. My solution is to create a new variable and set it to the identical length and format using the ATTRIB statement. Extra bonus is this will work when you have variables that are not only different lengths but different types, say character in one data set and numeric in the other.
You really only need two statements in your data step, an ATTRIB statement and then an assignment statement that sets the value of the variable you created to whatever the variable is you want to merge.
DATA dsname ;
ATTRIB newvar LENGTH = $49 ;
SET mydata2.dsname ;
newvar = oldvar ;
Repeat this step for the second data set and then merge (or concatenate) to your little heart’s delight.
The voice of experience:
Notice two things here: I created a temporary data set from my permanent one. Although SAS has gotten more forgiving over the years in not writing over your existing data sets when there is an error, it is still better to err on the side of caution and make sure all is wonderful before saving over that existing data, especially if it took you a lot of effort to get the data in that form.
Second, I created a new variable and kept the old one as is. I don’t always do this but it is good practice. You may be tempted to just use the first 9 digits because we all know social security numbers are 9 digits and then later you find that it was entered as 123-45-6789 and now you only have 123-45-67
—- Feel smarter after reading this blog?
Want to feel even smarter? Download and play our games! You can run around in our virtual world while reviewing your basic math skills. If you are too busy (seriously?) you can still give a game as a gift or donate a game to a classroom or school.
Some problems that seem really complex are quite simple when you look at them in the right way. Take this one, for example:
My hypothesis is that a major problem in math achievement is persistence. Students just give up at the first sign of trouble. I have three different data sets with student data from the Spirit Lake game. Many of the students in the student table are the control group, so they will have no data on game play. There is a table of answers to the math challenges and another table with answers to quizzes which students took only if they missed a math challenge. When students miss a math challenge in the game, depending on which educational resource they choose, they may do one of two or three different quizzes to get back into the game. Also, some of the quiz records were not from quizzes actually in the game but from supplemental activities we provided. So, how do I identify where in the process students drop out and present in a simple graphic to discuss with schools? Just to complicate matters, the username was different lengths in the different datasets and the variable for timestamp also had different names.
It turns out, the problem was not that difficult.
- Merge the student table with the answers (math challenges) and only include those students with at least one answer.
- Merge the student table with the quizzes and only include those students with at least one quiz
- Concatenate the data sets from steps 1 & 2
- Create a new userid variable and set it equal to the username
- Create a new “entered” variable and set it equal to whichever of the datetime fields exists on that record
- Delete the quizzes not included in the game.
- Sort the dataset by userid and the date and time entered.
- Keep the last record for each userid. Now you have their last date of activity.
- If there is a value for the math challenge field then that is the name of the last activity, otherwise the quiz name is the name for the last activity.
- Use a PROC FORMAT to assign each activity a value equal to the step in the game.
- Do a PROC FREQ using that format and the order = FORMATTED option.
Once I had the frequencies, I just put them into a table in a word document and shaded the columns to match the percentage. There may be a way in SAS/Graph or something else to do this automatically, but honestly, the table took me two minutes once I had the data.
I think it illustrates my points pretty clearly, which are:
- A sizable number of students drop out after the second problem.
- 25% of the students drop after the first difficulty they have (missing the second problem)
- Only a minority of students persist all the way to the end, less than 25% of the total sample
This isn’t based on a tiny sample, either. The data above represent a sample of 397 students.
In case you would like to see it, the code for steps 3-11 is below. Particularly useful is the PROC FORMAT. Notice that you can have multiple values have the same format, which was important here because players can take multiple paths that are still the same step in the sequence.
data persist ;
attrib userid length= $49 ;
set mydata2.sl_answers mydata2.sl_quizzes ;
entered = max(date_answered_dt,date_taken_dt) ;
**** DELETES QUIZZES IN EXTRA AND SUMMER SITE, NOT IN MAIN GAME ;
if quiztype in (“problemsolve”,”divide1long”,”multiplyby23″) then delete ;
userid = new_username ;
format entered datetime20. ;
proc sort data=persist ;
by quiztype ;
proc sort data=persist ;
by userid entered ;
data retention ;
set persist ;
by userid ;
if last.userid ;
attrib last_activity length= $14 ;
if inputform ne “” then last_activity = inputform ;
else last_activity = quiztype ;
proc freq data= retention ;
tables last_activity ;
proc format ;
“findcepansi” = “01”
“x2x9” = “02”
“math2x” = “02”
“math2_2” = “02”
“wolves1a” = “02”
“multiplyby5” = “03”
“multiplyby4” = “03”
“multiplyby3” = “04”
“wolves1b” = “05”
…. AND SO ON ….
“horseform2” = “21”
ods rtf file = “C:\Users\Spirit Lake\phaseII\pipeline.rtf” ;
proc freq data= retention order=formatted ;
tables last_activity ;
format last_activity $activity. ;
ods rtf close ;
Many years ago, I was walking through the exhibits at the county fair with my late husband (he was alive then, that’s why he was able to walk with me) and I lamented,
Look at those quilts. My grandmother makes quilts. Look at those crocheted tablecloths. My other grandmother crochets. Look at me – what do I make?
My wonderful husband turned to me and said in his good-old-boy, country accent,
Money. That’s what you make that your grandmothers didn’t make. You make money, darlin’.
Everyone is posting pictures of the cute Halloween costumes their mom made for them or that they made for their children. I never made a Halloween costume in my life, but here is a copy of some code I finished last weekend that makes a graph with different types of pastries. Another function I wrote (not shown here) changes it from Spanish to English. If you get it correct, it takes you to another problem that does bar graphs with actual bars.
I didn’t make a costume but I did make money from working on this project which The Spoiled One can use to buy whatever costume she likes.
Funny how a random sight can jog a memory, like today when I was walking around the neighborhood, taking a break from the marathon push to get our newest game out the door.
It was November 11, 1985. I was about eight months pregnant, and about two months into my doctoral program at the University of California. I came home to a surprise – 11 dozen tropical flowers on my doorstep. I called my still relatively new husband at work. Nope, he hadn’t sent them.
It wasn’t my birthday. It wasn’t our anniversary. It was too early for Christmas.
A couple of days later, I got a call from my sister. She had sent them to commemorate the one-year anniversary of me winning the world championships. She couldn’t believe I hadn’t remembered.
In the year since, I had married, moved to a new city, gotten pregnant, quit my job as an engineer, started a new job as a middle school math teacher and started on my Ph.D.
That day, at the world championships, winning seemed the most important thing in life.
A few months before, I had been in Europe. I competed at the British Open and placed third. Then, I went to the Tournoi d’Orleans and placed fifth. Not only was it the only time I had represented my country and come home empty-handed, but I hurt my knee, again, in London and tore something in my thumb in France. These were not little injuries, either. I’d had surgery on that knee less than two months prior. By the time I was 50, I needed a total knee replacement. My thumb doesn’t really work. I’ve been putting off surgery on that for years because, I mean, who really needs two thumbs and I’m busy.
So, two career-ending injuries, a loss and in pain. I had a layover in St. Louis where I was supposed to meet up with my sister. I cried all the way across the Atlantic but thought, at least I’ll see my sister. I got to St. Louis, called her house and she wasn’t home. She’d forgotten I was coming. Cell phones were 20 years in the future. Did I mention that I was in the middle of getting divorced and in a custody fight?
I got back on the plane, flew to Los Angeles, couldn’t remember where I had parked two weeks ago, limped around the airport parking lot for half an hour carrying my luggage (roller bags weren’t a thing yet), finally, found my car and drove home. I’d lost, no one loved me and I didn’t know if I’d be able to compete ever again. It was the worst day of my life.
I hadn’t thought of that day in the past twenty-five years. You’d think it would make me depressed to remember that, but it actually made me smile at how naive I was in my twenties. There have certainly been worse days than that. I lost that custody battle – temporarily. My new husband died when I was in my thirties.
Now THAT should make me depressed, certainly. Oddly, it doesn’t.
What it all reminded me is that you get over things – or you should. My same sister laughed at me, in a friendly way, when I told her I was going to school to get a doctorate. She said,
You just accomplished something that would be most people’s goal for a lifetime and now you go and set another one.
That’s as it should be. As The Spoiled One brilliantly advised me one day when I was frustrated trying to solve some programming problem:
Life is long, Mom. Don’t worry, you’ll get there.
So, my thought for the day is this:
Whether you think today is the worst day of your life or the best day of your life, if you keep going, it will get better.
Feel smarter after reading this blog? Want to be even smarter? Check out what I do on my day job – adventure games that teach math and Native American history. Buy for yourself or donate for a child or school.
Your mileage may vary, your life may vary, but there are a few lessons worth learning . As a public service, I have decided to share with you things I thought I knew but was initially wrong about.
- Your children don’t actually consider you a person until they are nearly 30 (if ever).
Many years ago, when teaching adolescent psychology, I remember the textbook author saying that most people really don’t consider their parents as people until they are in their thirties. At the time, my children were very young, and I thought that was just a ludicrous statement. Now, I’m pretty sure that he was right. I can’t tell you the number of young adults I have seen treating their parents in ways that they would never treat anyone else. Doubt me? Consider an adult in his/ her twenties whose parent is paying part (or all) of their rent. I know plenty of people in this situation, some are employed but ‘want to live somewhere nicer’ than they can afford. Others want to ‘follow my dream’. Those same young adults would never expect their friend, boss or co-worker to pay their rent – because, well, why the hell would another person who doesn’t live with you support you? Maybe that other adult (your parent) has a dream to travel the world or open a knitting school or spend all their money on cheap women and expensive whiskey.
(For those who wonder if this is personal, I would like to note that all of my children are self-supporting except for the one in high school, and I have no interest in spending my money on cheap women – or knitting schools. I do like expensive whiskey.)
Speaking of personal, though, lesson number one was brought home to me recently when I was complaining to The Perfect Jennifer about something stupid and unimportant, like having to walk down to the bank to deposit a check. I apologized for rambling on and made a comment about being a complainer and she corrected me,
“No, Mom, really you’re not. You never complain about the things like having to take care of three kids by yourself after Dad died, while starting a company at the same time. I’ll bet that was hard. It sounds really hard.”
She was 29.
This is the first time any of my children actually acknowledged that it was difficult for me, too. I have to say, that made my day. It WAS hard. A friend of mine lost her father when she was in high school, and she was telling me that it wasn’t until she was in her forties and had her own children in high school that she thought about all of the things her husband does and how hard it would be to take care of everything without him.
My point isn’t that their father’s death wasn’t unbelievably hard on the children, nor that I expected them to feel sorry for me when they were little kids. It is simply this, as children, everyone sees their parents primarily in terms of fulfilling their own needs, basically in terms of how they can use them. Children see the world as centered around them.
We often don’t consider our parents as actual adult humans with their own feelings, aspirations, difficulties, strengths and weaknesses until we become adults ourselves. Sometimes, not even then.
—– Want to learn even more ?
Play yourself, buy it for your children (however they think about you!)
Let’s get this out right up front – I have no question that there is discrimination in the tech industry. I gave an hour-long talk on this very subject at MIT a couple of weeks ago, where I pointed out that everyone’s first draft of pretty much everything is crap – your first game, first database – and some people we give encouragement and other people we give up on.
That’s not my point here. My point is that sometimes we are our own barriers by not applying to positions. Let me give you two examples.
First, as I wrote on my 7 Generation Games blog earlier, we reject disproportionately more male applicants for positions but yet our last four hires have all been men. This may change with the current positions (read on to find out why).
For the six positions we have advertised over the last couple of years, the application pool has looked like this:
We had one woman apply for the previous internship position we advertised, and we ended up hiring a male. If you look at this table, the odds of a woman being hired – 1 in 3, are greater than the odds of a man being hired, 1 in 5.5 . Yet, we hired twice as many men as women.
Why is that? Because more men apply. More unqualified men apply, which explains our higher rejection rate. If we explicitly state, “Must work in office five days a week”, we will get men (but no women) applying who live in, say, Sweden, and want to know if maybe that is negotiable (no.)
We have also recently filled 3 positions, and will soon fill two more, without advertising. In one of those cases, the person (male) contacted us and convinced us that he could do great work. All four of the other positions were filled by personal contacts. We called people we knew who were knowledgeable in the field and asked for recommendations.
We happen to know a lot of people who are Hispanic and Native American, so 3 of those positions ended up going to extremely well-qualified people from those groups. The one woman we hired out of those five positions was actually recommended by my 82-year-old mother who said,
“Your cousin, Jean, is a graphic artist, you should check out her work.”
As you can see from the photo of the 6-foot banner she made for us, she does do good work.
I see two factors at work here:
- Women are less likely to nominate themselves. While men will apply even if their meeting the qualifications seems to be a stretch (or a delusion), women are less likely to do so. I don’t know why. Fear of rejection?
- People are recommended by their networks and women seem to be less plugged into those networks. This is also true of minorities. We make no special effort to recruit Hispanic or Native American employees but since that is a lot of who we know, it is a lot of who THEY know and hence a lot of our referrals.
How do you increase your proportion of female applicants? You are going to laugh at this because it is the simplest thing ever. This time around, I wrote a blog post and tweets that specifically encouraged females to apply. And it worked! Well, maybe you would have predicted that, but not me. I would never have guessed.
Do you really want to hire Latino graphic artists or software developers? Come to the next Latino Tech meetup. Bonus: the food is awesome.
My point, which you may have now despaired of me having, is that affirmative action is a good thing on both sides. By affirmative action I mean being pro-active. If you are from an under-represented group, APPLY. Invite yourself to the dance. If you are an employer, reach out. It could be as easy as having a margarita during Hispanic Heritage Month or writing a blog post.
In both cases, you might be surprised how little effort yields big results.
Don’t forget to buy our games and play them. Fun! Plus, they’ll make you smarter.
I quit keeping track of how much money all the grants I have written totaled, but when we were doing a lot of work together, Dr. Erich Longie had it at $30 million, and it’s been several years and probably another $4- 5 million since then.
If you are interested in obtaining grant money, let me give you a few tips from the trenches. These may seem like another memo from Captain Obvious, but believe me, I have met many people mistaken about each of these.
- The money you receive has a specific purpose. You are proposing a legal obligation between you and the federal government (or other organization). Do not believe for one minute that this is “free money”. It is free in the sense that you do not have to pay it back or give up equity in your company, but you DO have to do the work promised in that proposal. The reason I don’t have $30 million is that the money went to paying salaries, buying supplies and equipment, supporting travel to conferences.
- Apply to grant programs that fit your project. NEVER try to convince an agency that they should fund something outside of their area because they won’t do it. Do NOT try to convince the manager of a program for rural youth to fund your project in New York City because urban kids really need help, too.
- Read the instructions. Yes, all 60 or 100 pages of them. Specifics to look at right off the bat:
- Page length. Don’t exceed it or your application won’t get reviewed.
- Font size and margins. Yes, they are that picky. If you think you can get around the page length by using 8-point font, think again. Your application won’t get reviewed.
- Budget. If they give a range, your budget should be in it, not higher and not lower. If there is a maximum, your budget should be close to it but NOT over or (you guessed it) your grant won’t get reviewed. Don’t be too low, either. Think about it, if they are giving grants that average $500,000 per year and you request $45,000, you probably are not doing the same work, addressing the same objectives as they were expecting for ten times as much money.
4. Copy and paste the criteria from the instructions into a document. That way, you won’t forget any. That is your outline. It usually looks something like this:
- Need for project
- Review of Relevant Research
- Goals and Objectives of Project
- Plan of Work/ Research Design
- Key Personnel
- Facilities/ Adequacy of Resources
There may be some differences, but pretty much everyone is going to want to know why this needs to be done, what other work has been done in this area, what you hope to accomplish, how you aim to do it, how you will know if you did it, why you think you are qualified, where you are going to do it and how much it will cost. 5. Start early. I cannot emphasize this enough. Under absolutely no conditions will I write a grant without at least a month’s notice. A month is pushing it and that is only in that rare situation when I’m not very busy that month and someone is paying me bags of money to either work night and day or to arrange my schedule so I can move a lot of work into the next month. I just submitted a grant I started on in June. Now, I obviously didn’t work full-time on it for four months, but allowing enough time made it possible to get on the agenda for the tribal council meeting, for example, and have a resolution passed supporting it. Discussions with sites for data collection, beta testing and training all take time.
6. Speaking of time – understand that the deadline is fixed. If you are used to working with businesses on contracts or negotiating deals, then you may be in for a bit of a shock. If the deadline is October 6th at 4 pm Eastern Standard Time, that is what it is. If you submit your proposal at 4:02 pm it is not going to be accepted.
7. Do NOT try to submit your grant in the last hour. Maybe it will get through and maybe it won’t. The system is often clogged with other people trying to submit in the last hour and the funding agency has no sympathy with you because the application probably said in big bold letters not to wait until the last minute to submit.
I have a lot more tips, and some near misses, including heading people off at the post office and faxing to Hawaii to get the post mark five hours earlier. I’d write more about that but I need to get to sleep, which is one thing you do less of when writing a grant
Your basic math skills will get better, you’ll learn more about Native American history and you can take out your aggression by spearing a bear. (Please don’t write me animal rights people – it’s a virtual bear. If there are any pixel rights people, I guess you can post in the comments below.)