Where we left off, the reliability was unacceptably low for our measure to assess students knowledge of sad iconmultiplication, division and other third and fourth grade math standards. We were sad.

One person, whose picture I have replaced with the mother from our game, Spirit Lake, so she can remain anonymous, said to me:

But there is nothing we can do about it, right?I mean, how can you stop kids from guessing?

mother from game

This was the wrong question. What we know about the measure could be summarized as this:

  1. Students in many low-performing schools were even further below grade level than we or the staff in their districts had anticipated. This is known as new and useful knowledge, because it helps to develop appropriate educational technology for these students. (Thanks to USDA Small Business Innovation Research funds for enabling this research.) 
  2. Because students did not know many of the answers, they often guessed at the correct answer.
  3. Because the questions were multiple choice, usually A-D, the students had a 25% probability of getting the correct answer just by chance, interjecting a significant amount of error when nearly all of the students were just guessing on the more difficult items.
  4. Three-fourths of the test items were below the fifth-grade level. In other words, if you had only gotten correct the answers three years below your grade level, the average seventh-grader should have scored 75% – generally, a C.

There are actually two ways to address this and we did both of them. The first is to give the test to students who are more likely to know the answers so less guessing occurs. We did this, administering the test to an additional 376 students in low-performing schools in grades four through eight. While the test scores were significantly higher (Mean of 53% as opposed to mean of 37% for the younger students) they were still low. The larger sample had a much higher reliability of 87. Hopefully, you remember from your basic statistics that restriction of range attenuates  the correlation. By increasing the range of scores, we increased our reliability.

The second thing we did was remove the probability of guessing correctly by changing almost all of the multiple choice questions into open-ended ones. There were a few where this was not possible, such as which of four graphs shows students liked eggs more than bacon . We administered this test to 140 seventh-graders. The reliability, again was much higher: .86

However, did we really solve the problem? After all, these students also were more likely to know (or at least, think they knew, but that’s another blog) the answer. The mean went up from 37% to 46%. 

To see whether the change in item type was effective for lower performing students, we selected out a sub-sample of third and fourth-graders from the second wave of testing. With this sample, we were able to see that reliability did improve substantially from .57 to. 71 . However, when we removed four outliers (students who received a score of 0), reliability dropped back down to .47.

What does this tell us? Depressingly, and this is a subject for a whole bunch of posts, that a test at or near their stated ‘grade level’ is going to have a floor effect for the average student in a low-performing school. That is, most of the students are going to score near the bottom.

It also tells us that curriculum needs to start AT LEAST two or three years below the students’ ostensible grade level so that they can be taught the prerequisite math skills they don’t know. This, too, is the subject for a lot of blog posts. 

If you’re a teacher (or parent) and you’d like students to take the test for practice, you can see it here

—-

For schools who use our games, we provide automated scoring and data analysis. If you are one of those schools and you’d like a report generated for your school, just let us know. There is no additional charge.

Last post I wrote a little about local norms versus national norms and gave the example of how the best-performing student in the area can still be below grade level.

Today, I want to talk a little about tests. As I mentioned previously, when we conducted the pretest prior to student playing our game, Spirit Lake,  the average student scored 37% on a test of mathematics standards for grades 2-5. These were questions that required them to say, subtract one three-digit number from another or  multiply two one-digit numbers.

Originally, we had written our tests to model the state standardized tests which, at the time, were multiple choice. This ended up presenting quite a problem. Here is a bit of test theory for you. A test score is made up two parts – true score variance and error variance.

True score variance exists when Bob gets an answer right and Fred gets it wrong because Bob really knows more math (and the correct answer) compared to Fred.

Error variance occurs when, for some reason, Bob gets the answer right and Fred gets it wrong even though there really is no difference between the two. That is, the variance between Fred and Bob is an error.  (If you want to be picky about it, you would say it was actually the variance from the mean was an error, but just hush.)

How could this happen? Well, the most likely explanation is that Bob guessed and happened to get lucky. (It could happen for other reasons – Fred really knew the answer but misread the question, etc.)

If very little guessing occurs on  a test, or if guesses have very little chance of being correct, then you don’t have to worry too much.

However, the test we used initially had four multiple-choice items for each question. The odds of guessing correctly were 1 in 4, that is, 25%. Because students turned out to be substantially further below grade level than we had anticipated, they did a LOT of guessing. In fact, for several of the items, the percentage of correct responses was close to the 25% students would get from randomly guessing.

When we computed the internal consistency reliability coefficient (Cronbach alpha) which measures the degree to which items in a test correlate with one another, it was a measly .57. In case you are wondering, no, this is not good. It shows a relatively high degree of error variance. So, we were sad.

SAS CODE FOR COMPUTING ALPHA

PROC CORR DATA = mydataset NOCORR ALPHA ;

VAR item1 – item24 ;

 

The very simple code above will give you coefficient alpha as well as the descriptive statistics for each item. Since we very wisely scored our items 0 = wrong, 1= right a mean of say, .22 would indicate that only 22% of students answered an item correctly.

To find out how we fixed this, read the next post.

To buy our games or donate one to a school, click here. Evaluated and developed based on actual data. How about that? Learn fractions, multiplication , statistics – take your pick!

steam_icon-300x142Fish lake woman

I hate the concept of those books with titles like “something or other for dummies”  or “idiot’s guide to whatever” because of the implication that if you don’t know microbiology or how to create a bonsai tree of take out your own appendix you must be a moron. I once had a student ask me if there was a structural equation modeling for dummies book. I told her that if you are doing structural equation modeling you’re no dummy. I’m assuming you’re no dummy and I felt like doing some posts on standardized testing without the jargon.

I haven’t been blogging about data analysis and programming lately because I have been doing so much of it. One project I completed recently was analysis of data from a multi-year pilot of our game, Spirit Lake. 

Buffalo

Before playing the game, students took a test to assess their mathematics achievement. Initially, we created a test that modeled the state standardized tests administered during the previous year, which were multiple choice. We knew that students in the schools were performing below grade level but how far below surprised both us and the school personnel. A sample of 93 students in grades 4 and 5 took a test that measured math standard for grades 2 through 5. The mean score was 37%. The highest score was 63%.

Think about this for a minute in terms of local and national norms. The student , let’s call him Bob, who received a 63% was the highest among students from two different schools across multiple classes. (These were small, rural schools.) So, Bob would be the ‘smartest’ kid in the area. With a standard deviation of 13%, Bob scored two standard deviations above the mean.

Let’s look at it from a different perspective, though. Bob, a fifth-grader, took a test where three-fourths of the questions were at least a year, if not, two or three, below his current grade level, and barely achieved a passing score. Compared to his local norm, Bob is a frigging genius. Compared to national norms, he’s none too bright. I actually met Bob and he is a very intelligent boy, but when most of his class still doesn’t know their multiplication tables, it’s hard for the teacher to get time to teach Bob decimals, and really, why should she worry, he’s acing every test. Of course, the class tests are a year below what should be his grade level.

One advantage of standardized testing, is that if every student in your school or district is performing below grade level it allows you to recognize the severity of the problem and not think “Oh, Bob is doing great.”

He wouldn’t be the first student I knew who went from a ‘gifted’ program in one community to a remedial program when he moved to a new, more affluent school.

Get Fish Lake here (yes, that’s another game) before it is released on Steam next month! Learn fractions, canoe rapids, spear fish. Buy for yourself or donate to a school for under ten bucks!

steam_icon-300x142

When I was in my twenties, nearing the end of my competitive years, Dr. James Wooley dropped by the club to visit. If you aren’t into judo, you probably don’t recognize his name as a two-time Olympian. By the time I was competing on the international scene, he had retired from competition, married and was in private practice in Orange County.

I asked whether he missed competition and he shook his head,

“Oh, lord, no!”

(Did I mention he was from Texas?)

“It was great but now I’m finally finished with school, seeing patients, I have a wife and we’re looking to start a family. It was great but I don’t miss it at all.”

From the wisdom of my twenty-something years, I did not believe him for one second. At the time, winning was the most important thing in my life. I thought about it the second my eyes opened in the morning, as I dropped to the floor and did 50 push-ups and 50 sit-ups to start the day. I dreamed about winning. I thought Jimmy was just putting a good face on being old and depressed.

Fast forward a decade or so, the first time it was the end of April and I had not even realized the national championships were happening until they were over. That used to be part of the calendar of my life – start training in January for the Nationals, win those in April. Take a break. Win whatever was the summer event – U.S. Open, Panamerican Games. Take a break.

I retired from competition, married, had more kids, earned a Ph.D., started businesses. Jimmy was right – I didn’t miss it and life did not suck.

Now the kids are adults. I have to send the absentee ballot for the youngest express mail to Boston so she can vote. I’m on the fourth business. Life is good.

I’m closer to 60 than 50 now and if you had asked me to imagine that when I was in my thirties, I’m sure I would have thought it would be depressing.

I still teach judo but after several surgeries on my knees and one on my hand, I don’t do it nearly as well as I once did. I have wrinkles, grey hair and investors who don’t want to talk to me because we all know that innovative ideas are the monopoly of young people.

Let me tell you some of the things that DON’T suck about being old.
1. I don’t have to worry about whether I will have saved enough for retirement, gotten an education, been reasonably successful in my career, raised children who were decent people. The answer is, “All of the above”. Much of the anxiety I had as a younger person is gone because those questions have been answered.

2. I wear what’s comfortable and I don’t give a damn what anyone thinks. My feet don’t hurt from wearing high heels. I don’t walk around cold because wearing a sweater would cover up my girlish figure. Both of my daughters, when they got married, felt the need to tell me that jeans and a hoodie were not acceptable wedding attire.

3. I know a lot of things – from how to make banana bread from scratch to how to code a game in JavaScript to how to interpret results for ordinal logistic regression. All that ‘fake it until you make it’ bullshit is in the past. I’m not faking it. I really do know what it means when my estimates fail to converge or when the knife doesn’t come out clean. I don’t worry about anyone finding me out because there’s nothing to find.

4. I like my husband and he likes me. Yes, we’re both old and wrinkled and grey. He’s lost 50 pounds in the last year, which shows a pretty damn impressive display of will power. He’s brilliant, a great father and makes a good martini. He can help The Spoiled One with her calculus homework and our junior developers with their C# code. He’s not a jerk (women in tech realize having a brilliant guy who is not a jerk is worth something in a lot of ways).

5. Life is easier. One of the advantages of being around a long time is that people get to know you. When you are young, you need to submit proposals to speak at conferences, submit articles to journals, apply for jobs. As you get older, people ask you to work/ write/ speak for them because they know from your previous work that you probably aren’t going to suck. You don’t have to prove yourself because you already did. (Except in the Opposite World of Silicon Valley where education and experience aren’t valued – but that’s a post for another day.)

Mom and Aunt Sylvia

Mom and Aunt Sylvia

Sometimes, I look at my mom, or older friends of mine, and wonder what it is like to be retired, to not have your calendar filled six months, or even 6 days, in advance. I wonder whether it sucks to have nothing you have to do in the day, to have not only your kids but your grandkids safely launched .

I’m guessing that it’s probably just fine.
I’m not just sitting around getting older. I’m also making games. You can buy them here.

characters traveling on map

Or,

you can download a free app, Making Camp, for your iPad here

Wigwam

All of my life, I have been a woman in a “man’s field”. I was the first American to win the world judo championships back in 1984, one of the few women majoring in business at Washington University in St. Louis in the 1970s. I had a professor tell me and the two other women in his class that we were ‘taking a spot that was needed by some man who would have to support a family’.

I was an industrial engineer at an aerospace company in the early 1980s, where there were so few women on the factory floor that it made for an interesting pregnancy as I was always trying to find where the heck was the women’s bathroom and all of my co-workers, being male, had no idea.

I started a company – The Julia Group – that did customized software development, creating databases, statistical analyses for on-going evaluations.

I started another company – 7 Generation Games – that makes educational video games that teach math, social studies and language.

You know what all of these have in common? At every single level, I was subjected to standards different from men. I won the world championships and came home to have people say, “Oh, you think you know judo? Do you know this technique?”

I WON THE FUCKING WORLD CHAMPIONSHIPS! YES, I KNOW JUDO!

When I applied for my first engineering job, I was asked if I had a masters degree in engineering. I did not. I had an MBA. However, I could program in the languages required, had the math and statistics courses stated in the job requirement, oh, and did I mention that the man I replaced in that job didn’t have a masters degree in engineering either nor did any of the men in my department?

How did I get the job? Well, there were some relatively esoteric languages they used, and I knew that, so I learned those on my own time and then when an opening came up and they needed someone right away, I was there. Unlike the man I replaced, who was hired on his potential and learned the languages on the job, I had to prove I could do it before they hired me.

I have run into these attempts at disqualification at every turn.
“It’s not that we won’t hire a woman but …”

Do you have a degree? Master? Ph.D. ?
Um, well, you do, um  , did you have a year of calculus, at least 4 years of statistics, publish articles in academic journals?
You did? Oh, well, did you present at scientific conferences? How about software conferences?
Yes, well, we see you started a company but do you have a product? Paying customers? Investors?

When I answer yes to all of that – well, is it available on Chrome, Windows, Mac, Android, in the App Store?

You see where I’m going with this. After a life time of being subjected to standards that don’t apply to the men around me, I find the experience of Hillary Clinton oh so familiar.

I never voted for Hillary Clinton before but I’m going to do it now.

Seriously, if you went through every email I ever wrote, every action of mine and you couldn’t find anything to nail me on so you are now going through the emails of my associates hoping to find something, you are pretty damn desperate to disqualify me.

Let’s be honest for a minute, shall we? We all know that these allegations against Clinton are pretty much bullshit. She deleted 30,000 emails? So fucking what? I delete 30,000 emails A MONTH and I’m pretty sure the Secretary of State gets a lot more emails than me.

Someone on her team said something not nice about Bernie Sanders? They discussed methods to beat Sanders and Trump?

That’s shocking? As Bernie Sanders, who is one of the few lights in a dark political year has said, “I bet if you looked through my staff emails you’d find some unkind things said about Hillary Clinton.”

Let’s address Benghazi. People died in Benghazi and that is an undeniable tragedy. People in our military and embassies have died throughout history and it is always a tragedy for their families. Why is this one instance different from all the others? Because no one ever made mistakes before?

No, it is because Hillary Clinton is a woman and there is a substantial minority in this country, male and (shockingly) female, who resent women who refuse to accept ‘their place’.

The venom spewed at Hillary Clinton is familiar to me. I have had people HATE me and I would wonder, “Why? What the heck have I ever done to you?”

What I have done is defy their prejudices, that they are ‘better’ because they are men, even if they are pretty mediocre at what they do. That they were right to swallow their own ambitions because ‘women can’t do things like that’.

Honestly, if someone went through every one of your private emails, every private paper of yours, reviewed every one of your actions for the past 30 or 50 years, then went through all of the correspondence of every one you knew and THEN they only selected out whatever was most negative, how would YOU look?

I swear. I’ve bailed people out of jail. I’ve applied for grants I didn’t get. I’ve had prototypes that were pretty buggy. Like most people, I think I’d come out looking pretty damn awful.

Be honest, it’s true for you, too.

If you have taken a microscope to Hillary Clinton for the past few decades and you have to resort to now scrutinizing everyone she ever did business with in your desperation to find an excuse not to give her the job, she must be pretty damn good.

I’m tired of the witch hunt.

Check out our latest game. You can download Making Camp free for your iPad. Play with your children, hand them your iPad to do something productive or take a little break yourself (you deserve it)

Wigwam

Occasionally, when I am teaching about a topic like repeated measures Analysis of Variance, a brave student will raise a hand and ask,

Seriously, professor, WHEN will I ever use this?

The aspiring director of a library, clinic, afterschool program, etc. does not see how statistics apply to conducting an outreach campaign or HIV screening or running a recreational program for youth – or whatever one of hundreds of other good causes that students intend to pursue with their graduate degrees. Honestly, they often look at the required research methods and statistics courses is a waste of time mandated for some unknown reason by the University, probably to keep professors employed. Often, they will find a way to do a dissertation using only qualitative analysis and never think about statistics again.

This is a huge mistake.

For all of those people who say, “I never used statistics in my career”, I would answer, “well, I never used French in my career either and you know why – because I never learned it very well.”

Now, those people who don’t see a real use for French probably aren’t convinced. However, to me, it’s pretty evident that if I could speak French I could be making games in both French and English.

Actually, statistics can answer the very most important question in any social program – does it work?

So, I had written a couple of blogs about the presentation I gave at SACNAS (Society for the Advancement of Chicanos and Native Americans in Science) where I discussed using statistics to identify need for intervention and mathematics for students prior to middle school. I also gave examples of teaching statistics concepts in games.

The question is, did these games work for increasing student scores?

For this – surprise! Surprise! Drumroll – – – we used repeated measures Analysis of Variance. If you look at the graph below you can see that the students who played the games improved substantially more from pretest to posttest than the students in the control group.

graph showing increase from pretest to posttest

This was a relatively small sample, because it was our first pilot study, and conducted in two small rural schools, that also happen to have very high rates of mobility and absenteeism, so we were only able to obtain complete data from 58 students.

Now, the results look impressive but where these differences higher than one would expect by chance with four groups (two grades from each school) of a fairly small size?

Well, when we look at the ANOVA results we see that the time by school interaction, which tests if one school changed more overtime than the other is quite significant (F = 7.13, P = .01). Yes, the P value equaled exactly .0100.

The time by school by grade 3 – way interaction was not significant. It’s worth noting that the fifth grade at the intervention school had less time playing the game due to logistical reasons – they had to schedule the computer lab as opposed to playing in their classroom, and sometimes, their class being scheduled later in the day, they missed playing the game altogether when school was let out early due to weather.

One way that I could reanalyze these data – and I will – would be to look at it not by grade but by time spent playing. So, instead of four groups, I would have three – those who played the game not at all, in other words, the control group, those who played at less than recommended and those who played it the recommended amount.

My point is that repeated measures ANOVA is just one of the many statistical techniques that can answer the most important questions in social programs – whether something works and under what conditions it works best. There’s also the question of who it works best for – and statistics can answer that too.

So, my answer to the student who questions if he or she will ever use this is, “if you’re smart you will.”

banner

Check out the games from which these data were collected. They are effective now and we are in the middle of a major update. Did I mention that you get free updates for life with all of our games?

For all of those who have asked us if these data are going to be published, the answer is yes, we have two articles in press that should come out in 2017.

We are working on more in our copious spare time that we do not have, but right now we are focusing on game updates.

 and on our new, free iPad game, Making Camp

wigwam button

Gather around the fire, young and old. By my observations in social media, TV and just eavesdropping on some of you youngsters, hipsters and – I don’t know what the hell to call you people hollering over there on the corner – there are some facts and experiences with which you are unfamiliar.

fire

Let’s start with ‘inner cities’ .

This week, I dropped someone off after a late meeting and ended up driving through Watts (known as south Los Angeles to some people – you know who you are) after 9 pm at night. It was dark, not because it is always dark in the ‘inner city’ but because it was night.

I stopped at a fast food place where there were a lot of men hanging around outside. All of the men were African-American or Latino. I ordered a burrito and gave the woman at the window ten bucks.

Here is what happened next ….

… She gave me the correct change and an enormous, delicious carne asada burrito. I drove away, eating my burrito, which was so gigantic my husband ate the other half when I got home.

What did you expect? I expected if there were that many people waiting for their order at that hour of night the food would be good, and I was correct.

Incidentally, on my way to dropping off my colleague at her house – yes, she has a degree, a professional job and lives in the city – we drove through downtown and were commenting on how much we both love the skyline and the main branch of the Los Angeles Public Library and wish we had time to go there more often. I mentioned that many of my co-workers when I worked at USC lived downtown and took the train in to work.

library

Inside the Los Angeles Public Library Main Branch

I’ve spent a whole lot of hours in my life in cities, mostly St. Louis, Tokyo, Minneapolis, Riverside (does that count as a city?), San Diego and Los Angeles. I’ve had a gun pulled on me three times in my life, two of those times by people I knew (yes, I should have a better choice of acquaintances ). Twice it happened when I was in a house in a rural area (different states, 20 years apart) and once when I was walking in the suburbs of St. Louis to visit a friend.

My point – and I do have one – is that the cities are not “hell holes” where African-American and Latino voters have it so bad that they have “nothing to lose” as a certain presidential candidate and his followers have characterized them. Yes, people do get shot in Los Angeles, including some people I have known, but people are not hunkering down in their houses, only leaving to replenish their supplies of food and bandages in some type of Mad Max scene as they barrel through the streets dodging bullets.

There are certainly problems, starting with inadequate funding for schools that desperately need maintenance, a lack of after school and recreational programs , not enough parks. There are also people sitting outside eating delicious burritos.

When I’m not eating burritos, I’m making games. You can buy them here.

characters traveling on map

Or,

you can download a free app, Making Camp, for your iPad here

Wigwam

Choke and arm bar

Yes, I am choking a blind lady but she’s a black belt and on the US Paralympic team so I don’t feel sorry for her

I will be the first to admit that I’m not the warm fuzzy type. Maybe you’re like me, you’d like to do good for your community but you just can’t see yourself as a physician.

Maybe your bedside manner is to snap at someone to quit being a whiner.

Or maybe you really are a sweet kind person but you are not very extroverted. You just can’t see yourself looking someone in the eye and asking them to tell you about their problems at home. Perhaps you really genuinely care about children in your community and really would like to help them succeed in school but the thought of speaking in front of the 30 people makes you break out into a cold sweat – even if the 30 people are all under 13 years old.

Maybe, like me, you really like math. To be specific, maybe you really like analyzing data, looking for correlations, inspecting distributions. Maybe, you really like programming. Or that’s what we called it in my day – now all the cool kids call it coding.

Does that mean that we are condemned to be a bunch of Silicon Valley dwelling, Soylent swigging, soulless drones with nothing to keep us warm at night but our stock options? In fact, quite the opposite! These last few years I have been having a lot of fun working with statistics in two very different ways.

First of all, I’ve been working with our team at 7 Generation Games to make adventure games that teach statistical concepts.

Let me give you an example. Some items are more valuable than others. Why? Try to figure  it out by looking at this distribution.

graphs showing frequency of different goods

Players can click on this interactive graph for help reading it. They have a sentence written with blanks to fill in to model academic language.

Once a student answers one or two questions in the game correctly, the reward is being able to play a related game – in this case, collecting items in the jungle. As you might guess, the more common items are worth less in the game.

jungle game

Here is a second example. Below, we have a section of our 3D game where the player is building a pyramid.

pyramid

To build your pyramid fast enough that the Emperor doesn’t decide to chop off your head, you want to get stronger than average workers. What is an easy way to determine if you have stronger than average workers? Find the median!

explanation on finding median

Players can also click a button to switch the page to an explanation in Spanish.

explanation of median in Spanish

Just because we were all out last night at the Latino Tech meet-up to celebrate Hispanic Heritage Month, don’t assume everything we make it is focused on Latinos.

our staff

Here is yet another example of teaching statistics in a game, this one re-tracing the Ojibwe migration.

question on averages from Forgotten trail

In this case, the player computes an average to figure how many miles need to be walked per day to get to the end of the trail in eight days.

Get this question right and you can play the next level, where you canoe down the river to meet up with your old uncle who will – surprise – pose another statistics problem before you can move on to the next level.

 Girl in canoe

So, there you have it! You can apply your knowledge of statistics to create adventure video games that teach students. As you can see, you also can apply knowledge of programming to meet the special needs of students whether it is to have a page read to them (did you notice the read it to me button in the page above?) Or to have it translated into a second language.

I’ll bet that you thought I was going to talk about using statistics to evaluate whether the games worked. That, is a post for another day.

_______

You can buy Forgotten Trail now. Only $4.99 . Yep, under five bucks. Runs on Mac, Windows or Chromebook.

Buy our games

burning carTo be honest, when I first began studying statistics social justice never entered the equation. Like most people in America, I think, I was concerned about problems like crime, poverty, low educational attainment of minority groups. Like most people, my concern didn’t translate into much actual effort on my part.

No, I took my first statistics course because it seemed really interesting. I made a C+ in it because it was Monday, Wednesday and Friday afternoons and the fraternity parties started on Friday afternoon, so I missed every third class. When later research of mine showed a negative correlation between absenteeism and grades I was not surprised. I had personal experience. I mentioned this too, because I have seen too many women and minority students discouraged from science and technical fields when they were not at the top of the class right away.

It’s a very long journey for my first statistics class in 1978 to now. Although I started learning statistics because I was just very interested in what I would call “messing around with data” and I like programming a lot, along the way I learned something interesting.

A lot of money is allocated based on statistics. Maybe not directly so, not very often does someone say to you,

“That’s a very interesting statistic. Have $9 million!”

Statistics do come into play. About 1/4 of a century ago, I realized that grant money often did not go to the program where the funds were needed most or the staff were most effective. No, they went to the programs that would best at writing grant proposals. These proposals included statistics on needs assessment and evaluation of prior efforts. Often, people who were really good at helping low income students raise their academic achievement or getting people with substance abuse disorders off of drugs were nowhere near as good at writing grant proposals.

For all of those proposals, statistics were required. What proportion of students in the target schools are achieving below grade level? What was the distribution of test scores of students in the previous three years and how does that compare to the state or national average? What evidence is there that the proposed program for academic enrichment will have any impact on the students at all?

Often, my very well meaning colleagues disagreed with the necessity for this type of analysis, even while they appreciated me doing it and made good use of the grant funds. Their point of view was that they knew what worked in their classrooms or clinics.

Personally, I feel that if all I had done in my career was bring tens of millions of dollars in grant money to programs that apply those funds to do good in their communities that would have been a satisfactory accomplishment. However, I’d like to argue that I did a little bit more good than that because I disagree with some of my esteemed colleagues that, “I know it when I see it“,  is adequate for determining program effectiveness.

I can give you many many reasons why statistics are essential. First of all, something I have seen over and over in my career is that what gets measured gets done. If you are measuring the number of tutoring sessions or the number of times students play your games or the duration of those sessions, that allows you to correlate the “dose” of treatment your students received with the “response” in terms of increased achievement.

Many times, I have seen programs that were initially judged ineffective because everyone who came through the door was lumped together whether they were seen 10 times, once or not at all, having left before they ever saw a tutor counselor or whatever. Tracking your interaction with people allows you to determine whether you are effective for people who spend some substantial amount of time with your program. It also lets you tell what percentage of the people fall through the cracks that is who come in, fill out a form to be part of your program and then drop out almost immediately.

In brief, effective application of statistics cannot only help you obtain money but also see that money from federal agencies, foundations, etc. is intelligently applied.

If you are interested, I will be speaking at the Society For the Advancement Of Chicanos And Native Americans In Science annual conference in Long Beach on  Saturday , Discovery and Societal Impact with Statistical Science. You can come to hear much more on this topic (or just read my next blog post).

———

Check out our latest game we will soon be using to collect data. You can download Making Camp free for your iPad. Play with your children, hand them your iPad to do something productive or take a little break yourself (you deserve it)

Wigwam

I’ve decided on my new career after we get 7 Generation Games into the hands of millions of players. I’m going to take some time off and become a travel blogger.  I’m not going to be your typical travel blogger though. You won’t be reading posts from me about backpacking because that involves sleeping in the dirt and not taking showers both of which, as The Invisible Developer frequently tells me, show great  lack of appreciation for  our ancestors hard work in creating a civilization where we have houses and running water.

Dirt

My bed is more comfortable than this

I’m also not going to be one of those people writing about exotic, fantastically expensive vacations all around the world. For one reason, those vacations generally entail 20 hours or more of flying, which is only considered a good time if you’re really, really into masochism.

Nope, Random Travel Blog is going to be just like this blog where I basically ramble on about whatever the hell I feel like that day. In practice for that, I thought today I would talk about what you got for a roughly 972% increase in hotel costs. Just randomly, I happen to have stayed in a lot of hotels lately that ranged in price from under $60 to nearly $600 a night. Those at both ends of the spectrum were less and more, respectively, than I would have chosen to pay but happened to be the only options available in the cities where I needed to be at the time. Having a lot of experience staying in hotels across the spectrum I got to thinking about what you get when you  pay one-third as much as usual or 3 times as much and whether it is worth it. So, here are my random observations.

 

If a hotel is really expensive you can bet  that the location has something to recommend it. It may be a beautiful place or in a place where you need to be to do business. I recently stayed in a hotel in San Jose and the area wasn’t particularly beautiful but it was close to where it needed to be early in the morning.

 

The inverse isn’t always true. For example, here is the view a few steps from the hotel I recently stayed at in Pierre, South Dakota.

River with trees along the bank It was actually really beautiful. However, my guess is that Pierre  is not a major travel destination. The hotel was very cheap.

If you are staying at a very expensive hotel they will have room service and the food will be good. Is that worth an extra 400 or $500? Also, of course, you’re paying $50 or $100 for the food on top of the hotel room cost. Well, since I like to work late and I eat food, being able to get room service late at night is worth something to me.

Interestingly, it’s the cheaper  hotels that tend to offer free breakfast. Is that worth anything to you? It’s not worth much to me because I don’t usually get up early enough to eat breakfast and the cheaper the hotel is the worse the breakfast, so I often wouldn’t eat it even if I did get up early.

A really expensive hotel will have good service. People at the front desk won’t be rude. Your room will be clean. If you ask for anything within reason, whether it is to have someone bring you a toothbrush you forgot or extra peanuts for the minibar they will bring it to you, and pretty quickly too. Expensive hotels have good security. You won’t get robbed or sexually harassed.

Here’s a funny thing that I have found – the things that I want most are not correlated with price – except for the not getting robbed part.

For example, I want coffee in the morning. All the cheapest hotels where I have ever stayed had a coffee maker. Some of the really expensive hotels had really nice coffeemakers and really nice coffee and others did not because they expected you to use the restaurant order coffee from room service. I find that very annoying.

Another thing that I personally need to have is good Internet access. Cheaper hotels tend to provide pre-free Internet access and it is often not terrible. Expensive hotels often charge for Internet access, I guess because they figure if you’re staying there you can afford an extra $15 or $25 a day to use the Internet. Whether it is free or paid, I’ve generally found that my personal hotspot on my phone is faster – and yes, that includes paying for the “premium” high-speed Internet in hotels.

So, am I saying that you should stay the cheapest hotel that you can find? No. Often, really cheap hotels are in really bad parts of town where you don’t want to hang out. Someone suggested to me that at our last meeting where we had to stay at a hotel that was very expensive it would have been cheaper if we got a rental car and drove into our meetings.  I disagree with that there were 2 of us who would have spent an extra 2 hours each in traffic every day. Often, whether it is flights with 3 stops or hotel rooms an hour from your destination these things are only good deals if your time has no value.

If you stay at a nicer hotel you are pretty much guaranteed good service where the cheaper ones you take your chance. This might be why you find older people more likely to stay at the higher end hotels because they need more assistance with bags or getting around the city. Also older people have more money.

Personally, I like to have a nice view and I don’t mind paying extra to have a view of the city or the Bay or whatever. Sometimes, though I’m getting in somewhere after midnight and have to be up before 9 AM to get to a meeting.   In that case, I may as well stay the cheapest hotel there is – and it’s pretty certain to have coffee.

Based on a whole lot of experience, I see a very large difference between most hotels that cost under $100 and hotels that cost $200 and $300 I don’t see a lot of difference between hotels that cost $300 and those that cost $600 other than the latter tend to be in higher-priced cities or during events when prices get jacked up.

In short, if you have spent every one of your vacations staying at the Hilton or the Marriott and you are wondering if you are really missing out not staying at some really high-end chic location, I think you can quit worrying.

Next Page →