May

5

“If everyone knows a thing it’s almost for sure it aint so.”

“It’s not so much the things you don’t know that hurt you as the things you know for sure that aint so.”

I don’t take anyone’s word for anything. Take those quotes, for example, which I’ve both heard attributed to Mark Twain, Will Rogers and several others, ironically by people who were just certain they were correct.

One thing everyone knows is that Americans suck in math. We are so far behind Asia, we are continually told, that we are soon all going to be learning how to say, “Would you like fries with that?” in Chinese.

There was an article in the Los Angeles Times today that profiled a mother who had an Excel spreadsheet with a schedule for her child from 8:00 a.m. to 11 p.m. seven days a week. She said she started in kindergarten, because life is hard and students need to learn to deal with it. Her son, as a tenth grader, scored a perfect 800 on the SAT. Rather than convincing me further that we suck at math, it made me question the goal of propelling a child  to perfect scores.

One thing writing a dissertation on intelligence testing taught me is that test scores are very, very far from absolute and objective. Two critical points to keep in mind:

1. Some group of people decide what is tested, inevitably the group of people that has the most power. If we insisted that being fluent in more than one language is a factor in achievement scores, Hispanic children would be getting admitted to elite institutions in droves. Before you discard this as a silly notion, think about the arguments made for including high math scores – these are relevant to courses students take, to careers. An argument could be made for functioning in a global market place, for the ability to read texts in the original Spanish (or whatever second language a student reads).   I could write a whole dissertation on this – oh wait, I did ! – the point is we make decisions about what goes into the tests and those decisions favor some people and not others.

2. The scores we use to evaluate both at an individual and larger (school, country) level are almost never how many questions were answered correctly, which you might logically think is your test score. There you go with the logic again. Cut it out. In fact, scores depart several steps from the number of correct answers. First, there is the issue of partial credit, yes or no and if yes, for what. Second, there is the step of standardizing scores. Usually this means setting the average at some arbitrary value, say 100.  If the average student gets 17 questions right, then that is set as a score of 100.  The standard deviation, the average amount by which people differ from the average, is also set at an arbitrary value, say 10.  (If you’re not familiar with these ideas, think of your family. We’re kind of short in my family, and if you went to a family re-union you’d probably find that the average woman is around 5’3″ give or take two inches. So, you can think of five feet three inches as being the average and two inches as being the standard deviation. If you are reading this and from a country on the metric system, 5’3″ is equal to a furlong plus a bushel, a peck and a hug around the neck.) To return to my long-forgotten point – if 84% of the people score 22 points or lower, than answering 22 questions correctly is given a score of 110. (The mean of 100 +  one standard deviation of 10).  The scores you see reported aren’t that closely related to the number of questions answered correctly and they tell you almost NOTHING about what precisely people do or do not know.

I think most statisticians know this. I am certain that nearly everyone who does analyses of educational tests knows this. But I am equally certain that the average person reading the newspaper does not. This is important because it has to do with our sucking or not.

My assumption, based on what I read in the papers and hear on TV is that American kids just don’t know basic math. So, I downloaded the TIMSS (Trends in International Mathematics and Science Study ) data and I also downloaded the items that had been released, to see what it is that American kids do and do not know. Here are a few examples:

Students were shown a  rectangle divided into twelve squares. Five of those twelve squares were shaded. Then, they were given five choices of circles that were partly shaded and asked:

“Which circle has approximately the same area shaded as the rectangle above?”

To solve this problem you need to figure that the rectangle has 5/12 shaded and understand that 5/12 is a little less than one-half. (The figures show a circle that is 7/8 shaded, 3/4, exactly one-half, a little more than one-half and a little less than one-half.)

This question was answered correctly by 80.2% of American eighth-graders.

The next question asked :

A gardener mixes 4.45 kilograms of rye grass with 2.735 kilograms of clover seed to make a mix for sowing a lawn area. How many kilograms of the lawn mix does he now have?

This question was answered correctly by 71.9% of American eighth-graders.

I must admit that I was surprised the figure was that low, although not extremely surprised, since I know many, many adults and some young kids who never do math like this. Every phone, every computer has a calculator on it and they just think this is a useless skill, like cursive. I happen to disagree and the world’s most spoiled thirteen-year-old is not allowed to use a calculator to do or check her math homework.

Another question dealt with inequalities:

X/3 > 8 is equivalent to….

To get this answer correct, you need to understand the idea of inequality and how to solve an equation with one unknown. Essentially, you need to reason something like 24/3 = 8 so X > 24 . This, of course, presupposes you also know that 24/3 = 8.

This question was answered correctly by 42.8% of American eighth-graders.

A question that was answered by even fewer was:

What is the perimeter of a square whose area is 100 meters?

To answer this you need to know:

This question was answered correctly by 26.5% of American eighth-graders.

One last question,

A bowl contains 36 colored beads all of the same size, some blue, some green, some red and the rest yellow. A bead is drawn from the bowl without looking. The probability that it is blue is 4/9. How many blue beads are in the bowl?

This question was answered correctly by 49.4% of American eighth-graders.

Are these percentages bad or good? Honestly, I thought the questions were pretty easy and I was surprised by the low percentages on some of them – but I do math for a living and I was in 8th grade almost forty years ago. So, I have known this stuff a very, very long time. I THINK some of the questions were actually what was taught in ninth or tenth grade when I was riding a brontosaurus to school, so the fact that eighth graders today don’t know this information doesn’t convince me we’re all a bunch of drooling idiots.

Here is a blasphemous question for you –  Does it matter if you know the answers in eighth grade? I’m serious. Is it worth having your child study from 8 a.m. to 11 p.m. so that he or she knows all of this in the eighth grade instead of the ninth grade?

A few weeks ago, I was looking for data for a proposal I was writing and came across a state Department of Education website that had a note on its pages on test scores that said proficiency meant something different according to the federal government definition and that many people could function perfectly fine will being scored below proficient in math.

At the time I dismissed this as an excuse for poor performance. Today, when I looked at the questions and the results, I was not so sure. My two older daughters are a journalist and a history teacher. Both have degrees from good institutions (NYU and USC).  I believe neither of them could answer the question about finding the perimeter of a square with an area of 100. Perhaps they could have answered it when they took their SATs or while they were taking the one mathematics course they took as undergraduates. I’m not sure. I’m fairly certain if they ever knew this information, they’ve totally forgotten it. The truth is, as much as I hate to admit it, that neither of them at any point in their lives will feel the lack of this knowledge.

On the other hand, my daughter who knocks people down for a living (she competes professionally in mixed martial arts) could almost certainly answer these questions off the top of her head, just because she likes math and has always been good at it.

What percentage of Americans (eighth-graders or not) SHOULD be able to answer these questions?

I have no idea what the answer to that is.

Some people would say 100%,  because they need to know this information to do well on the tests to get into a good college.  I’m not sure that is true. More and more, people are asking WHY you need to do well on the tests. If I want to be a sportswriter or a history teacher or a doctor, what good does it do me to be able to calculate the perimeter of a square given the area?

I think the mother in San Marino may be part of an education bubble that will burst just like the housing bubble has. I am far from the only person to be suggesting this. Not only has the cost of higher education reached astronomical levels where it exceeds the cost of a home in most parts of the country, but it also, for selective institutions, is costing more of your life. Not only are fewer people going to be able to pay it, but, perhaps like the housing bubble, more people are going to say, “This isn’t worth it.”

I did not work from 8 a.m. to 11 p.m. I spent several hours today reviewing grants. Then I went running down to the beach, because it was a beautiful day. I had a Corona while reading the LA Times. I analyzed the TIMSS data and I watched The Daily Show. I also checked my daughter’s math homework and pointed out the one answer she had incorrect. She figured it out and fixed it on her own.

Life is not hard. Life is good.

Apr

25

I was disappointed to see that the Open Data community is pretty inactive over at data.gov. With 305,000 datasets and counting released you’d think there’d be more than a handful of people posting over there.  I decided I would start on my own with the TIMSS data. This is the Trends in International Mathematics and Science Survey. Props to them for releasing their data along with their programs, codebooks and publications. It takes some nerve to open up your work to the public and let other people have the freedom to scrutinize it and possibly criticize it.

First, I downloaded three folders containing 20 files – six text data files, six codebooks and eight SAS programs and other documentation.

No one asked my opinion on this – but that’s never stopped me before. Presumably one benefit the government is hoping to obtain by releasing its data is to get feedback – crowd-sourcing, differing perspectives.

Here are a few observations on the TIMSS data. These are not saying there was anything wrong with the way the analyses were done but simply that different people, me, for example, might have different interests.

In their analyses, they seem to have an interest in whether a question was answered incorrectly or just not answered either because the student skipped that item, and went on to others, or because he or she didn’t make it that far in the test, presumably because time ran out. Personally, I am interested in whether the student got it right or wrong. I’m assuming if the student skipped it, the reason was probably that she didn’t know the answer. Deleting out the formats that specified whether the student answered, omitted or skipped saved me thousands of lines.

I created an array of all variables that had a 998 or 999 and changed those to missing.

Originally, for some reason I could not open the file and view it in SAS. I was wondering if the data viewer had a limit to the number of variables, but now I am just thinking at the time I had too many applications running at once and there wasn’t enough memory available, because now it opens up fine.

My first ARRAY for re-coding included all of the variables from the first mathematics item  to the end, BSREA05 .

I realized when I started running the means to check everything that this isn’t what I wanted because that ends up changing all of the subscale scores, gender and age to 0 or 1. I only want the actual test questions coded that way. So, I changed it to:

array rec{*}  M022043 -- S042164 ;

Yes, this recodes all of the science items, too, which I ended up dropping but the math and science items were interspersed and I sure the hell wasn’t going to type in 200+ variable names. My typing skills aren’t that good.

In fact, because I am a total lazy slacker, I did this:
proc means data = in.g8_achieve07  n ;
output out = sam ;
proc transpose data = sam ;
id _stat_ ;
proc sort data = data1 ;
by _name_ ;
proc print data = data1 noobs ;
var _name_ ;

Then, I copied the variable names beginning with S from the output, pasted it under the word DROP and had my drop statement.  Yes, I will do almost anything to avoid typing. My mother told me that my life’s goal should be NOT to become a secretary (she was a secretary for 25 years so I guess she knows whereof she speaks).

At first, I  just coded everything either incorrect or correct, ignoring the partial responses. Then, I got to wondering whether that would make any difference. So, I re-ran the scoring program giving half a point for partial response and one point for a full response. This is not identical to the TIMSS scoring, which gives 2 points for a correct response and 1 for a partial response. I guess the rationale is that a partial response to a really difficult question should be equivalent to a completely correct response to a much easier question. I am just guessing this is their reasoning,  and it does make some sense.

The reason I did it with the 1 point, .5 point is that it took me about one second to modify my program. You can see at right that whether I coded partial credit or not made very little difference over all, but for some items it was significant. Of course, for the items that didn’t have partial credit, it made absolutely no difference. The overall mean of item difficulty changed almost none – it was around .50 either way, which is actually optimal for item difficulty for a test.

Note that neither of the ways I scored the data were the way TIMSS scored it. All multiple choice items were one point in their method, SOME of the problems that required a written solution awarded 2 points with possible 1 point partial response credit. Others did not award partial credit. It’s not difficult to use the formats supplied with the TIMSS data to create a scoring program, but it will take longer than the one second it took me previously.

One reason I am fiddling with the scoring is that I want to see how robust these results are. People always say you can prove anything with statistics. Yeah, in the same way you can prove anything with an apple pie.

You can say, “If you listen very closely, this apple pie says President Obama is an alien” and some people will be stupid enough to believe you or simply very, very ill-informed. (Psst – apple pies can’t talk. Now you know.)

Before I go ahead and do any analyses by group I want to know if there are any global decisions that make a difference, like awarding partial credit or not. Everything I am doing so far just entails getting to know the data better – how it was coded, how it’s distributed, how different scoring criteria might make a difference. I was interested in this because right or wrong is a completely objective fact in mathematics – the area of the rectangle is 32 or it isn’t – but the decision to award partial credit or not is just that, a decision.

For now, though, I have to get back to work that pays actual money. Since it was Easter, I went to mass, of course, then out to eat with my lovely children, then to Universal Studios, which resulted in me getting back, doing  work for actual money until 3 a.m. and then updating my blog.

Therein lies the drawback of much of the analysis of open data, in that it relies on the goodwill of people like me to conduct and document their analyses for free – goodwill which is limited by the desire to occasionally see my children, the importance to me of being at mass on Easter, and the need to have enough cash for the cost of annual passes to Universal Studios.

Oh, and Happy Easter!

Apr

20

True confessions: I just don’t get the data hackathon

I am old. If I was not aware of this by looking at my U.S. birth certificate (which, like President Obama, I do have), I have America’s most spoiled thirteen-year-old to tell me,

“You’re old.”

(sometimes followed by – “and stupid” if I have just told her that no, I will not buy her an iPhone 4 because her current iPhone works just fine, or no, I will not buy her a $26 lip gloss at Sephora because THAT is, in fact, stupid, and no one who is thirteen and beautiful needs to wear cosmetics anyway. )

So, it has been established by four daughters in succession becoming thirteen, that I am old and out of it.

Perhaps this explains why the data hackathon and similar concepts mystify me. I just don’t see what of any use can be accomplished by people with no familiarity throwing themselves at data for 24 or 48 hours without sleep.

  1. Have none of you read all of the research on how sleep deprivation has a negative effect on cognitive performance? Do you think for some reason it doesn’t apply to you? Seriously, the ability to go without sleep is not a super-power. It’s more often a sign you’re using coke, which as we used to say in college, is God’s way of telling you that you’re making too much money.
  2. At a rock bottom minimum, to understand any dataset you need to know how it was coded. What is variable M03216, exactly? If that is the question, “What is 3 cubed?”  , then what do the answers A, B, C, D 97 98 and 99 mean? Which of A through D is 27, which is the correct answer, and what the hell are 97 – 99 ? These are usually some type of missing value indicator, possibly showing whether the student was not administered that particular item, didn’t get that far in the test or just didn’t answer it.
  3. Once you’ve figured out how the data are coded, you need to make some decisions about what to do with it AT THE ITEM LEVEL. What I mean by that is, for example, if an item wasn’t administered, I count the data as missing, because there was no way the student could answer it. If the student left it blank, I count it as wrong because I’m assuming, on a test, if you knew the right answer, you would have given it. What about if the student didn’t make it that far in the test? I count it as wrong then, also, but I do realize I have less degree of certainty in that case. My point is, these are decisions that need to be made for each item. You can do it in a blanket way – as in, I’m doing this for all items, or you can do it on a case by case basis. Whether you do it knowingly or not, something is going to happen with those items.
  4. Often, sampling and weighting are issues. If you are doing anything more than just sample descriptive statistics (which aren’t all that useful in and of themselves) you need to know something about how the data are sampled and weighted. Much of the government data available uses stratified sampling, and often certain strata are sampled disproportionately. AT A MINIMUM , you need to read some documentation to find out how the sampling is done. If you don’t, maybe you’ll be lucky and the actual sampling was random, which is the default assumption for every procedure I can imagine. Listen carefully – Hoping to get lucky is a very poor basis for an analysis plan.
  5. Data quality needs to be checked and data cleaning done. Unless you are a complete novice or a complete moron, you are going to start by doing things like checking for out-of-range values, reasonable means and standard deviations, the expected distribution, just to make sure that each variable you are going to be using in your analysis behaves the way it should. If there are no diseases recorded on Sundays in some zip codes, I doubt it’s because people don’t get sick those days but rather because the labs are closed. This may make a difference in the incidence of a disease if people who get sick on weekends go to another area to be treated. You need to understand your data. It’s that simple.
  6. Somewhere in there is the knowledge of analytic procedures part. For example, I wanted to do a factor analysis (just trust me, I had a good reason). Well, because of the way this particular dataset was put together, no one had complete data, so I couldn’t do PROC FACTOR. I thought possibly that I could use PROC CALIS with method = FIML – that is the full information maximum likelihood method. I went to the SAS documentation and flipped through the first several pages, glanced at some model by Bentler illustrating indicators and latent constructs, said to myself, “Blah blah, I know what that is”, skipped over the stuff on various types of covariance matrices, etc. I can guarantee you that the first time I looked at this (which was a long time ago), I did not flip through it. I probably spent 15 minutes looking at just that model making sure that I knew what F1 was, what v1, v2, etc. were and so on. That is one advantage of being old, you’ve seen a lot of this stuff and puzzled it out before.
  7. You need to actually have the software required. Although there is supposedly a version of SAS out there that allows the use of FIML for PROC CALIS none of the clients I have at the moment have the latest version of SAS, so I don’t have access to it. I believe FIML is only available in SAS 9.22.  You might think having the software available would be a benefit of a hackathon but the ones I have seen advertised all say to bring your own laptop, so I am assuming not.
  8. All of this is before knowledge of the actual content area, be it contributions to charity, earthquake severity, mathematics achievement or whatever, I assume there has been research done previously on the topic and it might be helpful to have that knowledge. I will guess that the organizers of the hackathon are assuming whoever attends will already have #5, 6 and 7, here. This is again a bit puzzling to me because the combination of that knowledge took me years of experience and I don’t think I am particularly slow-witted. It’s like all the job announcements you see that want someone twenty-five years old with at least ten years of experience. The hackathons seemed to be aimed at young people. (The advertised free coffee and all the fast food you can eat is not really much an enticement for most people in their fifties. If you can’t afford to buy your own coffee and fast food by that age, I think you took a wrong turn somewhere and perhaps you should hang up that DATA ROCK STAR t-shirt, buy a suit and go get a job.)

So, I don’t get it. I don’t get what you are hoping to pull together in 48 hours, with little sleep, little time, and often limited knowledge of the data. PERHAPS the idea is that if enough people try enough different random things that we will just get lucky??

Luck is not a research strategy.

Oh, and there’s this little thing called Type I error.

Apr

19

Anyone who tells you they know all of SAS is like that creepy guy at the fraternity party who swears that his father is the Duke of Canada, that is, they have a perception of themselves that is not in screaming distance of contact with reality.

Even though I have been using SAS for decades, I am still discovering new functions, tricks, tips and procedures all of the time. In fact, the one I came across today was so helpful, I even cross-posted this on SAScommunity.org

My problem and how I solved it

There are currently over 370,000 datasets on the data.gov site, not to mention the numerous others available from the National Center for Education Statistics and many other open data sources. One problem users of these data often encounter is that the formats used by the creator are not necessarily those desired by the end user. For example, many files have user-defined formats for each individual item such as:

value S9FMT
1 = “A”
2 = “B”
3 = “C”
4 = “D*”
6 = “NOT REACHED”
8 = “NOT ADMIN.”
9 = “OMITTED”
;
value S10FMT
1 = “A”
2 = “B”
3 = “C”
4 = “D”
5 = “E*”
6 = “NOT REACHED”
8 = “NOT ADMIN.”
9 = “OMITTED”
;

A common desire would be to have the items that were not administered to the student have a missing value and the rest scored as either correct or incorrect. At first thought, using the PUT function to get the formatted value might seem like a good idea, but that would require specifying the format. Since many of these datasets include several hundred variables and several hundred different formats, that’s not going to work.

Here is one solution using the 2007 dataset for eighth grade students from the Trends in International Mathematics and Science Study (TIMSS) :

%include "C:\Users\me\Documents\TIMSS\Tformats.sas" ;

data scored ;
set in.G8_ACHIEVE07 ;
attrib Fval length = $9. ;
array rec{*}  M022043 — BSSREA05 ;
do i = 1 to dim(rec) ;
FVAL = vvalue(rec{i}) ;
if FVAL  = “NOT ADMIN” then rec{i} = . ;
else if index(FVAL,”*”) > 0 or FVAL = “CORRECT R” then rec{i} = 1  ;
else rec{i} = 0 ;
end ;

The %INCLUDE statement includes the formats defined by the organization that created the dataset.
(TIMSS, like many of the original data sources, includes a folder of SAS files along with the data downloaded that read in text data, create formats and labels, merge files and perform other useful functions. )

The VVALUE function returns the formatted value of the variable.
In the program above, it is necessary to first recode the items that were not administered to have a missing value, and then score the students who were administered an item as having been correct or incorrect. In this particular example, all of the formatted values for correct responses either had an “*” next to the correct multiple choice value or the words “CORRECT RESPONSE” . Of course, this statement would need to be modified depending on the formatted values of your particular dataset.

Apr

18


I’ve never understood masochists. Back in the days when I was competing I would regularly get calls from creepy men who were willing to pay me big bucks to beat them up. As one of my lovely daughters said of the sumo wrestler who sent her a picture of himself posing in his diaper-thingie and asking her for a date,

In a word – eeew!

I am not a masochist. Nor a witch, for that matter, in case anyone is interested. And yet, I find myself spending not hours, but days poring over codebooks and technical manuals to understand open data datasets. The latest is the TIMSS (Trends in International Mathematics and Science Study) dataset.

There does not seem to be any substantial resource for analysis of open data or collaboration by people doing it.  I noticed that donorschoose.org has a competition going on, and that is certainly a worthwhile cause. It focused on their data although they do suggest merging with other possible datasets. There are also the community forums on the data.gov website, which get surprisingly little traffic given that over 300,000 raw datasets are available. Since I could not find any place else to post this information, I am putting it here, largely for myself for later use, but also for anyone else who might be working with TIMSS or similar data and find this information useful. If you do know of resources on analysis of open data, PLEASE post the information here!

Sampling – from the TIMSS technical report and user guide

I don’t believe anything anyone says unless I can prove it myself. My initial suspicion was that perhaps the country comparisons were not equivalent, that is, it may be that we had a very representative sample of students in the U.S. where other countries had more selective samples. For a HYPOTHETICAL example, if you had a country where nearly 100% of students get an education at least through the eighth grade (like in the U.S.) and you compared them to a country where the drop out rate before eighth grade was 50%, then you might find that the U.S. performed more poorly when that is not necessarily the case at all.

I still wonder if that might be true, but what I have concluded so far is that the TIMSS sample seems to be a pretty fair, representative sample of the U.S. Students from high poverty and central city schools are, in fact, slightly underrepresented, but the difference from the population is really so slight that it is not worth mentioning, even though I did just mention it.

What did I expect? Well, I hoped that this might be the case but one reads everything about education in the media from the average teacher in Wisconsin making $100,000 a year (AS IF!) to that high school drop out problems are due to  illegal aliens (slight effect if you remove non-citizens from the data, but very small compared to other factors).  The data on sampling available are extremely detailed and probably more complex than really necessary, in my opinion. However, my opinion is primarily based on the use I intend for these data which is not the same as the main goal of TIMSS.

Personally, I’m very interested in what exactly do American eighth-graders know, at a micro-level, that is, what questions did they get right and which did they get wrong? One of the benefits of making your data open to everyone is that it can be applied to answer questions that were not part of your original study. It opens up the possibility of getting a great deal more in the way of analyses than your research team can do on their own.

Of course, it also means that others can scrutinize every bit of your research design and hold it up for criticism. So, kudos for the TIMSS team for taking the plunge!

Administration and instrument

TIMSS documentation states that the test is designed to measure five areas of mathematics and three levels of difficulty. There are fourteen versions of the test and students receive one at random. They can impute plausible values for the items that were not administered from the ones that were. While I can speculate on some very good reasons for doing it that way – in particular, if you have one version of a very high stakes test, it won’t be that hard for people to get hold of that test and teach the answers to the exact questions that are on it. Having 14 different versions certainly makes cheating much harder. Regardless,  I would have preferred for my purposes that everyone had been given the same test because I’d like a very large N . When I look at the item frequencies, each item says “NOT ADMINISTERED” for about 86% of the subjects. Well, 1/14 is a lot less than 14%, in fact it is more like 7%, so, obviously, each of the 14 different parallel forms wasn’t completely different.

The good news so far is that TIMSS documentation and data have had almost everything I have wanted.

The bad news is that there is an almost overwhelming amount of information. The technical report and user guide is over 300 pages long. The codebook for the eighth grade mathematics achievement is 178 pages long and the eighth grade achievement dataset alone is 572 variables. I’m not particularly interested in science at this point, so I can drop all of those. On the other hand, there are no data on ethnicity or income in this dataset so it is clear I am going to have to merge these data with the school and student files at some point.

The programs include user-defined formats so you can get format errors if you run their programs as is. You have three options, one is to just delete their format statement, a second is to create the formats, either temporarily by copying them at the top of your program or, more sensibly, using a %INCLUDE statement or making them permanent formats and the third is to use the OPTIONS NOFMTERR when you don’t feel like messing with the formats. Usually I would not add formats permanently but since I can already sense that I am going to be using these data a lot, I’m going to go ahead and do that this time.

SURPRISE  – an age of 13 doesn’t mean 13 years old !

It’s going to be a lot of work, but worth it. I’ve already seen some very interesting statistics. I was astounded to see that only 1% of the students were under 14 at the time of testing. In fact, there were twice as many students who were 16 years old in the eighth grade as 13. Even if the testing is at the end of the year, that seems really low.

Just a little tip on age – the ages appear to be stored with decimal ages even though they print as integers. So, if you have a statement that says something like

If bsdage in (14,15) then  ….

You will find very few students.  In fact, you’ll get the students who are exactly 14 and 15.

My recommendation is when you read in the data NOT to use the age format TIMSS uses, which seems to round students to the nearest age. I don’t think most people think of age like after they’re five years old. They quit saying they are  “almost six” and give you their real age.

You can then cut the data however you like. I used a cut-off of under 13.5 years as “young” for the eighth grade and over 15.5 years as old.

Since my mother let three of her children skip grades in school against the advice of administrators, and it turned out with varying results, I have always been fascinated by the experiences of kids who are young for their grade. Personally, I think it was the best thing my mom ever did for me and will be forever grateful that she told the principal to stick it (actually, my mom would never say anything like that and for her to buck authority, especially a nun, was extraordinarily out of character).  I let my oldest daughter start  kindergarten at four and go off to college at 17. My brother, on the other hand, thought it was a great hardship for him and said he would never let his kids skip a year of school.

As I suspected, those who were young for their grade were disproportionately female (64%) while those who were old for their grade were disproportionately male (67%).  Whether this represents anything more than people accepting a stereotype that boys mature late, I don’t know.

The boys who were younger than average seemed to be doing quite well. Male or female, children who were young for their grade did best, children who were old for their grade did worst. My guess would be that children who were advanced were promoted and those who did poorly were held back, certainly not an earth-shattering revelation. What it does seem to suggest at a first glance, though, is for both males and females, being a grade ahead of children their same age isn’t related to academic problems in the eighth grade.

It’s 1 a.m. and I should probably be thinking about sleeping but I am just starting to get into the fun part of the data.

So, what have I learned from open data? In a nutshell, it’s a mountain of work to get started with a dataset of any complexity, but like most times in life, the work can pay off, sometimes in unexpected ways.

« go back

Blogroll

WP Themes