I’m just sitting here in Boston airport waiting to catch the bus to Rhode Island. For some reason, the WordPress app on my iPad doesn’t want to upload the images I downloaded from my phone. This is the sort of thing I would often get upset about. I also have several emails to answer already and it’s not even 5 am at home.Landing here is a bit of travel down Memory Lane because my daughter, Ronda, lived here for a few years while training for the Olympics and my other daughter, Maria, lived here with her for family for a few years, right after she had her first child.
Coming back to places always makes me reflect on what has happened in the time intervening. Life has been a rush lately – startup incubator, doing a seed round, making games, hiring – all of the things that go into moving from being a small business to a big one in a short period of time.
Everyone I know spends a lot of time worrying. When Ronda lived here, she was unhappy being away from home but convinced in her own mind that she needed to be here to develop further in judo. She did improve a lot, but did she really need to stay as long as she did? Who knows? When Maria lived here, they bought their first house, which they liked a lot, but then her husband decided to change jobs and they moved in rapid succession to Silicon Valley and then to Santa Monica (Silicon Beach). She worried a lot about how they were going to pay the bills and find a house big enough for three kids. As for me, I worried about all of my kids all the time, about Jenn living in San Francisco, one of the most expensive cities in the country, and trying to find a job with a history degree.
So … what happened? Well, Ronda went into mixed martial arts and movies, Maria and her husband started two businesses between them, Maria and Ronda wrote a book that has been on the New York Times bestseller list for weeks, Jenn got a teaching credential and masters degree from USC, got a job, married and moved into a house in the valley. Just as a random fact, Julia was on the Homecoming Court for her high school yesterday.
So, what did I worry about and what good did it do?
I want everything to be perfect all of the time. I do. This may explain some of my successes, but it also results in not being nearly as happy and relaxed as I could be.
Why CAN’T everything be perfect all of the time?
When I was about 10 years old, I asked my grandmother this exact question. She stopped cleaning the house and answered me very seriously.
Do you know what you get when the sun shines all of the time? A desert. There is a saying, ‘Into every life a little rain must fall.’ If you don’t have bad times, you won’t appreciate the good ones.
That’s definitely true for me. When I was younger, I had to make weight for judo tournaments, and that sometimes meant going without eating for a couple of days before weigh-ins. Ice cream has never tasted as good as it did right after weighing in. One reason I appreciate sleeping until 10 am – or I did before we started ramping up 7 Generation Games – is that there were years when I was going to school full-time and working full-time or working 3 jobs when my husband was sick and I had a family to support, that I never got enough sleep. So, just getting 8 hours sleep is something I appreciate every time it happens (and I assume will appreciate even more when I get back to the point where I can do it again).
It’s also true, that just like my grandmother pointed out that the rain makes the flowers grow, the stresses and difficulties make us better and stronger. Ronda learned judo, Jenn learned history, Maria learned marketing and I have learned more about financing and scaling up than I thought possible in such a short period of time.
Quit looking at what you haven’t got and start looking at where you’re going.
That, my dears, is the secret to happiness.
Do you really need to document everything?
Those who say that there is no such thing as a stupid question might be reconsidering their position right about now. Of course we need to document everything!
Perhaps my reluctance stems from my hatred of technical writers. If you are a tech writer and are a decent human being, the next time we are at the same event, please come up and introduce yourself. I would like to see what you look like. All of the technical writers I ever knew well were evil, which made me suspicious of the rest of you. I should also note here that if you respond to this by posting hateful comments you will have only reinforced the stereotype of tech writers = evil.
Supposedly, tech writers are hired because of their ability to communicate well, which makes me wonder why they insist on making such insulting comments as:
I translate what the programmers wrote into English so the normal people can understand it.
I take what the engineers say and put it into complete sentences.
Everyone knows that software engineers can’t communicate with other humans.
Hey, tech writers, you do know we’re standing right here and we can hear you, right?
Animosity toward an entire semi-profession aside, I caught myself wondering how much documentation was really necessary. I was looking through some code I had written months before, which, I have to confess had almost no comments in the code and no documentation anywhere outside of the code. Even though I hadn’t looked at it in four months (there was a comment with the date created!), I found it pretty easy to read and got to thinking that perhaps documentation was over-sold by literature majors who couldn’t find jobs.
Then, an uncharacteristic burst of rationality overtook me and I realized that our company is growing. The code was pretty clear to me because I’m fairly familiar with the canvas tag and using canvas for graphics. The program used two libraries with which I’m familiar – jquery and a library for making charts. There were 50 other ways the program was easy for me to read because I wrote it using what was easy and familiar for me. However, we’re a growing company, and as The Invisible Developer reminded me, whether it happens kicking and screaming or I go quietly, the handwriting is on the wall and I’m going to be doing less coding and more CEO’ing.
I know that if HE were to have taken the same program, there would be much swearing going on upstairs right now.
So, yes, documentation appears to be more necessary than originally believed.
Is the solution for me to go through and document everything? Sigh. If only I had infinite time.
What I think might work and be a good idea for a new employee is to have him or her start off with reading some of the code and documenting it. That would be a good way to get familiar with what we are doing before diving into a project. It would also be a good way for us to verify if that person understands what is going on in a particular piece of code.
Or, one might say that was a lazy way for me to get out of writing documentation – if one were a tech writer.
My ISP is currently not allowing me to upload or insert image files. Show your sympathy for me by buying games. The games will make you smarter, amuse you and enable me to afford a better host. Everyone wins!
I went to the Western Users of SAS Software conference last week and came home sad. It wasn’t for the reasons you’d guess.
- No one sexually harassed me (as if!).
- I did not forget how to code a confirmatory factor analysis in the middle of the pre-conference class I was teaching.
- The other statisticians didn’t make fun of me. (Not within earshot, anyway.)
- All the other SAS programmers didn’t refuse to sit and drink with me at the networking mixer.
It was actually during Maura Stokes’ presentation on new developments in statistics procedures that I started to get depressed. You might wonder why discussion of adding link=alogit in PROC LOGISTIC so that the adjacent category is used as the referent would be a cause for dismay or what is disheartening about changing the default number of imputations in PROC MI to 25 from 5.
The problem is that as I sat there I thought,
This is some interesting stuff and I probably won’t get the time to try hardly any of it.
Now, I do realize that not being able to play with the latest software is close to first on the list of First World Problems.
This isn’t the first time I’ve been in this situation. When I started my career, I was a pretty good programmer. That’s what we were called back then and it didn’t bother us. Software engineer, software developer and data scientist weren’t things yet.
After I earned my doctorate and started teaching, my husband was in an accident and I became the sole breadwinner for the family, so my consulting side gig grew and all of a sudden I had more people working for me than I could count. It was more like dozens than hundreds, but I just didn’t have any time to count them. I also didn’t have time to do nearly as much data analysis and coding as I would have liked and ended up in the unhappy situation that some of the people working for me were more up on the latest technology than I was. (NOTE to young people: If your boss is not up on the newest OS, language, update etc. it maybe isn’t because he or she is unintelligent, uninterested or an old fogey but perhaps, instead, is doing the things that bring in the money to pay your salary.)
After a while, I disentangled myself, paid the bills and remarried, not in that order, and was able to get back to doing work that really interested me. However, here I am again, spending more time building a company than building a product.
A few weeks ago when we began our three months as part of the Boom Startup ed tech accelerator, The Invisible Developer turned to me and said,
“You know, I’ll bet there came a point when Bill Gates started to spend more time building Microsoft than building operating systems.”
I’d pout about spending so much time in meetings on convertible notes, discount rates and forecast returns but then I remember that:
- All companies need some adult supervision, so it may as well be me.
- The alternative to let someone else take the responsibility and make the decisions.
—- This is my company —
You wouldn’t think there would be that much to say about scree plots. That is because you are like me and sometimes wrong.
The problem I often have teaching is that I assume people know a lot more than is reasonable to expect for someone coming into a course. Sometimes, I’m like a toddler who thinks that because she knows what color hat the baby was wearing yesterday that you do, too.
So …. a scree plot is a plot of the eigenvalues by the factor number. In the chart below, the first factor has an eigenvalue of about 5.5 while the eigenvalue of the second factor is around 1.5. (If you don’t know what an eigenvalue is, read this post. )
As I mentioned in the previous post, an eigenvalue greater than 1 explains more than a single item, but as you can tell by looking at the plot, some of those eigenvalues are barely higher than one. Should you keep them? Or not?
What is scree, anyway? Scree is a pile of debris at the base of a cliff. In a scree plot, the real factors are at the top of the cliff and the scree is the random factors at the bottom you should discard. So, based on this, you might decide you only have one real factor.
The idea is to discard all of the factors after the line starts to flatten out. But is that after 1 factor? It kind of flattens out after four? Maybe?
Sometimes a scree plot is really clear, but this one, not as much. So, what should you do next?
Hmm … maybe I should write another post on that.
I find this scree plot of eigenvalues very helpful in identifying the number of factors. A scree plot is a plot of the eigenvalues by the factor number.
I realized this is only helpful if one understands what an eigenvalue is.
Understand that the default is to factor analyze the correlation matrix and that means that your variables are standardized before analysis, with a variance of 1. So, that is the total amount of variance we are trying to explain for each variable.
Therefore, the total amount of variance to be explained in a matrix will equal the number of variables. If you have 10 variables, the total amount of variance to be explained is 10.
If you’ve ever looked a correlation matrix you will have noticed that all of the diagonals are 1. The correlation of an item with itself is 1.
An eigenvalue is the total amount of variance in the variables in the dataset explained by the common factor. (Mathematically, it’s the sum of the squared factor loadings. If you are interested in that, you can come to my class at WUSS on Wednesday morning. Or, possibly, I’ll blog about it next week.)
Now, if a factor has an eigenvalue of 1, it is pretty useless. That is because the whole purposes of factor analysis is to replace your 20 or 50 items that each explain of variance of 1 (their own variance) with a few common factors. A factor with an eigenvalue of 1 doesn’t explain any more variance than a single item.
Let’s say you have 24 items and your first factor has an eigenvalue of 6. Is that good? Yes, because that means that a single factor explains as much of the variance in the matrix of data as six items. If you could get four orthogonal factors, each with an eigenvalue close to 6, then you would have explained nearly 100% of the variance in your 24 items with just 4 factors.
Think about correlation matrices again. It’s not often you see an EXACTLY zero correlation. You’ll find correlations of .08, .03, .12 just by chance. Who knows, the same person had the highest score on sticks of bubble gum chewed and number of asses kicked (R.I.P. Rowdy Roddy Piper), it doesn’t mean that those two variables really have that much in common. This is why we look at statistical significance and how likely something is to occur by chance.
How do you tell if that factor has a higher degree of common variance, that is the eigenvalue is higher, than would be expected by chance? One way is the scree plot. You look at the eigenvalue for each factor and see where it drops off.
I would write more about this but my family is urging me to leave for a barbecue for Maria’s birthday, so you will have to last until tomorrow for a more detailed explanation of scree plots and why they are called that.