Several interesting random thoughts from JSM:
From a session by Freeda Cooner entitled, “Bayesian statistics are powerful, not magical”
ways in which Bayes results could be slanted (one hopes unwittingly) were discussed. One point worth repeating is that the validity assumes accurate priors. Kind of obvious, no? Yet, the question is, where did you get your prior probabilities? Did you base them on studies of use of this drug with adults and your current study is of children? Did you base them on studies of a “similar” drug but this is a study of a new drug?
As I said, when you think about this point, it is kind of obvious but I suspect people don’t think about it often enough.
A second interesting point was made by Milo Schield about “causal heterogeneity”. That is, we like to think if we are testing a new treatment that those who live survive because of the treatment (saved) and those who die do so as a result of the failure of the treatment (killed). That is, we act as if there are only two categories. In reality, he says, there are four groups. In addition to the “saved” and “killed” groups there are those who would have lived regardless “immune” and those who would have died regardless “doomed”.
Another point by Schield was that although we always say that correlation does not mean causation we almost always give examples of confounding variables. We say, for example, that although ice cream sales go up along with violent crime, eating ice cream doesn’t cause you to go after your neighbor with a baseball bat, unless perhaps your neighbor is spied eating ice cream off of your spouse. However, when we look at probability in terms of p-values we are really spending most of our introductory statistics courses testing whether or not observed relationships are a coincidence and we should emphasize guarding against coincidence more than confounding.
Personally, I think I do talk about this a lot, so if you do not, feel shame.
Another really interesting idea came from Chris Fonnenbeck. He was discussing putting his code up on github and I thought,
“Why don’t I do that? Why don’t other people in our company?”
I hate to admit that we had just never taken the time to do it, for which I now feel guilt because I do look for code on github occasionally, and, more often, just browse it looking for interesting ideas, or the hell of it.