{"id":891,"date":"2010-12-05T03:11:54","date_gmt":"2010-12-05T08:11:54","guid":{"rendered":"http:\/\/www.thejuliagroup.com\/blog\/?p=891"},"modified":"2010-12-05T03:28:26","modified_gmt":"2010-12-05T08:28:26","slug":"logistic-regression-proves-i-have-no-soul","status":"publish","type":"post","link":"https:\/\/www.thejuliagroup.com\/blog\/logistic-regression-proves-i-have-no-soul\/","title":{"rendered":"Logistic regression proves I have no soul"},"content":{"rendered":"<p>The <a href=\"http:\/\/www.reverb10.com\/december-3-moment\/\">#reverb10 prompt for December 3rd <\/a>was to write about a time when you felt most truly alive in 2010. There were more prompts,\u00a0 about what you wonder about and other examining-your-soul type of introspection. This isn&#8217;t that kind of blog. I don&#8217;t think I&#8217;m that type of person. For the record, the time I feel most alive is when I am with my family but I wasn&#8217;t the least bit interested in writing about how much I love my family right now. In fact, I was very interested in logistic regression.<\/p>\n<figure id=\"attachment_572\" aria-describedby=\"caption-attachment-572\" style=\"width: 260px\" class=\"wp-caption alignright\"><a href=\"http:\/\/www.thejuliagroup.com\/blog\/wp-content\/uploads\/2010\/07\/evaeats.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-572\" title=\"evaeats\" src=\"http:\/\/www.thejuliagroup.com\/blog\/wp-content\/uploads\/2010\/07\/evaeats.jpg\" alt=\"\" width=\"260\" height=\"249\" \/><\/a><figcaption id=\"caption-attachment-572\" class=\"wp-caption-text\">I&#39;ll get this down eventually<\/figcaption><\/figure>\n<p>Should YOU wonder about logistic regression? Well, that depends.<\/p>\n<ul>\n<li>Do you have a continuous, numeric dependent variable? If yes, do something else, maybe multiple regression.<\/li>\n<li>Is one of your variables a dependent variable? If no, do something else, maybe log-linear modeling.<\/li>\n<li>Do  you have more than one independent variable? If not do something else,  usually either a chi-square (if both your variables are categorical) or a  t-test if one of your variables is continuous.<\/li>\n<\/ul>\n<p>Logistic regression is the statistical technique of choice when you have a single dependent variable and multiple independent variables from which you would like to predict it.<\/p>\n<p>With logistic regression, the dependent variable you are modeling is the  PROBABILITY of the value of Y being a certain value divided by ONE  MINUS THE PROBABILITY. Let&#8217;s start with the simplest model, binary  logistic regression. There are two probabilities, married or not. We are  modeling the probability that an individual is married, yes or no.\u00a0  [Logistic regression is NOT what you would use to model how long a  marriage lasted. That would be survival analysis.]<\/p>\n<p>The logistic regression formula models the log of the odds ratio. That is<\/p>\n<p>The probability of y =1 \/ probability of y = 0<\/p>\n<p>So, the left side of your equation is<\/p>\n<p>ln(p \/ (1- p) )<\/p>\n<p>**** Very, mega- super-important point here &#8211; the p in this equation is  NOT the same old p as in p &lt; .05. No, au contraire. Completely  different. This is the probability of event = 1. For example, the  probability of being married. 1-p then would be 1 &#8211; the probability of  being married.\u00a0 Yes, that second number is the same as the probability of being single. You aren&#8217;t missing anything.<\/p>\n<p>I was, in this post going to use the probability of being a dumb-ass  but some people have written and told me that I am too hostile for a  statistician so I am trying to mend my ways, it being around the  holidays and all.<\/p>\n<p>The right side of the equation is the same old \u00df0 + \u00df1X1 + &#8230;\u00dfnXn<br \/>\nthat you are used to with Ordinary Least Squares (OLS) regression also known as multiple regression or multiple linear regression, or, if you are a complete weirdo, Monkey-Bob .<\/p>\n<p><strong>The ODDS RATIO is<\/strong><\/p>\n<p>The probability of y =1 \/ probability of y = 0\u00a0 when x =1<\/p>\n<p>divided by<\/p>\n<p>The probability of y = 1\/ probability of y = 0 when x = 0<\/p>\n<p>I presume the only reason you have read this far is that you have some  deep-rooted need or desire to understand logistic regression. An example  will help. I have discovered lately that I love my husband for a very  important reason. He is not a dumb ass. I have had multiple husbands  (not simultaneously, that would be polyandry and illegal in most states  and immoral according to certain anal-retentive religions) what they all  had in common, other than the obvious being married to me, is that they  were all in technical fields and pretty good at what they did. Let&#8217;s go  with the hypothesis that people who are in a technical field are more  likely to be married.\u00a0 Further, let&#8217;s say that we have sampled 100  people in computer science and 100 people in French literature. We find  that 90 of the computer scientists are married and 45 of the French  literature people.<\/p>\n<p>So, if the probability of marriage is 90\/100\u00a0 and the probability of not  married is 10\/100 then the odds ratio of\u00a0 9:1 for the computer  scientists. =\u00a0 9<\/p>\n<p>For the French literature people, the probability of marriage is 45\/  100\u00a0 and the probability of not being married is 55\/100\u00a0 = .818<\/p>\n<p>So, 9\/.818\u00a0 = 11.00<\/p>\n<p>This tells you that the odds of a computer scientist being married  versus single are 11 times that of a French literature professor. Also,  that you should study computer science instead of French.<\/p>\n<p>If you really had nothing else to do in your life and wanted to run this  using SPSS just to see if I was correct (really, now!) you would get  this output.<\/p>\n<p>Gasp! The value of\u00a0 \u00df0, that being our constant, is -.201 . The inverse of the log is Exp(x) also shown as &#8220;e to the x&#8221;. This is a function in SPSS, if you want to double-check. Also a function in SAS, Stata and Excel, but NOT on the calculator on my iPhone. Steve Jobs should feel shame.<\/p>\n<p>The value of\u00a0 Exp(-.201) = .818\u00a0 &#8211;\u00a0 the odds\u00a0 for French literature people.<\/p>\n<p>The value of Exp (2.398) = 11.00\u00a0 &#8211; the odds ratio for computer scientists versus French literature whatever you call them (unemployed would be my guess).<\/p>\n<p><a href=\"http:\/\/www.thejuliagroup.com\/blog\/wp-content\/uploads\/2010\/12\/logistic.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignright size-medium wp-image-892\" title=\"logistic\" src=\"http:\/\/www.thejuliagroup.com\/blog\/wp-content\/uploads\/2010\/12\/logistic-300x61.jpg\" alt=\"\" width=\"300\" height=\"61\" srcset=\"https:\/\/www.thejuliagroup.com\/blog\/wp-content\/uploads\/2010\/12\/logistic-300x61.jpg 300w, https:\/\/www.thejuliagroup.com\/blog\/wp-content\/uploads\/2010\/12\/logistic.jpg 561w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>Coincidence? I think not!<\/p>\n<p>In interpreting a logistic regression analysis you want to look at the significance of the parameter estimates (.000) and the parameter estimate, in this case the \u00df = 2.398. A positive coefficient says that the dependent is MORE likely if the variable has the value in question. In SPSS, that value is shown in parentheses. Notice it says cs(1) &#8211; that means when cs has the value of 1, the outcome is more likely to occur. How much more likely? Look to your right. (On the table, in this blog post,not to your right in your room. What are you thinking?) The odds are 11 times greater for computer scientists than for French literature whats-its.<\/p>\n<p><a href=\"http:\/\/docs.google.com\/viewer?a=v&amp;q=cache:CWhAJCP1k9oJ:www.upa.pdx.edu\/IOA\/newsom\/da2\/ho_logistic.pdf+logistic+regression&amp;hl=en&amp;gl=us&amp;pid=bl&amp;srcid=ADGEESj1DzZhmIeJa3oi6L0KZZj1jX59QRRWEaimGGv6MbEFKVSXMBU5d894akwftSFBjSk8_Rkl_HpaDM2Cv-Ri173-e4q-UNkDOO5c33s_tfIemBVIeGPS2reKOT8tPeoriFRMyaTX&amp;sig=AHIEtbSej5mxpiB7SgKhcaTFz2d7F9PTcQ\">A really good reference if you want a plain language introduction to logistic regression is by Newsom<\/a> . There are a lot of really bad references to logistic regression in very obscure language but I decided not to bother mentioning them.<\/p>\n<p>The syntax for producing this table in SPSS is below.<\/p>\n<p><span style=\"color: #000000; font-size: medium;\">LOGISTIC\u00a0REGRESSION\u00a0VARIABLES\u00a0married<br \/>\n\/METHOD=ENTER\u00a0cs<br \/>\n\/CONTRAST\u00a0(cs)=Indicator(1)<br \/>\n\/CRITERIA=PIN(.05)\u00a0POUT(.10)\u00a0ITERATE(20)\u00a0CUT(.5).<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The #reverb10 prompt for December 3rd was to write about a time when you felt most truly alive in 2010. There were more prompts,\u00a0 about what you wonder about and other examining-your-soul type of introspection. This isn&#8217;t that kind of blog. I don&#8217;t think I&#8217;m that type of person. For the record, the time I&#8230;<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[11],"tags":[],"class_list":["post-891","post","type-post","status-publish","format-standard","hentry","category-statistics"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts\/891","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/comments?post=891"}],"version-history":[{"count":7,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts\/891\/revisions"}],"predecessor-version":[{"id":894,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/posts\/891\/revisions\/894"}],"wp:attachment":[{"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/media?parent=891"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/categories?post=891"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.thejuliagroup.com\/blog\/wp-json\/wp\/v2\/tags?post=891"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}