0

My model consists of a dependent variable $y$ that can take values 0 and 1. The independent variable $x$ is a categorical variable that can take values, 1, 2 and 3. The logit model looks like this: $$L=\beta x$$ where $L$ are the log-odds of event $y=1$.

Now when I see my data graphically, I can clearly see the probability of $y=1$ increasing as $x$ increases from 1 to 3. However, when I run the regression above in Stata, I don't get significant values for $\beta$.

Given that I know that $x$ changing causes the probability of $y=1$ to change, can these insignificant results be interpreted? Are they useful at all?

Edit:

The sample size is very small, only 273 observations. $y=1$ occurs 204 times whereas $y=0$ occurs 69 times. This is a bar graph showing the percentage of $x=1, 2, 3$ that also have $y=1$:

enter image description here

Regression results:

enter image description here

  • The coefs are mislabeled above. 1,2 should be 2,3.
Parth
  • 103
  • 4
  • 1. No $\epsilon$ in model. 2. sample size and # of events (Y=1)? – user158565 Jul 31 '19 at 20:51
  • @user158565 edited – Parth Jul 31 '19 at 20:55
  • Given Yes/no = 204/69, the sample size is not too small. If there is no other covariate in the model, i can say that maybe your visually difference is kind of illusion. maybe you could add graph in your question. How do you use the $x$, continuous or categorical. – user158565 Jul 31 '19 at 21:03
  • @user158565 thanks for the input. I've uploaded the graph and the regression results. x is categorical. – Parth Jul 31 '19 at 21:24
  • It is really difficult to say there is difference between three groups based on graph and results of logistic regression. – user158565 Jul 31 '19 at 21:35
  • I think your logit coefficients are mislabeled. invlogit(1.176321)=.76428567 (category 1), invlogit(1.176321+.463422) =.83749996 (category 3, not 2), and invlogit(1.176321+.4106438)=.83018864 (category 2, not 1). – dimitriy Jul 31 '19 at 22:09
  • You might want to report the output of `margins your_cat_var` or `margins r.your_cat_var` after the logit to make the comparison to your bar chart easier. – dimitriy Jul 31 '19 at 22:12
  • Given LRT p value = 0.3463, no way to decrease p value < 0.05, and no way to argue that x has effect on y. – user158565 Aug 01 '19 at 01:21
  • @DimitriyV.Masterov yes you're right. They should be 2 and 3, not 1 and 2. Thanks. – Parth Aug 01 '19 at 04:56

2 Answers2

1

You have analysed the variable as a categorical variables in Stata, but in your ad-hoc analysis you refer to it as increasing (i.e ordinal). You could therefore consider using orthogonal polynomials to encode the input variable. Here are a couple of other questions that may give useful ideas of how to extend your analysis:

How to handle ordinal categorical variable as independent variable

Logistic regression and ordinal independent variables

Jinglesting
  • 256
  • 2
  • 9
1

Here's some toy code on the cars dataset showing how to reconcile the graph with the logit model output. The key question is whether the heights of the bars are actually different or whether those gaps could be a fluke of your sample. The logit followed by a margins command will give you some error bars that help you decide. This is easier to interpret than the logit coefficients which are on the log-odds scale.

Below I first make the bar graph, and use the logit to make the same bar graph with CIs. The second graph confirms that the slight difference between 1 and 2 is pretty indistinguishable, but that category 3 is unusual in the fraction of foreign cars: the CI for 3 has almost no overlap with CIs for 1 and 2. The third graph plots the the pair-wise differences with their CIs and formally confirms that intuition. Here we are checking whether the CIs for the differences contain zero or not.

enter image description here

Here's the code that produces the graph above:

. /* model data */
. sysuse auto, clear
(1978 Automobile Data)

. xtile x = mpg, nq(3)

. 
. /* bar graph */
. graph bar foreign , over(x) name(bar, replace)

. 
. /* logit analysis */
. logit foreign i.x, nolog

Logistic regression                             Number of obs     =         74
                                                LR chi2(2)        =       7.79
                                                Prob > chi2       =     0.0204
Log likelihood = -41.139873                     Pseudo R2         =     0.0865

------------------------------------------------------------------------------
     foreign |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           x |
          2  |   .1466035   .7057522     0.21   0.835    -1.236645    1.529852
          3  |   1.568616   .6478402     2.42   0.015     .2988724    2.838359
             |
       _cons |  -1.481605   .4954337    -2.99   0.003    -2.452637   -.5105723
------------------------------------------------------------------------------

. 
. /* bar graph with CIs */
. margins x

Adjusted predictions                            Number of obs     =         74
Model VCE    : OIM

Expression   : Pr(foreign), predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           x |
          1  |   .1851852   .0747568     2.48   0.013     .0386646    .3317058
          2  |   .2083333   .0828982     2.51   0.012     .0458559    .3708108
          3  |   .5217391   .1041586     5.01   0.000      .317592    .7258863
------------------------------------------------------------------------------

. marginsplot, name(phats, replace) recast(bar)

  Variables that uniquely identify margins: x

. 
. /* comparisons of bars with CIs */
. margins x, pwcompare(effects)

Pairwise comparisons of adjusted predictions    Number of obs     =         74
Model VCE    : OIM

Expression   : Pr(foreign), predict()

------------------------------------------------------------------------------
             |            Delta-method    Unadjusted           Unadjusted
             |   Contrast   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           x |
     2 vs 1  |   .0231481   .1116274     0.21   0.836    -.1956376    .2419339
     3 vs 1  |   .3365539   .1282092     2.63   0.009     .0852685    .5878393
     3 vs 2  |   .3134058   .1331207     2.35   0.019      .052494    .5743176
------------------------------------------------------------------------------

. marginsplot, unique name(pwu, replace) recast(bar)

  Variables that uniquely identify margins: _pw

      _pw enumerates all pairwise comparisons; _pw0 enumerates the reference categories; _pw1 enumerates the comparison categories.

. 
. graph combine bar phats pwu, holes(3) altshrink

Stata Code:

/* model data */
sysuse auto, clear
xtile x = mpg, nq(3)

/* bar graph */
graph bar foreign , over(x) name(bar, replace)

/* logit analysis */
logit foreign i.x, nolog

/* bar graph with CIs */
margins x
marginsplot, name(phats, replace) recast(bar)

/* comparisons of bars with CIs */
margins x, pwcompare(effects)
marginsplot, unique name(pwu, replace) recast(bar)

graph combine bar phats pwu, holes(3) altshrink
dimitriy
  • 31,081
  • 5
  • 63
  • 138