4

I am running a linear regression for Explanatory purposes. Y is continuous and all the explanatory variables are categorical. I understand that the regression coefficient of these variables is the difference in mean of that variable and the mean of the reference for that categorical. My model is as follows:

$Rates = 0.92 + 0.2 \mathrm{Comapny_{STK}} + 0.1 \mathrm{Company_{FLR}} + 0.2\mathrm{Location_{ATL}} - 0.8 \mathrm{Location_{NYC}} + 0.5 \mathrm{Location_{SA}} + \ldots + w_n x_n$

However, is there another method I could use so that the interpretation of the coefficients is similar to that when one runs a linear regression with just continuous explanatory variables, and I could then see the relative effect of each explanatory on $Rates$? I essentially want to see which among $Company$ and $Location$ has the biggest influence on $Rates$. To my understanding, two-way ANOVA's would not be able to answer which of the categoricals has a bigger influence. So I'm not too sure what sort of correlation measure or otherwise to use here.

Jojo
  • 369
  • 1
  • 12

1 Answers1

3

Remove the constant from the RHS and take all categories for each categorical value as dummy.

For example, say you have two categorical variables $X, Y$ and one continuous dependent variable $Z$. Assume each of the categorical variables have two categories: $1, 2$

You can model it in two ways

One (seemingly what you have done): $$E(Z |X, Y)=\beta_0 + \beta_1 D_x + \beta_2 D_y$$

Here $D_x=1$ if $X=x_2$ and $0$ otherwise. Similarly for $D_y$

From this, we have that $$\beta_0 = E(Z | X = x_1, Y=y_1)$$

Alternatively, you can model like this:

$$E(Z |X, Y)= \alpha_1 D_{x_1} + \alpha_2 D_{y_1} + \beta_1 D_{x_2} + \beta_2 D_{y_2}$$

Now each coefficient tells you what is the conditional mean of $Z$ in presence of your categorical variable.

Dayne
  • 2,113
  • 1
  • 7
  • 24
  • Thanks. Im still a bit confused: 1) For the first method, wouldn't the $\beta$ values still be having the interpretation that they are the difference in mean of that dummy and the mean of the reference dummy for that categorical? 2) And with the second method, isn't there a potential for multicollinearity issues, given that both dummies for each categorical is being regressed? – Jojo Oct 11 '20 at 16:09
  • 1
    In first method, $\beta_1$ is the change in $E(Z|X=x_1)$. What you want is change in $E(Z)$. In first method you get this by $\beta_0 + \beta_1$. In second method you get from $\beta_1$ directly the change in $E(Z)$. To you second question: No there won't be multicollinearity issue here as you have removed the constant term. Think in terms of the $X$ matrix. Only when the first column is $1$ and including separate dummies for all levels of the categorical variables, will the matrix become singular. – Dayne Oct 11 '20 at 17:46
  • 1
    Strictly speaking the above argument on $\beta_1$ is true when there is only one categorical value. But it conveys the idea. – Dayne Oct 11 '20 at 17:49
  • This helps. As the comment right above is based on only one categorical value, when there is more than one categorical variable, is $\beta_1$ just the change in $E(Z)$ while holding the other categoricals constant or how does the interpretation change? And to confirm, to get the influence of different categoricals on a continuous dependent, all one has to do is to set the constant to 0? Thank you – Jojo Oct 11 '20 at 17:59
  • 1
    In second method adding more variables won't change anything. It's still good. And yes making the constant zero does the job. To make this more clear, substitute $D_{x_1}=1-D_{x_2}$. Same for $y$. You'll get back first model with a constant. Second model is just the first model written differently to achieve a simpler desirable interpretation of coefficients. – Dayne Oct 11 '20 at 18:08
  • Thanks. And, again just so that I fully understand, the 2nd model coefficients would have the same interpretation, even in a scenario where a categorical has more than 2 categories? – Jojo Oct 11 '20 at 21:24
  • 1
    Yes. As long as there are dummies for each level – Dayne Oct 12 '20 at 02:13
  • Thanks.I understood method 2 but want to understand method 1 better. Looking at [this answer](https://stats.stackexchange.com/questions/120030/interpretation-of-betas-when-there-are-multiple-categorical-variables) Im a bit confused. It suggests $\beta_0+\beta_1$ gives the $E(Rates)$ when $D_x=1$ and $D_y=0$, so in my example it would be $E(Rates)$ when $Company = STK$ and $Location = NYC$ for example, where $NYC$ is the reference for $Location$. Here, I dont see a direct interpretation of $STK$'s effect on $Rates$? Would it be possible to write in words what the interpretation is? Thanks a lot – Jojo Oct 12 '20 at 21:10
  • 1
    In method 1, one dummy is less. Say you have not put a dummy for location $X$ and company $Y$. So $\beta_0$ is the mean value of rates when location is X and company is Y. Interpretation of $\beta_1$, which is coefficient of NYC's dummy, is that how much does the mean of rates (when location is NYC and company is Y) different from mean of the rates (when location is X and company is Y). In method two, coefficient of NYC dummy tells you how much is the average rates when location is NYC, regardless of what company is. – Dayne Oct 16 '20 at 15:39
  • 1
    Thank you, this confirms what I was thinking – Jojo Oct 16 '20 at 18:23