3

This is more of a soft question, but communication is part of statistician's job.

The background: I am trained as a statistician. My boss knows little about statistics.

The problem: He wants to fit a regression model with positive coefficients only. The reason is it is good for explanation. In other words, only positive coefficients match his expectations/economic theory/common sense. The method is fit first and delete variables with negative coefficients.

I don't think this is correct.

My reasons:

  1. Explanations/expectations/economic theory/common sense are very subjective. You have your model first, then you apply your explanation to the model. You can not do it the other way around.
  2. You delete a variable of negative coefficient, then some of the positive coefficients may change to negative.

My question: What would you say to explain why it is wrong? It has to be easy to understand for people who have little background in statistics.

Jill Clover
  • 615
  • 1
  • 6
  • 17
  • 2
    @AntoniParellada You are right. – Jill Clover Oct 24 '16 at 19:33
  • 3
    If your boss wants positive coefficients, why not solve the following quadratic program? $$\begin{array}{ll} \text{minimize} & \| \mathrm A \mathrm x - \mathrm b \|_2^2\\ \text{subject to} & \mathrm x \geq \mathrm 0_n\end{array}$$ – Rodrigo de Azevedo Oct 24 '16 at 19:42
  • @RodrigodeAzevedo Or use some other method of estimating the model(s) using constraints? – Alexis Oct 24 '16 at 20:03
  • 2
    This questions sounds like something in the comic Dilbert. Can't you just invert all variables with negative coefficients? That's what Dilbert would do. – JonB Oct 24 '16 at 20:49

1 Answers1

4

The idea of restricting coefficients to be positive is not inherently flawed but it has to be done in the right way. That sounds a bit controversial so let me explain before getting on to what you boss is doing wrong.

If we were fitting a physical model where laws are well understood we know that certain coefficients shouldn't be negative. For example, one coefficient might be equal the mass of the electron and we're experimenting to find out what that mass is. If the experiment involved many variables of regression then it's possible that statistical error or correlations between variables would cause the electron mass coefficient to be negative.

Of course physical laws are more rigid than economic theory. If the purpose of the regression is to inform us about the economics then insisting that variables are negative would be wrong (if we are very certain of economic theory then this at least tells us that the data collection was flawed). If the purpose is to create a model (Perhaps to use to optimize profit) then restricting coefficients to be positive might be beneficial. Negative coefficients could prevent you from optimizing (e.g. parabola model with a negative coefficient of $x^2$ has no minimum)

Having said that, your boss' idea to remove negative values is totally mislead. Even if you decide to restrict coefficients to be positive, you're aware that removing negative values is a poor way to do that.

If this was my boss I would explain that the regression model is trained with all variables considered. If you take out one variable then we have to repeat the regression from scratch with all remaining variables considered. Removing the one variable with a negative coefficient could lead to others which were previously positive to now be negative. Or likewise, coefficients which were initially negative might be positive if added into the model at the end.

Hugh
  • 3,659
  • 16
  • 22
  • 2
    Not to mention it leads to all the [problems with stepwise model selection](http://stats.stackexchange.com/questions/124979/investigating-interaction/124993#124993). – Alexis Oct 24 '16 at 20:12