0

Suppose we want to do simple linear regression. Before we do simple linear regression, we need to check these following assumptions (please correct me if I'm wrong):

  • Linear relationship
  • Normality of residual
  • Homoskedasticity of residual
  • No autocorrelation

My question is: When the data doesn't follow several or even all of the assumption, which one should be handled first?

I have several data that have one variable independent (X) and one variable dependent (Y).

  • My first data violates normality and autocorrelation assumption. When I handled the autocorrelation using Cochrane-Orcutt transformation, the data became normally distributed. In this case, did I only need to handle the autocorrelation issue?
  • My second data violates autocorrelation assumption. I handled it with the same method, but the data became non-normally distributed. What should I do?
  • My third data doesn't follow linearity and normality assumption. I tried to handle the normality issue first by transforming the orginal data to sqrt(X) and sqrt(Y) (I'm not sure this is right to do), then did linearity test again, and the result said that the data was linear.
  • My fourth data violates normality, heteroskedasticity, and autocorrelation assumption. Since transformation method will affect result of the other assumption test, which issue should be handled first to get right conclusion?
zetto
  • 1
  • 1
  • 1
  • If you are worried about autocorrelation, you presumably have at least three variables, with time (or space) as well to define serial order. – Nick Cox Apr 07 '16 at 09:32

2 Answers2

3

You started with a premise of simple linear (in the parameters) regression but quickly mentioned autocorrelation. Those two phrases are inconsistent.

Ignoring correlations within the data, the other problems IMHO are best dealt with by instead choosing a method that is transformation-invariant, does not assume equal variances, and does not assume a particular distribution of $Y$ for any particular $X$. That method would be a semiparametric ordinal regression model such as the proportional odds or proportional hazards model. Ordinal models work very well for continuous $Y$ as been discussed elsewhere on this site.

Frank Harrell
  • 74,029
  • 5
  • 148
  • 322
2

The most important assumptions are that the data is randomly sampled and that the error term and independent variables are not correlated. Only in these cases will you have wrong expected value for beta coefficient of the slope.

For autocorrelation you can simply use FGLS or HACSE, but OLS is not biased if there is autocorrelation, merely inefficient. Not a huge problem if you have lots of data.

Likewise the normality assumption is not important, if you have enough data.

Linearity of parameters assumption can not be violated if you are using OLS, it's an axiom. Rather you could have the wrong functional form for the variables. You should correct the functional form, if this is the case or the error term could be correlated with betas and OLS become biased. It will also output wrong forecasts.

Heteroscedasticity can also be corrected with FGLS, or by using HACSE. Heteroscedasticity does not lead to biased estimator either, so it's not very important.

Dole
  • 743
  • 3
  • 10
  • 22
  • 2
    I suspect that I agree with you, but mentioning "wrong functional form" some way down the list is puzzling. The most important condition is that there is a linear relationship. Whether you call it an axiom or an assumption divides the people. Mathematically-minded statistical people take it as given, so that they can deduce how estimators work, or don't. But practically-minded statistical people start with a problem and data, and the functional form is what to worry about first. Everything else is secondary. – Nick Cox Apr 07 '16 at 09:49
  • 1
    Apologies, I meant linearity in parameters being an axiom. Wrong functional form of the variables violates the Gauss Markov assumptions. The first of these is the Gauss Markov assumption so I was speculating the question was regarding that (linear relationship is not a condition after all). Edited now. – Dole Apr 07 '16 at 10:16
  • Sure, I take linearity in parameters as implied. Your view is the view of writers of econometrics textbooks. The world looks differently to empirical people. Also, mainstream statistics treatments of regression are typically more relaxed and less interested in the consequences of formal deductions. – Nick Cox Apr 07 '16 at 10:22
  • @NickCox Indeed that is the background I come from. I still don't see how the parameters can be anything but linear when using OLS by definition... Or why OLS would be biased if you don't have a linear relationship, as you can simply use the correct functional form. Could you elaborate? I suspect you must be talking about the transformed variable and not the actual variable (feel also free to edit the answer if there is something important I am missing). – Dole Apr 07 '16 at 10:25
  • 3
    We are starting in different places. You are assuming that the world is $y = X\beta$ and then worrying about the details of fitting the model. The practical person starts with $y, X$ and is considering what can go wrong if linear regression is applied, to which the first comment should be be careful about assuming $X\beta$. In fact, the practical person probably starts with $y$ and a bundle of other stuff to choose from that might end up in $X$. I have no disagreement about technical results, which are all of a form "if this, then that". I am just underlining that the context is wider. – Nick Cox Apr 07 '16 at 10:32
  • 2
    Clearly there are many issues around transformations, interactions, etc. in many practical contexts. But, apart from being a little amusing, the fact that I could identify your econ* background from just your wording is a reminder that there are different emphases here within statistical science. In my own field, getting a functional form that matches the underlying behaviour trumps all other considerations and all the machinery of inference is just decorative. But identifying the correct functional form is far from trivial. – Nick Cox Apr 07 '16 at 10:35