9

I'm a bit stuck with a problem here and any kind of help would help a lot :)

Just to give a clue about my data. I have 6 independant variables (IV) which are:

  • $X_1$ = Population -within a block-
  • $X_2$ = Households -within a block-
  • $X_3$ = Total Rooms -aggregated-
  • $X_4$ = Total Bedrood -aggregated-
  • $X_5$ = Median Income
  • $X_6$ = Ocean Proximity [Categorical]

and my dependant variable (DV) is $Y$ = Median House Price.

I ran a regression including all IVs but there is a violation for almost all OLS assumptions in addition to huge multicollinearity. Here are the residual plot and normality plot before any adjustments.

enter image description here

What I did then is transform all my IVs and my DV using the Box-Tidwell method which I think it is not the correct way of solving the issue so this is my first question what can I do to solve the normality issue?

The other problem is that even after transforming all variables I still have an issue in my residual plot which is the main problem here. I have a linear pattern on the graph that I don't know how to solve. I run my DV against each IV separately and still have the same issue. Here is the graph for the transformed model.

enter image description here

Avraham
  • 3,182
  • 21
  • 40
Aziz
  • 91
  • 2
  • 13
    The line shows that there is a hard upper bound on median home price. So perhaps the data refer only to blocks where the median is less than a certain value? If so you need to revise the model to account for this specific truncation. Better yet, get the full data without the truncation. – BigBendRegion Dec 28 '21 at 11:49
  • 1
    Having the x and y scales be the same would make it more clear what's going on. – Acccumulation Dec 29 '21 at 03:56

4 Answers4

23

Just to help you understand what you are looking at a bit better on your residual plot, your data looks something like this:

enter image description here

Your model is fine until the price gets capped; then you need to determine whether the rest of the model is valid or not. The capped price has to be due to unrecorded data above that price because you would not expect to see data like that in reality for your particular problem. So then you have to think about what the data looks like above that price. It may be that the linear relationship no longer holds once you go above the grey line and this would be a limitation of using a linear model here. The data may curve and flatten off in reality, in which case a logarithmic curve would fit much better, so it would be unwise to predict data above that line with a linear model.

Also, do you care what happens above the grey line, or do you only need the model for the part where the model is valid? If you are only interested in the portion of the model that is valid, then you don't need to worry about the rest. These are some of the things you might want to think about.

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
ryan132442
  • 361
  • 4
  • 1
    how did you see/derive that? The normality plot doesn't look bad (?) so do you conclude from the residuals ~ predicted? – Ben Dec 29 '21 at 07:34
  • 6
    @Ben I assume you mean how did I see the shape or that data? If so, it actually really easy. If you look at the residual plot, the horizontal line where the residual is equal to zero is the linear model. So the residual plot is essentially just a rotation of the linear model. If you rotate my drawing so that the purple line is horizontal, you are looking at the residual plot. This is only true for the 2 dimensional case where you have X and Y. When you have 6 X's like this question, the data will look like this, but it's in 6 dimensions so I cannot draw that, but the cap is still be there. – ryan132442 Dec 29 '21 at 12:12
  • 2
    well, that was surprisingly easy, indeed. Thank you! :) – Ben Dec 29 '21 at 12:18
10

I'm not sure why the linear model, with its many assumptions, was chosen as the default model. Ordinal semiparametric models are very efficient and are invariant to how Y is transformed. They allow for floor and ceiling effects, bimodality, and any other kind of distributional quirks you can throw at them. The most popular semiparametric models are the proportional odds and proportional hazards models.

I would also not want to assume up front that the continuous predictors operate linearly. I'd expand them using regression splines such as restricted cubic splines (aka natural splines). A detailed case study may be found in Chapter 11 of the RMS course notes.

Frank Harrell
  • 74,029
  • 5
  • 148
  • 322
9

Residuals following a linear boundary are typically the result of a ceiling effect or a floor effect. If your sample is large enough, the results are biased only slightly (note that linear models are quite robust to normality violations if you are working with a large sample, unlike, for example, homoskedasticity violations ).

Basically, you have two options. You can use a different model (e.g. lognormal regression, Poisson regression, depending on the specific data), or you can ignore the problem and rely on the fact that the results are only slightly biased since you have a large data set.

Daniel Dostal
  • 654
  • 3
  • 10
6

Regarding the linear pattern: this happens frequently when your response takes the same value for a number of observations. For those observations, $(y-pred)$ is of course a linear function of $pred$, as $y$ is constant.

F. Tusell
  • 7,733
  • 19
  • 34