1

I'm trying to perform LASSO regression on a dataset, and the following 2 photos show the histograms of 8 attributes. I'd like to transform them in some way to improve the model. Data transformation is bit of an art so I'd like to hear your thoughts on how you would transform it (if at all), and why. For instance, do you HAVE to transform a skewed data or is it not a deal breaker? So far, I've learned techniques like log transform, binning, and ranking the data putting the ranks into quantiles.

enter image description here

enter image description here

I wouldn't do anything with "Therapy" since that's already a binary attribute with 0 or 1. "Age" and "ORGANNUM" is fairly uniformly or normally distributed so I wouldn't transform them.

"BLIL6" and "BLLBILI" seems like good candidates for binning, or creating categorical variables out of them, since they have extreme values. But what if I don't?

I'm not sure about PRAPACHE, BLLPLAT. Is it okay to leave them be?

Please advise in regards to regression on the best practices and why. Thanks

user173729
  • 193
  • 5
  • It's a scatter plot matrix that deserves first attention, not a set of histograms. But I agree that the minor modes of your BL* variables might be problematic if they are associated with multlvariate outliers. – Nick Cox Sep 26 '21 at 17:52

1 Answers1

2

You shouldn't necessarily worry too much about the marginal distribution of your variables. The assumptions of linear regression are about the distribution of the residuals.

This is a common source of confusion - see this answer for example.

That being said, it could well be the case that you end up with a better model after transforming some of your variables. The way to figure this out is by fitting different models and seeing what works best for your data (in terms of goodness of fit / residual analysis).

Adrià Luz
  • 746
  • 11
  • What's more important than the residuals is whether the structure you're fitting is about right, e.g. approximately linear if that is the structure you're fitting. However, that's also a chicken and egg question to some extent. Here the occurrence of outliers is more likely to be diagnostic than overall skewness. – Nick Cox Sep 26 '21 at 17:50