0

I am currently working on regression models and am unsure whether we should transform the data. From what I learned, it is recommended that we should:

  1. transform the continuous predictors (e.g. log-transform, spline transform) prior to the logistic regression.
  2. transform the outcome variable in the linear regression (e.g. log-transform) if it is not normally distributed.

Are these the correct summaries/approaches? If not, can someone please correct them?

Also, I have two follow-up questions:

  1. Is there any way we can transform the "categorical variable" prior to the logistic (or linear) regression? If so, how can we do that?
  2. Can spline transformation be applied to the outcome variable in the linear regression (I never see this done in the papers I read)?
R Beginner
  • 191
  • 1
  • 5

1 Answers1

2

Main question 1:

Please take a look at the highest-voted data-transformation posts on this site. For regression models, the idea is to have a linear relationship between the outcome (in an appropriate scale) and the predictor. For logistic regression, that would be a relationship between log-odds and the predictor. Splines can be useful, as they let the data tell you what the transformation should be.

Main question 2:

That's wrong. See this answer, for example: "One transforms the dependent variable to achieve approximate symmetry and homoscedasticity of the residuals." Residuals about the regression are key, not the outcome values themselves. Normality of residuals is nice, but not strictly necessary.

Follow-up question 1:

There's no need to transform an unordered categorical variable in an unpenalized regression. Each level of the predictor has its own additive contribution to the overall linear predictor.

If your categories are ordered then you could consider orthogonal polynomial coding, or penalized maximum likelihood or Bayesian shrinkage. See the sections of Frank Harrell's course notes or textbook on "fitting ordinal predictors."

Follow-up question 2:

Ordinal regression allows for something similar to what you want, maintaining the ordering of numeric outcome values while allowing for differences in effective step sizes between outcomes from what you would have with a simple numeric outcome scale. Although it's often described for outcomes with a small number of ordered levels, ordinal regression can be applied to much more general continuous outcomes. The Harrell references above also cover that topic.

EdM
  • 57,766
  • 7
  • 66
  • 187
  • In response to the 1st question, can you pls elaborate on the part "Splines can be useful, as they let the data tell you what the transformation should be."? Can I simply use spline transformation for all the continuous predictors prior to the logistic regression regardless? In other words, can spline transformation do more harm than good in some cases? – R Beginner Dec 07 '21 at 22:49
  • Also, do you know how we can obtain the odds ratio from the spline-transformed predictors? – R Beginner Dec 08 '21 at 00:27
  • @RBeginner you fit splines to continuous predictors as _part of_ the logistic regression, not as a pre-processing step. See the portions of Harrell's notes or book on restricted cubic splines, for example. If you have a large enough data set, starting with such splines can both indicate whether there is any substantial non-linear association with outcome and provide a useful fit if that's the case. With such a spline, however, there no longer is a single association of odds ratio with a unit increase of the predictor. One then illustrates with examples at chosen predictor values. – EdM Dec 08 '21 at 19:21
  • Thank you for such good feedback. I forgot to mention that I am working with not so big dataset (only ~50 participants). Does the spline function still apply to my problem? If not, is there any alternative transformation like can help me improve my model fit? – R Beginner Dec 08 '21 at 19:27
  • 1
    @RBeginner there's a danger of overfitting if you have a lot of predictors and just a few cases. Harrell's notes and book, in particular chapter 4 of each, are great resources for learning how to deal with these problems. A spline could still be done, you just might not be able to make it as flexible as you could with a larger data set. It depends on how many other predictors there are and, in logistic regression, the size of the minority class rather than the total number of cases. – EdM Dec 08 '21 at 19:36
  • Thank you for the feedback. Do you know how to interpret the odds ratio for the spline-transformed predictor? – R Beginner Dec 09 '21 at 04:50
  • 1
    @RBeginner with a spline fit there isn't a single odds-ratio coefficient for the predictor. With a spline the change in odds ratio associated with a change in predictor value depends on the specific predictor values. To illustrate, you can plot the log-odds predicted from the spline fit as a function of predictor value. See for example Fig. 10.6 in the logistic regression chapters of Harrell's RMS book or notes. It can also help to show specific examples representative of the types of predictor values that you find in practice. – EdM Dec 09 '21 at 15:32