Linear and non linear Regression models for single variable

Question

I want to know if there is any regression model for single variable other than simple linear regression. I usually use tree based regression models when there are more than 1 feature and for data with only 1 independent variable, I cant think of any other model other than simple linear model. For simple regression problems, I end up using data transformation into non linear if necessary but I cant think of other distribution free or robust models that work well on single variable model.

My question is, are there any other models (linear or non linear models) that work generally well on single independent variable with 1 dependent variable other than simple linear regression? I tried tree based regressors and MARS on single variable problem but they dont seem to work very well when there are not many features ...

Answer 1

Well, depending on the nature of the response variable $Y$ there are many possible regression models using only one predictor $x$ (and an intercept, mostly.)

• Logistic regression when $Y$ is binary

• Other glm's (generalized linear models)

• If $Y$ is ordinal, ordinal regression, How to handle ordinal categorical variable as independent variable

• If $Y$ is categorical/nominal, multinomial regression

• ...

You should read up on generalized linear models, which is a natural starting point. Look at Understanding of GLM, Is there a GLM bible? or search this site.

Answer 2

Using basis expansion one can easily extend simple linear regression into non-linear models.

Here is an example of how basis expansion works (with Fourier and polynomial basis).

Depending on the data, we can chose the right model to fit. In the link, we are trying to fit a periodic data, so it is better to use Fourier basis. Note that it is one independent variable and one dependent variable setting, and it works similar to "multiple regression". But we are fitting a non-linear model.

What's wrong to fit periodic data with polynomials?