I am new to machine learning and I am simultaneously studying linear and logistic regression. Logistic regression is when there is one dependent variable and there may be more than one independent variable
Multivariable linear regression attempts to find a linear relationship between multiple variables where every independent variable x must have a dependent variable y.
Can anyone tell me the difference between the two?
In linear regression, you predict the target variable $y$ using a model that is based on a linear function of the predictors $X_1,X_2,\dots,X_k$
$$ y = \beta_0 + \beta_1 X_2 + \beta_2 X_2 + \dots + \beta_k X_k + \varepsilon $$
In logistic regression, the target variable is binary, and you are trying to predict probability of "success" $p(y=1)$ using a linear predictor $\eta$ passed through the link function $\sigma$ (logistic function is the most common choice), i.e.
$$ \eta = \beta_0 + \beta_1 X_2 + \beta_2 X_2 + \dots + \beta_k X_k \\ p(y=1) = \sigma(\eta) $$
So both can be functions of a single, or multiple variables. The differences are that the target variable differs (values on real line vs binary), and logistic regression, as other generalized linear models, uses a link function.
In multiple linear regression (MLR), you still have one dependent variable, and the relation is as follows: $$y_i=\beta_0+\sum_{i=1}^n\beta_ix_i+\epsilon$$ In logistic regression (LR), what you predict is a probability, namely the class posterior, and is bounded unlike MLR output. They're similar in the sense that they both use linear combination of features. The probability of class 1 is estimated as follows in LR: $$p_i=\frac{1}{1+e^{-(\beta_0+\sum\beta_i x_i)}}$$
So, LR is actually taking $\text{logit}$ of MLR output.
