Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [intercept]

The intercept in regression-type models is the value of the Y variable when all X variables are 0.

The intercept in regression-type models is the value of the response / outcome ($Y$) variable when all predictor / explanatory ($X$) variables are $0$.

307 questions
160
votes
9 answers

When is it ok to remove the intercept in a linear regression model?

I am running linear regression models and wondering what the conditions are for removing the intercept term. In comparing results from two different regressions where one has the intercept and the other does not, I notice that the $R^2$ of the…
analyticsPierce
  • 1,793
  • 3
  • 12
  • 6
128
votes
2 answers

Removal of statistically significant intercept term increases $R^2$ in linear model

In a simple linear model with a single explanatory variable, $\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$ I find that removing the intercept term improves the fit greatly (value of $R^2$ goes from 0.3 to 0.9). However, the intercept term…
Ernest A
  • 2,062
  • 3
  • 17
  • 16
26
votes
8 answers

When forcing intercept of 0 in linear regression is acceptable/advisable

I have a regression model to estimate the completion time of a process, based on various factors. I have 200 trials of these processes, where the 9 factors being measured vary widely. When I perform a linear regression of the 9 factors (and all 2…
Zack Newsham
  • 675
  • 1
  • 8
  • 12
24
votes
1 answer

How to treat categorical predictors in LASSO

I am running a LASSO that has some categorical variable predictors and some continuous ones. I have a question about the categorical variables. The first step I understand is to break each of them into dummies, standardize them for fair…
Hatshepsut
  • 1,429
  • 4
  • 15
  • 26
22
votes
6 answers

Reason for not shrinking the bias (intercept) term in regression

For a linear model $y=\beta_0+x\beta+\varepsilon$, the shrinkage term is always $P(\beta) $. What is the reason that we do not shrink the bias (intercept) term $\beta_0$? Should we shrink the bias term in the neural network models?
yliueagle
  • 755
  • 2
  • 6
  • 10
20
votes
3 answers

Why would one suppress the intercept in linear regression?

In a number of statistical packages including SAS, SPSS and maybe more, there is an option to "suppress the intercept". Why would you want to do that?
user333
  • 6,621
  • 17
  • 44
  • 54
18
votes
3 answers

Intercept term in logistic regression

Suppose we have the following logistic regression model: $$\text{logit}(p) = \beta_0+\beta_{1}x_{1} + \beta_{2}x_{2}$$ Is $\beta_0$ the odds of the event when $x_1 = 0$ and $x_2=0$? In other words, it is the odds of the event when $x_1$ and $x_2$…
logisticgu
  • 301
  • 1
  • 3
  • 5
16
votes
1 answer

The difference between with or without intercept model in logistic regression

I like to understand the difference between with or without intercept model in logistic regression Is there any difference between them except that with the intercept the coefficients regard the log(odds ratio) relative to the baseline group and…
user148087
  • 161
  • 1
  • 3
14
votes
1 answer

Why the does the intercept of my null model not equal the mean when I log transform the outcome variable? How do I interpret it?

I have an outcome variable that is right skewed, so I log transformed it. I made a null model with only the log-transformed outcome variable, but when I exponentiate the estimate, it does not equal the mean. Concerned it was issues with my data, I…
J.Sabree
  • 243
  • 1
  • 4
13
votes
5 answers

Different usage of the term "Bias" in stats/machine learning

I think I've seen about 4 different usages of the word "bias" in stats/ML, and all these usages seem to be non-related. I just wanted to get clarification that the usages are indeed non-related. Here are the 4 I've seen: (1) "Bias"-variance…
student010101
  • 334
  • 2
  • 10
11
votes
2 answers

What does the formula y ~ x + 0 in R actually calculate?

What is the statistical difference between doing a linear regression in R with the formula set to y ~ x + 0 instead of y ~ x? How do I interpret those two different results?
JimBoy
  • 1,006
  • 8
  • 15
9
votes
2 answers

Regression through the origin

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1) \\ (5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3) $$ How can we find the best fitting line $y=ax$ through the points? My calculator has the option…
EdwardHarrison
  • 91
  • 1
  • 2
9
votes
1 answer

Understanding the intercept value in a multiple linear regression with categorical values

I'm failing to understand the value of the intercept value in a multiple linear regression with categorical values. Taking the "warpbreaks" data set as an example, when I do: > lm(breaks ~ wool, data=warpbreaks) Call: lm(formula = breaks ~ wool,…
Joao Azevedo
  • 407
  • 1
  • 3
  • 9
8
votes
2 answers

Bias initialization in convolutional neural network

What is the correct way to initialize biases in convolutional neural networks (tf.zeros, tf.truncated_normal, tf.random_normal), and why? Should biases be constant? Do we even need biases in a deep neural network (like VGG-16)? In a siamese neural…
Hello Lili
  • 319
  • 1
  • 3
  • 9
8
votes
4 answers

Does the intercept in a logistic regression capture the unobserved effects?

Theoretically, does the intercept term in a logistic regression model capture all unobserved effects? In other words, in a logistic regression model with a perfect fit (i.e. all relevant variables are included), the intercept term should be zero…
student_of_life
  • 113
  • 6
1
2 3
…
20 21