2

Let say we have a dataset, $\mathbf{X}$ of $m$ instances, and $n$ features, and a target scalar variable $\mathbf{y}$ ($m$ instances).

Now I want to do a regression so, I try to fit a hyperplane $ y = \mathbf{x} .\mathbf{w}$ + c.

Note : $\mathbf{w}$ is a $ n \times 1$ vector of coefficients that we need to find out.

and the $\mathbf{W} = (\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \mathbf{y}$.

(Least Squares PseudoinVerse)

Now does mean centering reduce $c=0$, ie does mean centering make the fitting hyperplane pass through the origin of the new coordinate system formed after mean centering is perormed?

ttnphns
  • 51,648
  • 40
  • 253
  • 462
user76170
  • 639
  • 2
  • 8
  • 9

1 Answers1

3

To answer the question in the text: If you center both $\mathbf{X}$ and $\mathbf{y}$, then $c$ will be 0 (up to machine precision).

To answer the question in the title: It is not necessary, but it can sometimes help in the interpretation of results, especially if you include interaction terms. Even than I would normally center on some substantively meaningful value within the range of the data, rather than the mean.

Maarten Buis
  • 19,189
  • 29
  • 59
  • 2
    A simple example is a regression on calendar year, and nothing else. If you use years such as 2014, then the intercept will be the value predicted for the year 0. Setting aside the small detail that there was no such year, such an intercept will in many applications be way outside the range of the data and/or look absurd in any case. Switching to an origin of some round date roughly in the middle of your data (e.g. using year $-$ 1950, or year $-$ 2000) will ensure a meaningful intercept (which can then be ignored, or pointed out to beginners as perfectly sensible). – Nick Cox Oct 22 '14 at 11:26