0

I am reading An Introduction to Statistical Learning with Applications in R by G. James, D. Witten, T. Hastie and R. Tibshiran 2013 after taking a basic statistics course a little while ago.

On page 21 it states the parametric method for determining a $Y = f(X)$:

First, we make an assumption about the functional form, or shape, of $f$. For example, one very simple assumption is that $f$ is linear in $X$:

$$ f(X)= \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_p X_p. $$

This is a linear model, which will be discussed extensively in Chapter 3. Once we have assumed that $f$ is linear, the problem of estimating $f$ is greatly simplified. Instead of having to estimate an entirely arbitrary $p$-dimensional function $f(X)$, one only needs to estimate the $p + 1$ coefficients $\beta_0, \beta_1,\dots,\beta_p$.

My question is, how does beta $\beta$ fit into the estimate of $f(X)$ and therefore $Y$ when I would normally associate beta $\beta$ with something that is not linear or am I reading the symbol incorrectly? Is this referring to the angle at which the line positively or negatively slopes?

Sorry if this is a poorly written question and I am a little nervous about posting on here.

Community
  • 1
tcratius
  • 105
  • 5
  • What do you mean by "I would normally associate beta β with something that is not linear"? – Tim Oct 23 '18 at 06:56
  • In the sense that there is normal distributions, uniform, beta. This is probably where my lack of knowledge of stats is letting me down. But then you also have alpha and beta with hypothesis testing (0 - 1) which made me think maybe it has something to do with the angle of the line and sigh, I now that I remember beta is the slope. I don't know who termed basic statistics "basic", it is anything but basic. – tcratius Oct 23 '18 at 11:56
  • Let us consider the one-dimensional situation, i.e. there is only $X_1$. Then we assume that the function looks like $Y = f(X) = \beta_0 + \beta_1X$. By 'associating something nonlinear', do you mean that the function should have a different form like $f(X) = \beta_0 + \beta_1 \cdot X + \beta_2\cdot X^2 + ...$ or so? – Fabian Werner Oct 23 '18 at 12:10
  • Sadly there are only so many Greek letters to go around, so they all end up meaning dozens of things. – jwimberley Oct 23 '18 at 12:13
  • I think @jwimberley hit the nail on the head. Is it ok to ask this instead: Ok, so if I have this right, $ β0,β1,…,βp $ is where the line intercepts $ X $and $ Y $ The values of $ X1, X2 ... Xp $ are not needed for the estimate than what does it mean by $ p + 1 $ coefficients $ β0,β1,…,βp $ ? I can re-edit my question above. – tcratius Oct 23 '18 at 12:18

1 Answers1

0

Your question is not completely clear for me, but since this is a question about notation, let me explain it for you.

We start with some predicted variable $Y$ and $p$ features $X = (X_1,X_2,\dots,X_p)$ that we are going to use to predict $Y$. What we are trying to achieve is to find some function $f$ of $X$ such that $Y = f(X)$. More precisely, we are looking for something like $Y \approx f(X)$, since it can be assumed that the data will be noisy and we won't find a perfect fit that describes $Y$ exactly.

In the quoted paragraph, as an example of function $f$, a linear function (i.e. $y = ax + b$ by definition) is used:

$$ f(X) = \beta_0 + \beta_1 X_1 + \beta_2 X_ 2 + \beta_p X_p $$

It has $p+1$ parameters: the intercept $\beta_0$ and $p$ weights $\beta_1,\beta_2,\dots,\beta_p$ for each of the features $X_1,X_2,\dots,X_p$.

To obtain the result, you simply substitute the $Y$ and $X_1,X_2,\dots,X_p$ with relevant vectors of numbers. $\beta_0,\beta_1,\dots,\beta_p$ are unknown parameters we need to estimate. There are different ways of finding the parameters, e.g. by maximizing the likelihood function, or minimizing some loss function.

All of the above symbols, $Y,X,\beta,f$, are arbitrary and if you wish you may use anything you want for them, the above choice is just a popular convention, the symbols by themselves have no special meaning.

Tim
  • 108,699
  • 20
  • 212
  • 390
  • Ah huh, so when I used the lm() function in R then the function is estimating the intercept $β0$ and $p$ weights comprised of $β1,β2,…,βp$ for the variable $X$ = ${X1, X2, ..., Xp}$. So essential you guys and ladies are the core of Data Science team! Thank you, that was helpful, at least I will be able to talk your language, I just wish I knew where I fit into the equation, only time will tell. – tcratius Oct 23 '18 at 13:04
  • @ConradThiele if this answers your question, please consider "accepting" the answer (the "v" button), so that others know that the question is "solved". – Tim Oct 23 '18 at 13:07