0

a) If our training data x in all reals is of the form y = beta* x^4

I know that y (labels) are not linearly related to our examples, but can linear regression still be used here?

b) how about the same if y = 3^beta1 * x^ beta2

c) if x is in R^2 and y = (beta * x_1 * x_2^4) and we want to learn beta.

betas are the coefficients we are trying to learn

Adam Rainah
  • 1
  • 1
  • I don't follow: in both cases $y$ is a linear function of the explanatory variables. The latter are the $x^4$ in the first case and $3x^5$ in the second case. – whuber Oct 23 '21 at 17:38
  • I updated the part b. My confusion is do we strictly need linear powers of x wrt to y for linear regression to be usable? In part a) beta is the slope coefficient (for instance) and it is linear but x (our independent data) is not. – Adam Rainah Oct 23 '21 at 17:42
  • Based on your comment I am a bit confused. y and x^4 are linearly related? Wouldn't the plot be a parabola between them? – Adam Rainah Oct 23 '21 at 17:43
  • 1
    (1) Please see https://stats.stackexchange.com/questions/148638. To answer part (b) requires that you specify your model fully: specifically, *how do you model the potential errors on the right hand side?* If they are *multiplicative* and all data are positive, then taking logarithms will do the trick; otherwise, taking logs might be invalid. (2) Yes, clearly $y$ and $x^4$ are linearly related in the equation $y=\beta x^4.$ However, $y$ and $x$ are *not* linearly related. But that doesn't matter, because given $x$ you can compute $x^4$ and use it in your model. – whuber Oct 23 '21 at 17:45
  • I'd say we can ignore the potential errors. The core of what I am trying to understand is what 2 entities should have linear relationship for us to e able to use linear regression. I added a part c). What would happen if our data was in R^2 like that? – Adam Rainah Oct 23 '21 at 17:51
  • 1) What is your justification for ignoring the error terms? // 2) Why do you describe them as “potential” errors? // 3) Why don’t you have any intercept terms? // 4) You will answer your A and C by [watching MathematicalMonk (Jeffrey Miller).](https://m.youtube.com/watch?v=rVviNyIR-fI) // 5) @whuber I interpret all three equations to implicitly use $\mathbb E [y]$ rather than just $y$, but wouldn’t we be just as in trouble with multiplicative errors for $y_i = \beta x_i^4 \epsilon_i$ as for additive errors in $y_i=3^{\beta_1}x_i^{\beta_2}+\epsilon_i?$ – Dave Oct 23 '21 at 19:04
  • @Dave I'm unsure what you mean. But taking logs of $y =\beta x^4\epsilon$ gives $\log(y)=\log(\beta)+4\log(x) + \log(\epsilon),$ which is a perfectly fine linear regression model (giving $\hat\beta = $ the mean of $\log(y_i)$ minus $4$ times the mean of $\log(x_i)$). – whuber Oct 23 '21 at 21:31
  • 1
    @whuber But what if that one has an additive error term? – Dave Oct 23 '21 at 21:38
  • @Dave the intercepts were missing because these are custom functions where I am debating if presented in this form can we learn alphas and betas. Makes sense? From the thread it sounds like, only after some transformations these can be learnt if we don't do any of those, lin reg can't directly be applied – Adam Rainah Oct 23 '21 at 23:45
  • 1
    You’re still missing the point whuber is trying to convey about the error term, however. Regression doesn’t just predict points; those predicted points are conditional expectations. If you assume an additive error term in B, which replacing $y$ with $\mathbb E[y]$ implies, the log transform is questionable. – Dave Oct 24 '21 at 00:29
  • 1
    The term "linear regression" usually means that the parameters $\beta_i$ enter linearly in the model, not the predictors $x_i$. – cdalitz Oct 24 '21 at 09:27

0 Answers0