10

A colleague of mine sent me this problem apparently making the rounds on the internet:

If $3 = 18, 4 = 32, 5 = 50, 6 = 72, 7 = 98$, Then, $10 =$ ?

The answer seems to be 200.

3*6  
4*8  
5*10  
6*12  
7*14  
8*16  
9*18  
10*20=200

When I do a linear regression in R:

data     <- data.frame(a=c(3,4,5,6,7), b=c(18,32,50,72,98))  
lm1      <- lm(b~a, data=data)  
new.data <- data.frame(a=c(10,20,30))  
predict  <- predict(lm1, newdata=new.data, interval='prediction')

I get: 

  fit      lwr      upr  
1 154 127.5518 180.4482  
2 354 287.0626 420.9374  
3 554 444.2602 663.7398

So my linear model is predicting $10 = 154$.

When I plot the data it looks linear... but obviously I assumed something that is not correct.

I'm trying to learn how to best use linear models in R. What is the proper way to analyze this series? Where did I go wrong?

Jeromy Anglim
  • 42,044
  • 23
  • 146
  • 250
Brett Phinney
  • 413
  • 1
  • 4
  • 11
  • 7
    *Ahem*. (i) The expression of the problem is nonsensical. How can 3=18? Surely the intent is something like $f(3) = 18$; (ii) if you can see enough to write $18=3\times 6$, $32=4\times 8$, etc., surely you can then see enough to split the second term in each of those ($6=3\times 2$, $8=4\times 2$, and so on) to then write: $18=3\times 3\times 2$, $32=4\times 4\times 2$, etc, and *instantly* spot the quadratic, $f(x) = 2x^2$. (You did the hard part, the next step is even simpler!) – Glen_b Jan 22 '14 at 00:17
  • 4
    Additionally, did the problem specify a minimum information content criterion on the answer? If I remember my math correctly, there are an uncountably infinite number of functions that fit these points, all giving different answers for $f(10)$. I'm not typically pedantic, but time-waster emails deserve it. – bright-star Jan 22 '14 at 02:14
  • 1
    @TrevorAlexander if you think this question is a waste of time, why bother to respond to it? Clearly some people find it interesting. – jwg Jan 22 '14 at 10:13
  • @jwg because [someone is wrong on the internet](http://imgs.xkcd.com/comics/duty_calls.png). ;) – bright-star Jan 22 '14 at 23:28

3 Answers3

23

A regression model, such as the one fit by lm() implicitly assumes that the underlying data generating process is probabilistic. You are assuming that the rule you are trying to model is deterministic. Therefore, there is a mismatch between what you are trying to do and the way you are trying to do it.

There are other software (i.e., not R) that is explicitly designed to find / fit the simplest function to deterministic data (an example would be Eureqa). There may be an R package for that (that I don't know of), but R is intended for statistical modeling of probabilistic data.

As for the answer that lm() gave you, it looks reasonable, and could be right. However, I gather the context in which this problem was presented strongly implied that it should be understood as deterministic. If that hadn't been the case, and you were wondering if the fit was reasonable, one thing you might notice is that the two extreme data points are above the regression line, while the middle data are all below it. This suggests a mis-specified functional form. This can also be seen in the residuals vs. fitted plot (plot(lm1, which=1):

enter image description here

As for the model fit by @AlexWilliams, it looks much better:

enter image description here

Scortchi - Reinstate Monica
  • 27,560
  • 8
  • 81
  • 248
gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
  • 17
    +1 The residual plot tells the story in such a way that one can't miss it. Indeed, it shows why the OP's 'looks linear' is often misleading - many curved functions can look 'nearly straight' if we only look at a few points not close to a turning point. *If you think it's linear, take that line out and see what's left over!* – Glen_b Jan 22 '14 at 00:24
  • 1
    Incredibly useful info! Thank you, I really appreciate it – Brett Phinney Jan 22 '14 at 00:59
  • 1
    This has absolutely nothing to do with the distinction between probabilistic and deterministic data. Linear regression would fit and extrapolate deterministic data if it were linear. It would fail to predict well for probabilistic data if the underlying model was quadratic. – jwg Jan 22 '14 at 10:12
  • 3
    @jwg: It has a lot to do with it. Or would you always fit a sequence of $n$ observations with an $(n-1)$th degree polynomial when no lower-degree polynomial gave a perfect fit? – Scortchi - Reinstate Monica Jan 22 '14 at 10:59
  • I don't think he's looking for a perfect fit. He's trying to understand why the extrapolated value is so far off. – jwg Jan 22 '14 at 11:18
22

The trend is quadratic not linear. Try:

lm1 <- lm(b~I(a^2), data=data)

Update: Here is the code.

data <- data.frame(a=c(3,4,5,6,7),b=c(18,32,50,72,98))
lm1 <- lm(b~I(a^2), data=data)
new.data <- data.frame(a=c(10,20,30))
predict(lm1, newdata = new.data, interval='prediction')

And output:

   fit  lwr  upr
1  200  200  200
2  800  800  800
3 1800 1800 1800
ahwillia
  • 2,406
  • 1
  • 14
  • 26
  • This answer seems a little circular to me: the whole point of the problem is to recognize the quadratic behavior. You correctly point out that once the quadratic behavior is specified, linear regression can find the coefficients. But in effect *you already did the crucial analysis* by the time you wrote down the first line of this answer. – whuber Jan 22 '14 at 16:57
  • 5
    @whuber - The question is why a linear model fails. It fails because the functional form isn't linear, it's quadratic. I wanted to give the answer simple and to the point. Gung's answer does a good job of going into the details, and shows how you can use the residual plots to come up with a better model. (I just did it on pen and paper.) I agree his answer is more detailed and complete and I have upvoted it. – ahwillia Jan 22 '14 at 17:33
13

I hesitate to add to the excellent answers given by Alex Williams and gung, but there is a further point that should I think be made. The question uses the phrases 'linear regression' and 'linear model', possibly suggesting that they mean the same. However, the usual meaning of 'linear regression' refers to the Classical Linear Regression Model (CLRM) in which 'linear' means 'linear in the parameters'. This is a condition on the parameters, not on the independent variables. So a quadratic model such as:

$$Y_i = \beta_1 + \beta_2X_i^2$$

is still linear in the sense of CLRM, because it is linear in the parameters $\beta_1$ and $\beta_2$. By contrast, the model:

$$Y_i = \beta_1 + \beta_2X_i$$

is linear in its parameters and also linear in $X_i$. Rather than calling it a linear model, a more precise statement is that it is linear in its parameters and has linear functional form. So it can be said that the series can be analysed by a model that is linear in its parameters, provided it has quadratic functional form (as shown by Alex Williams), but not by a model having linear functional form.

Adam Bailey
  • 1,602
  • 11
  • 20