3

Lets assume we only have 2 variables, x and y, and our model (in R code) is lm(y~x). Given that any 2 points can be used to form a line, I know it doesn't make much sense to try to fit this model if I only have 2 observations.

What about the case where I have 3 observations? Would this model pass the laugh test?

Zach
  • 22,308
  • 18
  • 114
  • 158
  • 3
    What *specifically* is the problem with just two observations? Before you answer, check out the [sample-of-one confidence interval technique](http://stats.stackexchange.com/a/1836/): it can be applied to obtain a CI around the slope when only two points are available. – whuber Dec 12 '11 at 19:31

4 Answers4

4

If the data points are aggregates (for example state or country level averages) then it might not be a bad idea to fit a regression with just three points, since they'd represent many more underlying observations. Weighted least squares would probably be a better strategy than just OLS, but it would depend on the situation.

Gray
  • 517
  • 3
  • 10
  • Lets's sat the data are not aggregates: each point represents one point. – Zach Dec 12 '11 at 20:01
  • I think what @Mark Lamias and Gray are hinting at, Zach, is that your question is perhaps a little too indefinite to answer. If you could make it more specific (especially in the ways they suggest), it might be able to stand as a good (answerable) question. Otherwise it risks being closed. – whuber Dec 12 '11 at 22:36
  • [Ecological Fallacy](http://en.wikipedia.org/wiki/Ecological_fallacy) – Andy W Dec 13 '11 at 12:58
3

Of course it can be appropriate. In fact, it can be appropriate when you only have two points. The problem here is that you have neglected to indicate what you are trying to do with the regression analysis. The answer to that question will ultimately lead to the "correct" answer.

StatsStudent
  • 10,205
  • 4
  • 37
  • 68
2

There are some models that deal with few data points coming from aggregates.

See Kim HJ, Fay MP, Feuer EJ, Midthune DN. Permutation tests for joinpoint regression with applications to cancer rates. Stat Med 2000;19:335-51 (correction: 2001;20:655).

and NCI provide this software http://surveillance.cancer.gov/joinpoint/

munozedg
  • 41
  • 2
  • How would a *change point* regression model work with only 3 points? You would only have 2 possible lines! – Andy W Dec 13 '11 at 13:02
2

I think the question is how usefull the model will be. You probaly wont be able to describe the real world, makeing a model that will have a high chance to be at least poor to describe the world.

If you are useing R, you could play and figure what your model with 3 points will look like. for example:

#lets think that this 10000 points is the real world

set.seed(2)
x<-runif(10000)

#with a Deterministic Model
y.det<-2+5*x

#and this noise
y<-y.det+rnorm(10000)

#makeing a dataframe of the data
data<-data.frame(cbind(sample=1:10000,x=x,y=y))

#makeing a regression with 30 points to see if we can recover the
#model of the real world
sample.points<-sample(1:10000,30)
with(data[which(data$sample%in%sample.points),],summary(lm(y~x)))

#we get a good estimative of the real world that is y=2+5x+error
#but we can make a regression with 3 points too, a smaller sample of the
#real world

sample.points<-sample(1:10000,3)
with(data[which(data$sample%in%sample.points),],summary(lm(y~x)))

#hummm we got a higher error
#lets make many models and see what happens

#a dataframe to put our estimatives
model.estimates<-data.frame(matrix(c(rep(NA,3)),nrow=1,ncol=3,
dimnames=list(1,c("Points.Used","Intercep", "Slope"))))

#lets make one hundred models useing 3 points to estimate the model
#then lets use 5 points, 10,50,100 and 200 points to do de same thing
#and check if we can capture the shape of the real world from these models
for ( n in c(3,5,10,50,100,200) ) {
  for(i in 1:100) {
    model.estimates<-rbind(model.estimates,c(n,NA,NA))
    sample.points<-sample(1:10000,n)
    model.estimates[nrow(model.estimates),2]<-with(data[which(data$sample%in%sample.points),],coef(lm(y~x)))[1]         
    model.estimates[nrow(model.estimates),3]<-with(data[which(data$sample %in%sample.points),],coef(lm(y~x)))[2]
    }
  }

#removeing the first line, probaly there is a more elegant way to do the loop
#but i dont know
model.estimates<-model.estimates[-c(1),]

#now lets make a boxplot of the parameters we estimated to see what happens

par(mfrow=c(1,2))

boxplot(Intercep~Points.Used,data=model.estimates,main="Intercep",
xlab="Samples done",ylab="Parameter Value")

boxplot(Slope~Points.Used,data=model.estimates,main="Slope",
xlab="Samples done",ylab="Parameter Value")

Although you can make a model with 3 points, it will most likely be wrong :(, leading to bad decisions...

Augusto Ribas
  • 36
  • 4