22

What would be a "reasonable" minimal number of observations to look for a trend over time with a linear regression? what about fitting a quadratic model?

I work with composite indices of inequality in health (SII,RII), and have only 4 waves of the survey, so 4 points (1997,2001,2004,2008).

I am not statistician, but I have the intuitive impression 4 points are not sufficient. Do you have an answer, and/or references ?

Thanks a lot,

Françoise

Peter Flom
  • 94,055
  • 35
  • 143
  • 276
Francoise
  • 221
  • 1
  • 2
  • 3
  • 4
    The usual rule of thumb is 10 points for each independent variable. – Peter Flom Sep 23 '12 at 12:26
  • 1
    How are your indices measured? If they include estimates of variability, then two could be enough (using a t-test or its analog). The basic statistical principle that applies here is that when random variation is an unlikely explanation of what you are observing, then you have the right to attribute any apparent trend to non-random causes. When the trend is strong, very few data values may be needed to come to such a conclusion, all generic "rules of thumb" notwithstanding. – whuber Sep 24 '12 at 15:41

2 Answers2

14

Peters rule of thumb of 10 per covariate is a reasonable rule. A straight line can be fit perfectly with any two points regardless of the amount of noise in the response values and a quadratic can be fit perfectly with just 3 points. So clearly in almost any circumstance it would be proper to say that 4 points is insufficient. However, like most rules of thumb it does not cover every situation. Cases where the noise term in the model has a large variance will require more samples than a similar case where the error variance is small.

The required number of sample points does depend on objects. If you are doing exploratory analysis just to see if one model (say linear in a covariate) looks better than another (say a quadratic function of the covariate) less than 10 points may be enough. But if you want very accurate estimates of the correlation and regression coefficients for the covariates you could need more than 10 per covariate. An accuracy of prediction criterion could require even more samples than accurate parameter estimates. Note that the variance of the estimates and prediction all involve the variance of the models error term.

Michael R. Chernick
  • 39,640
  • 28
  • 74
  • 143
  • Good points, Michael; I was trying to keep it simple. :-). Given the original question's subject, I would be very surprised if less than 10 points were adequate. Measures of inequality in health seem likely to have a lot of error, and relationships with time are unlikely to be highly linear. Do you know of any articles on this? It's an interesting topic that comes up a lot. – Peter Flom Sep 23 '12 at 13:09
  • @PeterFlom I dont. I would look at van Belle'a book on statistical rules of thumb to see if he uses a rule like the one you mentioned. The nice thing about his book is that he explains the rationale behind every rule. I agree with you that a rule saying take at least 10 per covariate is pretty good and using less would be rarely safe except in some exploratory cases. In the health sciences where I work the noise term seems to always be large but perhaps some tightly controlled physics or engineering experiments could have very precise measurements and hence small random error. – Michael R. Chernick Sep 23 '12 at 13:21
  • I was just trying to point out the possibility of small noise leading to needing fewer than 10 points even though the possibility might be remote. – Michael R. Chernick Sep 23 '12 at 13:23
  • yes, I agree. And it could well be the case in physics, say, or any area where a very high $R^2$ is expected and theory is strong and error is small. – Peter Flom Sep 23 '12 at 13:29
  • +1, good info, but it's also worth mentioning that if your estimator is unbiased, you can have a saturated model & still have an estimate of the parameters, if that's all you need. You won't have an estimate of the variability or be able to do inference. However, in some cases where there are many effects to estimate & data are sufficiently hard to get, saturated models are sometimes used. So eg, in this case, you could get an estimate of the function w/ the quadratic w/ 3 points. I don't necessarily mean that it's a good thing, but that is the real lower bound & the reason why. – gung - Reinstate Monica Sep 23 '12 at 19:20
  • @gung nice point I was hesitant to suggest doing anything like that. But you do point to a situation where it might be done and it makes my point that lower sample sizes are sometimes okay if accurate estimation is not a goal. – Michael R. Chernick Sep 23 '12 at 21:07
0

As mentioned by Michael a good rule of thumb is 10, you can also check it out on wiki https://en.wikipedia.org/wiki/One_in_ten_rule

Gedas Miksenas
  • 101
  • 1