Why is it necessary to have the number of independent variables less than the number of data points in a regression setup? If suppose, it is the other way round, i.e. the number of independent variables far exceed the number of observations, then what should be the way out?
Asked
Active
Viewed 496 times
1 Answers
12
Consider one of the simplest possible case. One independent variable (so 2 parameters, including the constant). One data point.
Plot your one data point
Draw a straight line through that one point. Draw a different straight line through the same point. Draw a third one. ... and so on.
$\hspace{3cm}$
They all fit the data perfectly. Which one are you going to pick?
The problem is similar with two points and two predictors (with a plane through two points it's a bit like trying to rest a sheet of plywood on top of a picket fence - stable in one direction, but it's a see-saw in the other).
Glen_b
- 257,508
- 32
- 553
- 939