Is it possible to perform a regression analysis without a dependent variable?

Question

I have a data set with 5 independent variables. Is it possible to do a regression analysis without the presence of a dependent variable?

> names(bb)
[1] "BB1"  "BB2"  "BB3" "BB4"   "BB5" 

> x1<-bb$BB1
> x2<-bb$BB2
> x3<-bb$BB3
> x4<-bb$BB4
> x5<-bb$BB5

> OLS<-lm(y~x1+x2+x3+x4+x5, data = bb)

> summary(OLS)

Call:
  lm(formula = y ~ x1 + x2 + x3 + x4 + x5, data = bb)

Residuals:
  Min      1Q  Median      3Q     Max 
-168.79  -74.18  -19.31   53.45  222.43 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)  
(Intercept) 151.2558    69.6001   2.173   0.0325 *
x1            0.3859     1.4976   0.258   0.7972  
x2            0.2746     0.1940   1.416   0.1605  
x3           -2.6899     2.5056  -1.074   0.2861  
x4           -2.2511     1.1785  -1.910   0.0595 .
x5           -0.3369     1.6231  -0.208   0.8360  
---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 95.18 on 85 degrees of freedom
Multiple R-squared:  0.09186,   Adjusted R-squared:  0.03844 
F-statistic:  1.72 on 5 and 85 DF,  p-value: 0.1388

Answer 1

No, it isn't possible. And R gives an error message when you try. What appears to have happened in your case is that y was defined somewhere else and you forgot that it was there.

E.g.

set.seed(1075)
x1 <- rnorm(100)
x2 <- rnorm(100)

df1 <- as.data.frame(cbind(x1,x2))

m1 <- lm(y~x1+x2, data = df1)

gives an error. But if we add

y <- rnorm(100)

m1 <- lm(y~x1+x2, data = df1)

there is no error. This might be regarded as good or bad behavior by R, but it has nothing to do with regression.

Answer 2

One constructive way to understand the question is to interpret it as asking for a way to find a nontrivial linear combination of variables that is as close to zero as possible.

We have to impose some kind of nontriviality condition, for otherwise the solution given by all zero coefficients is always available--and useless. A reasonable condition to impose is to set some kind of lower bound on the overall size of the coefficients.

For instance, suppose we were to require the sum of squares of coefficients to be no less than unity. Letting $X$ be, as usual, the $n\times p$ matrix of $n$ observations of $p$ variables (perhaps centered, perhaps normalized), the problem can be formulated

$$\text{Minimize } ||X\beta||^2 \text{ subject to } ||\beta||^2 = 1.$$

This problem was solved in my answer at https://stats.stackexchange.com/a/74328/919: $X\beta$ must be a principal component of $X$ with a smallest eigenvalue.

Such a "regression," as indicated in that thread, finds a way to express any one of the $p$ variables as a linear combination of the remaining $p-1$ variables, perhaps with some error. However, it does not single out any of the variables as a "response" and thereby avoids the asymmetry imposed by Least Squares regression. Geometrically, it amounts to finding the best-fitting $p-1$-dimensional hyperplane to the $p$-dimensional point cloud determined by the observations (in the least squares sense).

Here, to illustrate, is a "no-response regression" of $100$ observations of $p=2$ variables, displayed as a point cloud. The model matrix was centered. The solution is shown as the red line. For reference, shown in blue in the background is the least squares fit of the model $Y \sim X$. Unlike the "no-response" fit, it exhibits regression to the mean due to the asymmetrical treatment of $X$ and $Y$.

Answer 3

In short, No. Regression analysis is supervised in nature by definition (and by implementation as well). To help you understand why, let me describe, in simple words, what regression analysis does.

A regression analysis is essentially finding coefficients for variables in a function. We do not know what is this "function" beforehand, and therefore we try to find the function by guessing it. The way we find a close match with the actual function is by minimising the difference with actual value (i.e. the known value of dependent variable) and what we are getting as the result of our guess. Therefore, we have to have a way to know this difference i.e. error. Without dependent variable, this is impossible. Therefore, a regression analysis is not possible without dependent variable.

If you do not want to use dependent variable, can you elaborate what you are attempting to achieve? There are many unsupervised algorithms (i.e. to work without dependent variable). But a bit more details is needed before such an algorithm can be suggested.

Answer 4

I am assuming you mean a regression that would use no explanatory variables, which seems to be the {x1,...,x5} in your set.

This would mean creating a model of the type $\hat y = \alpha$, which means your response variable $Y$ is not dependent on the values of any other variable (including itself); all predicted values $\hat y$ will be the same, $\alpha$. The observed values $y$ will thus equal $\alpha + \varepsilon$, where $\varepsilon$ is the residual error of your model.

This may sound like a silly regression model, but it is essentially what happens when you use a summary statistic like the mean: you're representing all values of a dataset by one single number. Statistical procedures such as ANOVA compare this simple model $\hat y = \bar y$ with a more complex model such as $\hat y = \alpha + X\beta$ in order to measure the gain in error reduction provided by the more complex model.