0

I am working with a standard multivariate linear regression model ($Y = X \lambda + \epsilon$, $Y$ and $\epsilon$ of length $n$, $X$ is $n$ x $m$, $\lambda$ of length $m$). $X$ is said to be "column-standardized", which I take to mean each column has mean $0$ and variance $1$, and each $\epsilon_i \sim N(0, \sigma^2)$. I am looking for the distribution of $X^T \epsilon$. Specifically, I have two questions.

  1. If $X$ is column-standardized, does that mean each $X_{ij} \sim N(0, 1)$? It seems like it should, since this is just a z-transformation, but then I'm curious as to why the procedure needs to be done on columns separately. Is it because we assume that each column comes from a unique distribution with different original variances?

  2. If every column in standardized, then each entry $(X^T\epsilon)_j = \sum_{i=1}^{n} X_{ij} \epsilon_i$, which I think is a linear combination of standard normal distributions where the coefficients are also normal. Sums of normals are normal, and so I think that $(X^T\epsilon)_j \sim N(0, \_)$, but I'm having a hard time figuring out the variance. I don't think it's $n\sigma^2$, but I can't figure out what else it would be.

Any help would be greatly appreciated!

Benjamin Doughty
  • 121
  • 4
  • You have a _product_ of two independent normal random variables in $X_ij\epsilon_i$, See e.g. _[here](https://stats.stackexchange.com/a/15986/6633)_ for the formula for the variance of the product. Then, use standard formulas for the variance of a sum of independent random variables. – Dilip Sarwate Jun 30 '18 at 15:55
  • 1
    Standardizing a variable does not make it normally distributed. – Alecos Papadopoulos Jun 30 '18 at 17:44

1 Answers1

1

If the $X$'s are assumed independent from the $\epsilon$'s, then simple algebra produces a nice looking formula, irrespective of distributional assumptions. In general notation,

$$\text{Var}(XY) = E(X^2Y^2) - [E(XY)]^2 = E(X^2)E(Y^2) - [E(X)E(Y)]^2$$

Using the symbols $\sigma^2$ and $\mu$ we have

$$\text{Var}(XY) = (\sigma^2_x +\mu_x^2)(\sigma^2_y+\mu_y^2) - \mu_x^2\mu_y^2$$

and simplifying and rearranging,

$$\text{Var}(XY) = \sigma^2_x\sigma^2_y + \mu_x^2\sigma^2_y+\mu_y^2\sigma^2_x$$

In case of zero means, this reduces to

$$\text{Var}(XY) = \sigma^2_x\sigma^2_y $$

Let's not forget that the fact that we obtained $\text{Var}(XY) = \text{Var}(X)\text{Var}(Y) $ depends not only on the independence assumption, but also on the zero-mean assumption.

Alecos Papadopoulos
  • 52,923
  • 5
  • 131
  • 241
  • This is essentially what Macro wrote in the link provided in my comment on the OP's question. – Dilip Sarwate Jun 30 '18 at 21:05
  • Thanks very much Alecos and Dilip. Following up -- so $E(X_{ij}\epsilon_i)=0$ and $Var(X_{ij}\epsilon_i)=\sigma^2$, but $X_{ij}\epsilon_i$ isn't necessarily the product of two normals. Do you have a sense of what kind of distribution it might follow? – Benjamin Doughty Jul 02 '18 at 04:28