3

Nikolaenko et al. claims that in ridge regression $A\beta=b$, where $A=X^TX+\lambda I \in R^{d\times d}$ and $b=X^Ty \in R^d$ (page 3), it can be decomposed into:

$$A=\sum\limits_{i=1}^{n}A_i+\lambda I \text{ and } b=\sum\limits_{i=1}^{n}b_i$$

where $A_i=x_ix_i^T$ and $b_i=y_ix_i$ (page 4)

However, I think it's not true because $X^TX\neq\sum\limits_{i=1}^nx_ix_i^T$. Consider a simple example:

They are very good scholars so I assume I either miss something in the paper or had wrong conclusions somewhere.

So, is the decomposition valid?

xtt
  • 724
  • 1
  • 6
  • 10
  • What do you think $x_i$ are? They should be rows of $X$ written as columns (i.e. data points written as vectors). So each term in the sum is a matrix of the same size as $A$. – amoeba Jan 18 '17 at 10:26
  • Shouldn't $x_i$ be a vector of $x_{i1}, x_{i2}, ..., x_{ik}$(k being the number of columns)? Then $x_ix_i^T$ should also also be vector of the same length k, no? – xtt Jan 18 '17 at 10:47
  • 1
    I see now. $x_ix_i^T$ refers to the outer product, which is a matrix of the same size as $A$. Thanks – xtt Jan 18 '17 at 10:51
  • I believe this is addressed at http://stats.stackexchange.com/questions/220243. – whuber Jan 18 '17 at 14:39

0 Answers0