Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

381 questions
516
votes
3 answers

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix $\mathbf X$. How does it work? What is the…
amoeba
  • 93,463
  • 28
  • 275
  • 317
156
votes
1 answer

How to reverse PCA and reconstruct original variables from several principal components?

Principal component analysis (PCA) can be used for dimensionality reduction. After such dimensionality reduction is performed, how can one approximately reconstruct the original variables/features from a small number of principal…
amoeba
  • 93,463
  • 28
  • 275
  • 317
80
votes
3 answers

What is the intuition behind SVD?

I have read about singular value decomposition (SVD). In almost all textbooks it is mentioned that it factorizes the matrix into three matrices with given specification. But what is the intuition behind splitting the matrix in such form? PCA and…
SHASHANK GUPTA
  • 1,139
  • 2
  • 10
  • 17
51
votes
3 answers

How does centering make a difference in PCA (for SVD and eigen decomposition)?

What difference does centering (or de-meaning) your data make for PCA? I've heard that it makes the maths easier or that it prevents the first PC from being dominated by the variables' means, but I feel like I haven't been able to firmly grasp the…
Zenit
  • 1,586
  • 2
  • 17
  • 19
45
votes
7 answers

Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?

I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment, and in the lecture video from Andrew Ng, they do singular value decomposition instead of eigenvector decomposition of…
DongukJu
  • 593
  • 1
  • 5
  • 5
43
votes
1 answer

PCA and Correspondence analysis in their relation to Biplot

Biplot is often used to display results of principal component analysis (and of related techniques). It is a dual or overlay scatterplot showing component loadings and component scores simultaneously. I was informed by @amoeba today that he has…
ttnphns
  • 51,648
  • 40
  • 253
  • 462
39
votes
2 answers

Theory behind partial least squares regression

Can anyone recommend a good exposition of the theory behind partial least squares regression (available online) for someone who understands SVD and PCA? I have looked at many sources online and have not found anything that had the right combination…
ClarPaul
  • 1,130
  • 11
  • 18
35
votes
1 answer

What are the advantages of kernel PCA over standard PCA?

I want to implement an algorithm in a paper which uses kernel SVD to decompose a data matrix. So I have been reading materials about kernel methods and kernel PCA etc. But it still is very obscure to me especially when it comes to mathematical…
CyberPlayerOne
  • 2,009
  • 3
  • 22
  • 30
35
votes
5 answers

How do I use the SVD in collaborative filtering?

I'm a bit confused with how the SVD is used in collaborative filtering. Suppose I have a social graph, and I build an adjacency matrix from the edges, then take an SVD (let's forget about regularization, learning rates, sparsity optimizations, etc),…
Vishal
  • 1,101
  • 3
  • 12
  • 19
33
votes
1 answer

Dimensionality reduction (SVD or PCA) on a large, sparse matrix

/edit: Further follow up now you can use irlba::prcomp_irlba /edit: following up on my own post. irlba now has "center" and "scale" arguments, which let you use it to calculate principle components, e.g: pc <- M %*% irlba(M, nv=5, nu=0,…
Zach
  • 22,308
  • 18
  • 114
  • 158
31
votes
2 answers

How to compute SVD of a huge sparse matrix?

What is the best way to compute singular value decomposition (SVD) of a very large positive matrix (65M x 3.4M) where data is extremely sparse? Less than 0.1% of the matrix is non zero. I need a way that: will fit into memory (I know that online…
Sonia
  • 413
  • 1
  • 4
  • 5
30
votes
1 answer

SVD of correlated matrix should be additive but doesn't appear to be

I'm just trying to replicate a claim made in the following paper, Finding Correlated Biclusters from Gene Expression Data, which is: Proposition 4. If $X_{IJ}=R_{I}C^{T}_{J}$. then we have: i. If $R_{I}$ is a perfect bicluster with additive model,…
zzk
  • 697
  • 6
  • 14
30
votes
7 answers

Testing for linear dependence among the columns of a matrix

I have a correlation matrix of security returns whose determinant is zero. (This is a bit surprising since the sample correlation matrix and the corresponding covariance matrix should theoretically be positive definite.) My hypothesis is that at…
Ram Ahluwalia
  • 3,003
  • 6
  • 27
  • 38
28
votes
2 answers

Why PCA of data by means of SVD of the data?

This question is about an efficient way to compute principal components. Many texts on linear PCA advocate using singular-value decomposition of the casewise data. That is, if we have data $\bf X$ and want to replace the variables (its columns) by…
ttnphns
  • 51,648
  • 40
  • 253
  • 462
28
votes
3 answers

LSA vs. PCA (document clustering)

I'm investigation various techniques used in document clustering and I would like to clear some doubts concerning PCA (principal component analysis) and LSA (latent semantic analysis). First thing - what are the differences between them? I know that…
user1315305
  • 1,199
  • 4
  • 14
  • 15
1
2 3
…
25 26