Questions tagged [svd]

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics.

In linear algebra, the singular-value decomposition (SVD) is a factorization of a real or complex matrix.

Formally, the singular value decomposition of an $m \times n$ real or complex matrix $M$ is a factorisation of the form $UAV^*$ where $U$ is an $m\times m$ real or complex unitary matrix, $A$ is an $m\times n$ rectangular diagonal matrix with non-negative real numbers on the diagonal, and $V$ is an $n\times n$ real or complex unitary matrix.

The singular-value decomposition can be computed using the following observations:

  • The left-singular vectors of $M$ are a set of orthonormal eigenvectors of $MM^*$.
  • The right-singular vectors of $M$ are a set of orthonormal eigenvectors of $M^*M$.
  • The non-zero singular values of $M$ (found on the diagonal entries of $A$) are the square roots of the non-zero eigenvalues of both $M^*M$ and $MM^*$.

Source: Wikipedia.

1448 questions
444
votes
4 answers

What is the intuitive relationship between SVD and PCA?

Singular value decomposition (SVD) and principal component analysis (PCA) are two eigenvalue methods used to reduce a high-dimensional data set into fewer dimensions while retaining important information. Online articles say that these methods are…
193
votes
7 answers

Intuitively, what is the difference between Eigendecomposition and Singular Value Decomposition?

I'm trying to intuitively understand the difference between SVD and eigendecomposition. From my understanding, eigendecomposition seeks to describe a linear transformation as a sequence of three basic operations ($P^{-1}DP$) on a vector: Rotation…
user541686
  • 13,374
  • 16
  • 50
  • 94
177
votes
7 answers

What is the difference between "singular value" and "eigenvalue"?

I am trying to prove some statements about singular value decomposition, but I am not sure what the difference between singular value and eigenvalue is. Is "singular value" just another name for eigenvalue?
61
votes
2 answers

What do eigenvalues have to do with pictures?

I am trying to write a program that will perform OCR on a mobile phone, and I recently encountered this article : Can someone explain this to me ?
60
votes
4 answers

How unique are $U$ and $V$ in the Singular Value Decomposition?

According to Wikipedia: A common convention is to list the singular values in descending order. In this case, the diagonal matrix $\Sigma$ is uniquely determined by $M$ (though the matrices $U$ and $V$ are not). My question is, are $U$ and $V$…
55
votes
2 answers

Why does the spectral norm equal the largest singular value?

This may be a trivial question yet I was unable to find an answer: $$\left \| A \right \| _2=\sqrt{\lambda_{\text{max}}(A^{^*}A)}=\sigma_{\text{max}}(A)$$ where the spectral norm $\left \| A \right \| _2$ of a complex matrix $A$ is defined as…
mathemage
  • 760
  • 1
  • 6
  • 14
40
votes
6 answers

How can you explain the Singular Value Decomposition to non-specialists?

In two days, I am giving a presentation about a search engine I have been making the past summer. My research involved the use of singular value decompositions, i.e., $A = U \Sigma V^T$. I took a high school course on Linear Algebra last year, but…
38
votes
1 answer

Relationship between eigendecomposition and singular value decomposition

Let $A \in \mathbb{R}^{n\times n}$ be a real symmetric matrix. Please help me clear up some confusion about the relationship between the singular value decomposition of $A$ and the eigen-decomposition of $A$. Let $A = U\Sigma V^T$ be the SVD of…
30
votes
3 answers

How is the null space related to singular value decomposition?

It is said that a matrix's null space can be derived from QR or SVD. I tried an example: $$A= \begin{bmatrix} 1&3\\ 1&2\\ 1&-1\\ 2&1\\ \end{bmatrix} $$ I'm convinced that QR (more precisely, the last two columns of Q) gives the null space: $$Q=…
whitegreen
  • 1,493
  • 1
  • 14
  • 23
27
votes
4 answers

Why does SVD provide the least squares and least norm solution to $ A x = b $?

I am studying the Singular Value Decomposition and its properties. It is widely used in order to solve equations of the form $Ax=b$. I have seen the following: When we have the equation system $Ax=b$, we calculate the SVD of A as $A=U\Sigma V^T$.…
Ufuk Can Bicici
  • 2,896
  • 2
  • 25
  • 49
25
votes
4 answers

Why are singular values always non-negative?

I have read that the singular values of any matrix $A$ are non-negative (e.g. wikipedia). Is there a reason why? The first possible step to get the SVD of a matrix $A$ is to compute $A^{T}A$. Then the singular values are the square root of the…
Louis
  • 361
  • 1
  • 3
  • 7
24
votes
8 answers

Relation between Cholesky and SVD

When we have a symmetric matrix $A = LL^*$, we can obtain L using Cholesky decomposition of $A$ ($L^*$ is $L$ transposed). Can anyone tell me how we can get this same $L$ using SVD or Eigen decomposition? Thank you.
23
votes
3 answers

How does the SVD solve the least squares problem?

How do I prove that the least-squares solution for $$\text{minimize} \quad \|Ax-b\|_2$$ is $A^{+} b$, where $A^{+}$ is the pseudoinverse of $A$?
Elnaz
  • 609
  • 1
  • 6
  • 13
23
votes
7 answers

Understanding the singular value decomposition (SVD)

Please, would someone be so kind and explain what exactly happens when Singular Value Decomposition is applied on a matrix? What are singular values, left singular, and right singular vectors? I know they are matrices of specific form, I know how to…
Celdor
  • 611
  • 5
  • 15
23
votes
4 answers

Strang's proof of SVD and intuition behind matrices $U$ and $V$

In lecture 29 of MIT 18.06, Professor Gilbert Strang "proves" the singular value decomposition (SVD) by assuming that we can write $A = U\Sigma V^T$ and then deriving what $U$, $\Sigma$, and $V$ must be based on the eigendecomposition of $$ AA^T =…
samlaf
  • 830
  • 1
  • 6
  • 11
1
2 3
96 97