Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [matrix]

A matrix (plural matrices) is a rectangular array of numbers, symbols, or expressions arranged in rows and columns. The individual items in a matrix are called its elements or entries.

Matrices are rectangular arrays of numbers. A major application of matrices is to represent linear transformations:

A matrix is a concise and useful way of uniquely representing and working with linear transformations. In particular, every linear transformation can be represented by a matrix and every matrix corresponds to a unique linear transformation. The matrix, and its close relative the determinant, are extremely important concepts in linear algebra and were first formulated by Sylvester (1851) and Cayley.

Excerpt reference: Wikipedia (adapted).

849 questions
516
votes
3 answers

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix $\mathbf X$. How does it work? What is the…
amoeba
  • 93,463
  • 28
  • 275
  • 317
187
votes
10 answers

Why the sudden fascination with tensors?

I've noticed lately that a lot of people are developing tensor equivalents of many methods (tensor factorization, tensor kernels, tensors for topic modeling, etc) I'm wondering, why is the world suddenly fascinated with tensors? Are there recent…
Y. S.
  • 1,237
  • 3
  • 9
  • 14
129
votes
6 answers

Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?

For a given data matrix $A$ (with variables in columns and data points in rows), it seems like $A^TA$ plays an important role in statistics. For example, it is an important part of the analytical solution of ordinary least squares. Or, for PCA, its…
Alec
  • 2,185
  • 4
  • 17
  • 14
89
votes
1 answer

What correlation makes a matrix singular and what are implications of singularity or near-singularity?

I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My question here is what would you consider a "highly"…
Error404
  • 1,261
  • 2
  • 13
  • 18
80
votes
3 answers

What is the intuition behind SVD?

I have read about singular value decomposition (SVD). In almost all textbooks it is mentioned that it factorizes the matrix into three matrices with given specification. But what is the intuition behind splitting the matrix in such form? PCA and…
SHASHANK GUPTA
  • 1,139
  • 2
  • 10
  • 17
62
votes
2 answers

What does the inverse of covariance matrix say about data? (Intuitively)

I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?" Edit: Thanks for replies After taking some great courses, I'd like to add some points: It is measure of information,…
Arya
  • 873
  • 1
  • 7
  • 8
60
votes
5 answers

Is every covariance matrix positive definite?

I guess the answer should be yes, but I still feel something is not right. There should be some general results in the literature, could anyone help me?
Jingjings
  • 1,173
  • 1
  • 9
  • 13
57
votes
9 answers

Reference book for linear algebra applied to statistics?

I have been working in R for a bit and have been faced with things like PCA, SVD, QR decompositions and many such linear algebra results (when inspecting estimating weighted regressions and such) so I wanted to know if anyone has a recommendation on…
Palace Chan
  • 759
  • 3
  • 9
  • 17
39
votes
3 answers

How to take derivative of multivariate normal density?

Say I have multivariate normal $N(\mu, \Sigma)$ density. I want to get the second (partial) derivative w.r.t. $\mu$. Not sure how to take derivative of a matrix. Wiki says take the derivative element by element inside the matrix. I am working…
user1061210
  • 1,005
  • 3
  • 13
  • 19
33
votes
1 answer

If I generate a random symmetric matrix, what's the chance it is positive definite?

I got a strange question when I was experimenting some convex optimizations. The question is: Suppose I randomly (say standard normal distribution) generate a $N \times N$ symmetric matrix, (for example, I generate upper triangular matrix, and fill…
Haitao Du
  • 32,885
  • 17
  • 118
  • 213
32
votes
1 answer

Bound for Arithmetic Harmonic mean inequality for matrices?

NOTE: This question has originally been posted in MSE, but it did not generate any interest. It was first posted there, because the question itself is a pure matrix-algebra question. Nevertheless, since the motive has to do with statistics and…
Alecos Papadopoulos
  • 52,923
  • 5
  • 131
  • 241
26
votes
3 answers

Column-wise matrix normalization in R

I would like to perform column-wise normalization of a matrix in R. Given a matrix m, I want to normalize each column by dividing each element by the sum of the column. One (hackish) way to do this is as follows: m / t(replicate(nrow(m),…
mavam
  • 690
  • 2
  • 7
  • 14
26
votes
1 answer

Multivariate normal posterior

This is a very simple question but I can't find the derivation anywhere on the internet or in a book. I would like to see the derivation of how one Bayesian updates a multivariate normal distribution. For example: imagine that $$…
Alex
  • 743
  • 1
  • 9
  • 16
22
votes
5 answers

Why bother with low rank approximations?

If you have a matrix with n rows and m columns, you can use SVD or other methods to calculate a low-rank approximation of the given matrix. However, the low rank approximation will still have n rows and m columns. How can low-rank-approximations be…
Zach
  • 22,308
  • 18
  • 114
  • 158
22
votes
4 answers

Textbooks on Matrix Calculus?

See this question on Math SE. Short story: I read The Elements of Statistical Learning and got frustrated when I was trying to verify some of the results, e.g., given $$\text{RSS}(\beta) =…
Clarinetist
  • 3,761
  • 3
  • 25
  • 70
1
2 3
…
56 57