Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [covariance]

Covariance is a quantity used to measure the strength and direction of the linear relationship between two variables. The covariance is unscaled, & thus often difficult to interpret; when scaled by the variables' SDs, it becomes Pearson's correlation coefficient.

Covariance is a quantity used to measure the strength and direction of the linear relationship between two variables. The covariance between $X$ and $Y$ is defined as $${\rm cov}(X,Y) = E \left[ \left( X-E(X) \right) \left( Y-E(Y) \right) \right] = E(XY) - E(X)E(Y) $$ Since the magnitude is difficult to interpret in isolation, the covariance is often scaled by the standard deviations of $X$ and $Y$ to produce the Pearson product-moment correlation coefficient.

1576 questions
262
votes
10 answers

How would you explain covariance to someone who understands only the mean?

...assuming that I'm able to augment their knowledge about variance in an intuitive fashion ( Understanding "variance" intuitively ) or by saying: It's the average distance of the data values from the 'mean' - and since variance is in square units,…
PhD
  • 13,429
  • 19
  • 45
  • 47
196
votes
7 answers

PCA on correlation or covariance?

What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
Random
  • 2,140
  • 3
  • 13
  • 8
126
votes
6 answers

How would you explain the difference between correlation and covariance?

Following up on this question, How would you explain covariance to someone who understands only the mean?, which addresses the issue of explaining covariance to a lay person, brought up a similar question in my mind. How would one explain to a…
pmgjones
  • 5,543
  • 8
  • 36
  • 36
107
votes
2 answers

What is covariance in plain language?

What is covariance in plain language and how is it linked to the terms dependence, correlation and variance-covariance structure with respect to repeated-measures designs?
abc
  • 1,747
  • 3
  • 17
  • 32
76
votes
8 answers

How and why do normalization and feature scaling work?

I see that lots of machine learning algorithms work better with mean cancellation and covariance equalization. For example, Neural Networks tend to converge faster, and K-Means generally gives better clustering with pre-processed features. I do not…
erogol
  • 1,427
  • 1
  • 15
  • 26
73
votes
5 answers

Covariance and independence?

I read from my textbook that $\text{cov}(X,Y)=0$ does not guarantee X and Y are independent. But if they are independent, their covariance must be 0. I could not think of any proper example yet; could someone provide one?
Flying pig
  • 5,689
  • 11
  • 32
  • 31
62
votes
2 answers

What does the inverse of covariance matrix say about data? (Intuitively)

I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?" Edit: Thanks for replies After taking some great courses, I'd like to add some points: It is measure of information,…
Arya
  • 873
  • 1
  • 7
  • 8
60
votes
5 answers

Is every covariance matrix positive definite?

I guess the answer should be yes, but I still feel something is not right. There should be some general results in the literature, could anyone help me?
Jingjings
  • 1,173
  • 1
  • 9
  • 13
53
votes
4 answers

Is a sample covariance matrix always symmetric and positive definite?

When computing the covariance matrix of a sample, is one then guaranteed to get a symmetric and positive-definite matrix? Currently my problem has a sample of 4600 observation vectors and 24 dimensions.
Morten
  • 918
  • 1
  • 9
  • 11
45
votes
4 answers

Why does inversion of a covariance matrix yield partial correlations between random variables?

I heard that partial correlations between random variables can be found by inverting the covariance matrix and taking appropriate cells from such resulting precision matrix (this fact is mentioned in http://en.wikipedia.org/wiki/Partial_correlation,…
michal
  • 1,138
  • 3
  • 11
  • 14
40
votes
7 answers

Why shouldn't the denominator of the covariance estimator be n-2 rather than n-1?

The denominator of the (unbiased) variance estimator is $n-1$ as there are $n$ observations and only one parameter is being estimated. $$ \mathbb{V}\left(X\right)=\frac{\sum_{i=1}^{n}\left(X_{i}-\overline{X}\right)^{2}}{n-1} $$ By the same token I…
MYaseen208
  • 2,379
  • 7
  • 32
  • 46
36
votes
1 answer

Covariance of a random vector after a linear transformation

If $\mathbf {Z}$ is random vector and $A$ is a fixed matrix, could someone explain why $$\mathrm{cov}[A \mathbf {Z}]= A \mathrm{cov}[\mathbf {Z}]A^\top.$$
user92612
  • 695
  • 1
  • 5
  • 10
29
votes
5 answers

A measure of "variance" from the covariance matrix?

If the data is 1d, the variance shows the extent to which the data points are different from each other. If the data is multi-dimensional, we'll get a covariance matrix. Is there a measure that gives a single number of how the data points are…
dontloo
  • 13,692
  • 7
  • 51
  • 80
26
votes
4 answers

How to ensure properties of covariance matrix when fitting multivariate normal model using maximum likelihood?

Suppose I have the following model $$y_i=f(x_i,\theta)+\varepsilon_i$$ where $y_i\in \mathbb{R}^K$ , $x_i$ is a vector of explanatory variables, $\theta$ is the parameters of non-linear function $f$ and $\varepsilon_i\sim N(0,\Sigma)$, where…
mpiktas
  • 33,140
  • 5
  • 82
  • 138
25
votes
3 answers

What does a non positive definite covariance matrix tell me about my data?

I have a number of multivariate observations and would like to evaluate the probability density across all variables. It is assumed that the data is normally distributed. At low numbers of variables everything works as I would expect, but moving to…
ralight
  • 353
  • 1
  • 3
  • 5
1
2 3
…
99 100