Questions tagged [covariance]

Covariance is a quantity used to measure the strength and direction of the linear relationship between two variables. The covariance is unscaled, & thus often difficult to interpret; when scaled by the variables' SDs, it becomes Pearson's correlation coefficient.

Covariance is a quantity used to measure the strength and direction of the linear relationship between two variables. The covariance between $X$ and $Y$ is defined as $${\rm cov}(X,Y) = E \left[ \left( X-E(X) \right) \left( Y-E(Y) \right) \right] = E(XY) - E(X)E(Y) $$ Since the magnitude is difficult to interpret in isolation, the covariance is often scaled by the standard deviations of $X$ and $Y$ to produce the Pearson product-moment correlation coefficient.

1789 questions
298
votes
11 answers

How would you explain covariance to someone who understands only the mean?

...assuming that I'm able to augment their knowledge about variance in an intuitive fashion ( Understanding "variance" intuitively ) or by saying: It's the average distance of the data values from the 'mean' - and since variance is in square units,…
PhD
  • 15,115
222
votes
7 answers

PCA on correlation or covariance?

What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
Random
  • 2,400
  • 3
  • 14
  • 8
135
votes
6 answers

How would you explain the difference between correlation and covariance?

Following up on this question, How would you explain covariance to someone who understands only the mean? , which addresses the issue of explaining covariance to a lay person, brought up a similar question in my mind. How would one explain to a…
pmgjones
  • 5,971
112
votes
2 answers

What is covariance in plain language?

What is covariance in plain language and how is it linked to the terms dependence , correlation and variance-covariance structure with respect to repeated-measures designs?
abc
  • 1,851
83
votes
5 answers

Can you provide an example of two variables with zero covariance that are not independent?

I read from my textbook that if two variables $X$ and $Y$ are independent, their covariance must be 0. But the converse is not true: $\text{cov}(X,Y)=0$ does not guarantee that X and Y are independent. I could not think of any proper example yet;…
Flying pig
  • 6,579
81
votes
8 answers

How and why do normalization and feature scaling work?

I see that lots of machine learning algorithms work better with mean cancellation and covariance equalization. For example, Neural Networks tend to converge faster, and K-Means generally gives better clustering with pre-processed features. I do not…
erogol
  • 1,677
75
votes
5 answers

Is a sample covariance matrix always symmetric and positive definite?

When computing the covariance matrix of a sample, is one then guaranteed to get a symmetric and positive-definite matrix? Currently my problem has a sample of 4600 observation vectors and 24 dimensions.
Morten
  • 1,148
  • 1
  • 10
  • 11
71
votes
2 answers

What does the inverse of covariance matrix say about data? (Intuitively)

I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?" Edit: Thanks for replies After taking some great courses, I'd like to add some points: It is measure of information,…
Arya
  • 973
67
votes
5 answers

Is every covariance matrix positive definite?

I guess the answer should be yes, but I still feel something is not right. There should be some general results in the literature, could anyone help me?
Jingjings
  • 1,323
55
votes
4 answers

Why does inversion of a covariance matrix yield partial correlations between random variables?

I heard that partial correlations between random variables can be found by inverting the covariance matrix and taking appropriate cells from such resulting precision matrix (this fact is mentioned in http://en.wikipedia.org/wiki/Partial_correlation…
michal
  • 1,328
  • 3
  • 12
  • 14
49
votes
7 answers

Why shouldn't the denominator of the covariance estimator be n-2 rather than n-1?

The denominator of the (unbiased) variance estimator is $n-1$ as there are $n$ observations and only one parameter is being estimated. $$ \mathbb{V}\left(X\right)=\frac{\sum_{i=1}^{n}\left(X_{i}-\overline{X}\right)^{2}}{n-1} $$ By the same token I…
MYaseen208
  • 2,809
43
votes
7 answers

Why does a linear least squares fit appear to have a bias when applied to simple test data?

I used python to generate a correlated data set for testing, and then plotted a basic linear least-squares fit. The result looked a bit strange to me, because the line doesn't really seem to pass "centrally" through the data. It looks a little…
teeeeee
  • 599
41
votes
5 answers

A measure of "variance" from the covariance matrix?

If the data is 1d, the variance shows the extent to which the data points are different from each other. If the data is multi-dimensional, we'll get a covariance matrix. Is there a measure that gives a single number of how the data points are…
dontloo
  • 17,318
41
votes
1 answer

Covariance of a random vector after a linear transformation

If $\mathbf {Z}$ is random vector and $A$ is a fixed matrix, could someone explain why $$\mathrm{cov}[A \mathbf {Z}]= A \mathrm{cov}[\mathbf {Z}]A^\top.$$
user92612
  • 775
34
votes
3 answers

How does the formula for generating correlated random variables work?

If we have 2 normal, uncorrelated random variables $X_1, X_2$ then we can create 2 correlated random variables with the formula $Y=\rho X_1+ \sqrt{1-\rho^2} X_2$ and then $Y$ will have a correlation $\rho$ with $X_1$. Can someone explain where this…
Lanza
  • 599
1
2 3
…
99 100