Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

408 questions
639
votes
5 answers

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix $\mathbf X$. How does it work? What is the…
amoeba
  • 110,174
176
votes
1 answer

How to reverse PCA and reconstruct original variables from several principal components?

Principal component analysis (PCA) can be used for dimensionality reduction. After such dimensionality reduction is performed, how can one approximately reconstruct the original variables/features from a small number of principal components?…
amoeba
  • 110,174
98
votes
3 answers

What is the intuition behind SVD?

I have read about singular value decomposition (SVD). In almost all textbooks it is mentioned that it factorizes the matrix into three matrices with given specification. But what is the intuition behind splitting the matrix in such form? PCA and…
SHASHANK GUPTA
  • 1,419
63
votes
3 answers

How does centering make a difference in PCA (for SVD and eigen decomposition)?

What difference does centering (or de-meaning) your data make for PCA? I've heard that it makes the maths easier or that it prevents the first PC from being dominated by the variables' means, but I feel like I haven't been able to firmly grasp the…
Zenit
  • 1,906
53
votes
7 answers

Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?

I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment , and in the lecture video from Andrew Ng , they do singular value decomposition instead of eigenvector decomposition of…
DongukJu
  • 673
48
votes
1 answer

PCA and Correspondence analysis in their relation to Biplot

Biplot is often used to display results of principal component analysis (and of related techniques). It is a dual or overlay scatterplot showing component loadings and component scores simultaneously. I was informed by @amoeba today that he has…
ttnphns
  • 61,413
43
votes
3 answers

Theory behind partial least squares regression

Can anyone recommend a good exposition of the theory behind partial least squares regression (available online) for someone who understands SVD and PCA? I have looked at many sources online and have not found anything that had the right combination…
ClarPaul
  • 1,310
40
votes
1 answer

What are the advantages of kernel PCA over standard PCA?

I want to implement an algorithm in a paper which uses kernel SVD to decompose a data matrix. So I have been reading materials about kernel methods and kernel PCA etc. But it still is very obscure to me especially when it comes to mathematical…
CyberPlayerOne
  • 2,505
37
votes
5 answers

How do I use the SVD in collaborative filtering?

I'm a bit confused with how the SVD is used in collaborative filtering. Suppose I have a social graph, and I build an adjacency matrix from the edges, then take an SVD (let's forget about regularization, learning rates, sparsity optimizations, etc),…
Vishal
  • 1,210
34
votes
1 answer

Dimensionality reduction (SVD or PCA) on a large, sparse matrix

/edit: Further follow up now you can use irlba::prcomp_irlba /edit: following up on my own post. irlba now has "center" and "scale" arguments, which let you use it to calculate principle components, e.g: pc <- M %*% irlba(M, nv=5, nu=0,…
Zach
  • 25,116
33
votes
2 answers

How to compute SVD of a huge sparse matrix?

What is the best way to compute singular value decomposition (SVD) of a very large positive matrix (65M x 3.4M) where data is extremely sparse? Less than 0.1% of the matrix is non zero. I need a way that: will fit into memory (I know that online…
Sonia
  • 433
32
votes
2 answers

Why PCA of data by means of SVD of the data?

This question is about an efficient way to compute principal components. Many texts on linear PCA advocate using singular-value decomposition of the casewise data . That is, if we have data $\bf X$ and want to replace the variables (its columns ) by…
ttnphns
  • 61,413
32
votes
1 answer

The proof of shrinking coefficients using ridge regression through "spectral decomposition"

I have understood how ridge regression shrinks coefficients towards zero geometrically. Moreover I know how to prove that in the special "Orthonormal Case," but I am confused how that works in the general case via "Spectral decomposition."
jeza
  • 2,159
32
votes
7 answers

Testing for linear dependence among the columns of a matrix

I have a correlation matrix of security returns whose determinant is zero. (This is a bit surprising since the sample correlation matrix and the corresponding covariance matrix should theoretically be positive definite.) My hypothesis is that at…
Ram Ahluwalia
  • 3,131
32
votes
1 answer

What norm of the reconstruction error is minimized by the low-rank approximation matrix obtained with PCA?

Given a PCA (or SVD) approximation of matrix $X$ with a matrix $\hat X$, we know that $\hat X$ is the best low-rank approximation of $X$. Is this according to the induced $\parallel \cdot \parallel_2$ norm (i.e. the largest eigenvalue norm) or…
Donbeo
  • 3,237
1
2 3
…
27 28