Contents - Index

Singular value decomposition


Singular value decomposition (SVD) is a highly sophisticated and complicated mathematical technique for decomposing a matrix. Please consult a mathematical book for in depth knowledge. 


SVD can be viewed from two perspectives:


1) SVD decomposes a matrix into three parts: The singular values, the row eigenvectors, and the column eigenvectors. The singular value matrix is a r-dimension diagonal matrix, the row eigenvector matrix is a r-column matrix, and the column eigenvector matrix is a r-row matrix, with 'rank' r = min (n,m), n being the number of rows, and m the number of columns of the original matrix.  Or


2) SVD decomposes a matrix into r = min(m,n) principal components (PC). For this reason SVD is sometimes referred to as principal component analysis (PCA), although this is not accurate. The PC's are so ranked that the information (variation) explained by each PC are: PC1 > PC2 > PC3 >...>PCr. Each PC consists of a singular value, an entry vector of m elements, and a tester vector of n elements.


Note: A 2-D biplot displays the first two PC's, PC1 vs. PC2, and a 3D-biplot displays the first 3 PC's, i.e., PC1, PC2, and PC3. Biplots other than PC1 vs. PC2 are also possible in GGEbiplot. Usually biplot refers to a 2-D biplot of PC1 vs. PC2.


In Depth


SVD may be regarded as the reverse process of matrix multiplication. Two matrices can be multiplied if the number of columns in the first matrix equals to the number of rows in the second matrix. For example, a matrix A(n,r) and a matrix B(r, m) can be multiplied to produce a matrix C(n, m):


  A(n,r) * B(r,m) => C(n,m).


SVD is to decompose C so that A and B can be recovered. In reality, there is no unique solution in this "reverse" process. An analogy may help understand this. We know 4 * 6 = 24. However, when 24 is decomposed, we can have 24 => 3 * 8, 24 => 2 * 12, 24 => 4 * 6, and so on. There is a unique solution if 24 is decomposed into three parts: 24 => 2 * 4 * 3, with 2 and 3 as prime numbers (that cannot be further divided). 


Similarly, C can not be uniquely decomposed to recover A and B. However, it can be uniquely decomposed into three parts: 


  C(n,m) => a(n,r) * l(r) * b(r,m). 


The middle part l(r) is called singular values. To display C in a biplot, l(r) has to be partitioned between a and b such that


  A'(n,r)  = a(n,r) * l(r)^alpha, 


  B'(r,m) = b(r,m) * l(r)^beta,


  alpha + beta = 1, with alpha and beta are non-negative.


This is the so-called singular value partitioning, which is one of the important issues in biplot analysis.