Principal components analysis:
The purpose of principal component analysis is to derive a small number of linear combinations (principal components) of a set of variables that retain as much of the information in the original variables as possible. This technique is often used when there are large numbers of variables, and you wish to reduce them to a smaller number of variable combinations by combining similar variables (ones that contain much the same information).
Principal components are linear combinations of variables that retain maximal amount of information about the variables. The term “maximal amount of information” here means the best least-square fit, or, in other words, maximal ability to explain variance of the original data.
In technical terms, a principal component for a given set of N-dimensional data, is a linear combination of the original variables with coefficients equal to the components of an eigenvector of the correlation or covariance matrix. Principal components are usually sorted by descending order of the eigenvalues – i.e. the first principal component corresponds to the eigenvector with the maximal eigenvalue.