PCA Using Correlation & Covariance Matrix (R Example)

 

When we perform a Principal Component Analysis (PCA), this can be based on either the covariance matrix or the correlation matrix.

In this tutorial, we will explain, theoretically, what’s the main difference between using the correlation matrix and using the covariance matrix when performing a PCA. We will also see these differences when applying this theory based on a data sample using the R programming language.

This is the content we will talk about:

Let’s take a look.

 

Sample Data

To explain this tutorial, we will be using the USArrests data set, which contains statistics about violent crime rates in each of the 50 US states. We can see how it looks like using the head() function:

head(USArrests)

USArrests data set pca

 

PCA Using Correlation Matrix

A correlation matrix is a table used to summarize data to compute a PCA (or other analyses). This table displays the correlation coefficients between the variables in a data set.

When our data contains variables which have been measured using different scales, we should use the correlation matrix to calculate the PCA: this is equivalent to standardize all the variables (which means make them have a mean of 0 and a standard deviation of 1). In fact, this matrix is a standardized version of the covariance matrix.

In some way, the correlation matrix ensures that the differences in the measurement scales are taken into account, and it helps us to avoid biasing the PCA by the magnitude of the change in the variables.

Let’s see how a PCA looks like using the correlation matrix in our data set. For specifying the correlation matrix when performing the PCA, we will use scale=TRUE inside the prcomp() function:

cor_USArrests <- prcomp(USArrests,
                     scale=TRUE)
 
summary(cor_USArrests)
 
# Importance of components:
#                           PC1    PC2     PC3     PC4
# Standard deviation     1.5749 0.9949 0.59713 0.41645
# Proportion of Variance 0.6201 0.2474 0.08914 0.04336
# Cumulative Proportion  0.6201 0.8675 0.95664 1.00000

We can visualize the results in a biplot:

biplot(cor_USArrests)

USArrests biplot correlation matrix pca

 

PCA Using Covariance Matrix

A covariance matrix is also a table which shows the covariance values between the elements in a data set.

Given that in our sample data, not every variable has been measured in the same scale, performing the PCA on a covariance matrix will give different results than performing the PCA on a correlation matrix. Let’s see what happens with our example by using the covariance matrix specifying scale=FALSE:

cov_USArrests <- prcomp(USArrests,
                     scale=FALSE)
 
summary(cov_USArrests)
 
# Importance of components:
#                            PC1      PC2    PC3     PC4
# Standard deviation     83.7324 14.21240 6.4894 2.48279
# Proportion of Variance  0.9655  0.02782 0.0058 0.00085
# Cumulative Proportion   0.9655  0.99335 0.9991 1.00000

The importance of each component has changed compared with the results obtained using the correlation matrix.

biplot(cov_USArrests)

USArrests biplot covariance matrix pca

You may want to consider the fact that the covariance matrix is sensitive to variations in the dispersion of the data, while the correlation matrix isn’t, and this leads to better-defined factor structures. However, this covariance matrix should be used only if our variables are commensurable. Moreover, when using the covariance matrix, we can’t be sure that the differences in measurement scales will be taken into account.

In our example, a PCA on a correlation matrix is much more informative and reveals a structure, as we can see in the first graph in contrast to the second biplot. Nevertheless, which matrix to use to perform our PCA will depend on the nature of our data.

 

Video, Further Resources & Summary

Do you need more explanations on the difference of performing a PCA using a correlation matrix or a covariance matrix? Then you should have a look at the following YouTube video of the Statistics Globe YouTube channel.

 

The YouTube video will be added soon.

 

You can also check some other tutorials on Statistics Globe:

This post has shown the differences between performing a PCA with a correlation matrix and a covariance matrix. In case you have further questions, you may leave a comment below.

 

Paula Villasante Soriano Statistician & R Programmer

This page was created in collaboration with Paula Villasante Soriano. Please have a look at Paula’s author page to get further information about her academic background and the other articles she has written for Statistics Globe.

 

Subscribe to the Statistics Globe Newsletter

Get regular updates on the latest tutorials, offers & news at Statistics Globe.
I hate spam & you may opt out anytime: Privacy Policy.


Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.

Menu
Top