15 - Principal components analysis and factor analysis

Summary

In the previous chapter we defined the idea of a multivariate observation and looked at multivariate analysis techniques for discovering and confirming the presence of special groups among the observed individuals. In the present chapter we will look at methods designed specifically to reduce the dimensionality of the data.

Reducing the dimensionality of multivariate data

Suppose that their scores on p variables, X₁, X₂, …, X_p, have been observed for each of n subjects (see §15.3 for a language testing example). Typically, the variables will be intercorrelated, each variable having a higher correlation with some of the other variables than it does with the remainder. It is quite common that the pattern of intercorrelations is rather complex. As is often the case, the major statistical interest will lie in considering the differences between individuals. The special problem to be faced now is that the subjects can be different in a variety of ways. Two subjects may have similar scores for some of the variables and quite dissimilar scores on some of the others. If the number of variables is large, then it may be difficult to decide whether subject A differs more from subject B than from subject C since the pattern of differences may be quite dissimilar in the two cases.