13 - Regression on principal component or discriminant scores

Summary

Dimension reduction techniques reduce the number of candidate explanatory variables. Perhaps best known is the replacement of a large number of candidate explanatory variables by the first few principal components. The hope is that they will adequately summarize the information in the candidate explanatory variables. In favorable circumstances, simple modifications of the components will give new variables that are readily interpretable, but this is not always the case.

Propensity scores, often simply called propensities, may be helpful where a response is compared between two groups – a control and a treatment group – that have not been assigned randomly. The response may for example, in a medical context, be death rate in some interval of time. Variables that are not of direct interest, but which may in part explain any differences between the two groups, are commonly known as explanatory variables. Results from such analyses are likely to be suggestive rather than definitive, irrespective of the methodology used to account for explanatory variable effects.

Propensities aim to capture, in a single variable, the explanatory variable effects that are important in accounting for differences between two groups. The propensity score, commonly derived from a discriminant analysis, then becomes the only explanatory variable in the regression calculation.