9 - Linear Regression

Summary

The linear regression or least squares methodl is introduced first by example and then theoretically based on four simple and widely applicable assumptions. The theory explicitly pertains to predictor variables that are random variables, as is usually the case with actual data. The hat matrix is defined and its use to detect leverage is specified. Statistical inference in linear regression is outlined, incluing least squares analysis of variance, assessing the regression parameters and assessing the randomness and correlation of the residuals. Use of the bootstrap and jackknife to obtain confidence intervals is specified. The importance of assessing the least squares model is emphasized and illustrated using several data sets. Robust and bounded influence estimators that remove bias caused by outliers and leverage points are derived and implemented with Matlab. Methods that reduce the bias caused by noise in the predictor variables are described. Shrinkage estimators for variable selection are elucidated. Finally, logistic regression that pertains when the response variable is categorical is described.