Multiple linear regression generalizes straight line regression to allow multiple explanatory (or predictor) variables, in this chapter under the normal errors assumption. The focus may be on accurate prediction. Or it may, alternatively or additionally, be on the regression coefficients themselves. Simplistic interpretations of coefficients can be grossly misleading. Later chapters elaborate on the ideas and methods developed in this chapter, applying them in new contexts. The attaching of causal interpretations to model coefficients must be justified both by reference to subject area knowledge and by careful checks to ensure that they are not artefacts of the correlation structure. There is attention to regression diagnostics, to assessment, and comparison of models. Variable selection strategies can readily over-fit. Hence the importance of training/test approaches and cross-validation. The potential is demonstrated for errors in x to seriously bias regression coefficients. Strong multicollinearity leads to large variance inflation factors.

Previous chapters considered the Euclidean norm, the spectral norm, and the Frobenius norm. These three norms are particularly important, but there are many other important norms for applications. This chapter discusses vector norms, matrix norms, and operator norms, and uses these norms to analyze the convergence of sequences. It revisits the Moore–Penrose pseudoinverse from a norm-minimizing perspective. It applies norms to the orthogonal Procrustes problem and its extensions.

When we use ordinary least squares (OLS) regression with data sampled from a larger population, there are several assumptions that need to be met for the results to be reliably extended to the larger population. In the first part of this chapter, we discuss each of these assumptions. We note some of the problems that will occur if one or more of them are violated. In the second part of the chapter, we turn to issues of regression diagnostics, that is, methods and approaches for determining whether the assumptions are met in the sample data. In the third and last section of the chapter, we discuss the topic of robust regression. We note that, under ideal conditions, OLS regression is preferred over other regression methods. But sometimes when some of the OLS regression assumptions are not met, the OLS regression breaks down and should not be used for the analysis. In such situations, regression methods less demanding than OLS may be introduced. Robust regression is one such method. It sometimes performs in a more satisfactory manner than OLS when some of the OLS assumptions are not met and when there are other statistical problems in the analysis.