Simple linear regression is extended to multiple linear regression (for multiple predictor variables) and to multivariate linear regression for (multiple response variables). Regression with circular data and/or categorical data is covered. How to select predictors and how to avoid overfitting with techniques such as ridge regression and lasso are followed by quantile regression. The assumption of Gaussian noise or residual is removed in generalized least squares, with applications to optimal fingerprinting in climate change.

The method of least squares will fit any model to a data set, but is the resulting model "good"?One criterion is that the model should fit the data significantly better than a simpler model with fewer predictors. After all, if the fit is not significantly better, then the model with fewer predictors is almost as good. For linear models, this approach is equivalent to testing if selected regression parameters vanish. This chapter discusses procedures for testing such hypotheses. In interpreting such hypotheses, it is important to recognize that a regression parameter for a given predictor quantifies the expected rate of change of the predict and while holding the other predictors constant. Equivalently, the regression parameter quantifies the dependence between two variables after controlling or regressing out other predictors. These concepts are important for identifying a confounding variable, which is a third variable that influences two variables to produce a correlation between those two variables. This chapter also discusses how detection and attribution of climate change can be framed in a regression model framework.