Chapter 15 covers CORRELATION AND SIMPLE REGRESSION AS INFERENTIAL TECHNIQUES and includes the following specific topics, among others:Bivariate Normal Distribution, Statistical Significance Test of Correlation, Confidence Intervals, Statistical Significance of b-Weight, Fit of the Overall Regression Equation, R and R-squared, Adjusted R-squared, Regression Diagnostics, Residual Plots, Influential Observations, Discrepancy, Leverage, Influence, and Power Analyses.

Chapter 15 covers correlation and simple regression as inferential techniques and includes the following specific topics, among others: bivariate normal distribution, statistical significance test of correlation, confidence intervals, statistical significance of the b weight, fit of the overall regression equation, R and R-squared, adjusted R-squared, regression diagnostics, residual plots, influential observations, discrepancy, leverage, influence, and power analysis.

In this chapter we present brief discussions of a few statistical topics not covered in earlier chapters. We first cover structural equation models, factor analysis, and path analysis. In future work fitting regression models in the social sciences, we frequently see reference to one or more of them. In the second section of the chapter, we address in summary form a few topics already discussed but which we believe require some additional attention. For instance, as part of our discussion of ordinary least squares regression, we covered in Chapter 8 the topic of regression diagnostics. But regression diagnostics is not an issue applicable only to OLS regression; so we present here a further discussion. Similarly, we expand with some additional commentary our earlier discussions of addressing issues of survey design (covered in Chapter 10) and multilevel models (covered in Chapter 16).

When we use ordinary least squares (OLS) regression with data sampled from a larger population, there are several assumptions that need to be met for the results to be reliably extended to the larger population. In the first part of this chapter, we discuss each of these assumptions. We note some of the problems that will occur if one or more of them are violated. In the second part of the chapter, we turn to issues of regression diagnostics, that is, methods and approaches for determining whether the assumptions are met in the sample data. In the third and last section of the chapter, we discuss the topic of robust regression. We note that, under ideal conditions, OLS regression is preferred over other regression methods. But sometimes when some of the OLS regression assumptions are not met, the OLS regression breaks down and should not be used for the analysis. In such situations, regression methods less demanding than OLS may be introduced. Robust regression is one such method. It sometimes performs in a more satisfactory manner than OLS when some of the OLS assumptions are not met and when there are other statistical problems in the analysis.

The simple linear regression is one of the most frequently employed statistical model. Linear regression is used to describe the relationship between two numerical variables, but it also serves as a building block for more complex statistical methods, such as the multivariate ordination. We start by comparing the concepts of regression and correlation, before introducing the equation of the simple linear regression. We also explain the decomposition of the observed values of the response variable into fitted values and regression residuals. Following this is a discussion regarding the hypotheses that can be tested for a regression model, distinguishing the F-ratio based test from the t tests of individual regression coefficients. The calculation of confidence and prediction intervals allow us to enhance diagrams displaying the fitted model. A separate section is devoted to the graphs of regression diagnostics and their interpretation, as well as to the effects of log-transforming the variables to linearise their relationship. Additional specialised sections deal with regression through the origin and its possible dangers, regression using a predictor with random variation, and with linear calibration. The methods described in this chapter are accompanied by a carefully-explained guide to the R code needed for their use, including the effects and lmodel2 packages.

This chapter deals with

• Linear regression notions and mathematical formations

• The assumptions that a linear regression model is based on

• Identifying linear relationships between a dependent variable and independent variables

• Residual plots, leverage and influential points

• Evaluate the degree (effect) to which one variable influences the positive or negative change of another variable

• The geometrical interpretation of the intercept and the slope, and various metrics and tests used to assess the quality of the results

• The interpretation of coefficients

• Studying casual relationships (under specific conditions)

• Building a predictive model by fitting a regression line to the data

• Evaluating the correctness of the model

• Model overfitting

• Ordinary least squares (OLS)

• Exploratory OLS

• Geographically weighted regression (GWR)

After a thorough study of the theory and lab sections, you will be able to

• Distinguish which regression method is more appropriate for the problem at hand and the data available

• Interpret statistics and diagnostics used to evaluate a regression model

• Interpret the outcomes of a regression model such as the coefficient estimates, the scatter plots, the statistics and the maps created

• Identify if relationships, associations or linkages among dependent and independent variables exist

• Analyze data through regression analysis in Matlab

• Conduct exploratory regression analysis in ArcGIS

• Conduct geographically weighted regression in ArcGIS