5 - Linear Regression Models

Summary

Introduction

The maximum likelihood framework set out in Part ONE is now applied to estimating and testing regression models. This chapter focusses on linear models, where the conditional mean of a dependent variable is specified to be a linear function of a set of exogenous variables. Extensions to this basic model are investigated in Chapter 6 (nonlinear regression), Chapter 7 (autocorrelation) and Chapter 8 (heteroskedasticity).

Single equation models include the linear regression model and the constant mean model. For single equation regression models, the maximum likelihood estimator has an analytical solution that is equivalent to the ordinary least squares estimator. The class of multiple equation models includes simultaneous equation models with multiple dependent and exogenous variables, seemingly unrelated systems and recursive models. In this instance, the maximum likelihood estimator is known as the full information maximum likelihood (FIML) estimator because the entire system is used to estimate all of the model parameters jointly. The FIML estimator is related to the instrumental variable estimator commonly used to estimate simultaneous models and, in some cases, the two estimators are equivalent. Unlike linear single equation models, analytical solutions of the maximum likelihood estimator for systems of linear equations are only available in certain special cases.

Many of the examples considered in Part ONE specify the distribution of the observable random variable, y_t. Regression models, by contrast, specify the distribution of the unobservable disturbance, u_t, which means that maximum likelihood estimation cannot be used directly because this method requires evaluating the log-likelihood function at the observed values of the data.