8 - Empirical Validation of Agent-Based Models

Summary

Introduction

Generally speaking, validation involves a judgment over the quality of a model. How good is model A is? Is it better or worse than model B? A model can be good from a certain point of view and bad, or inadequate, from another one. Also, validation is not necessarily a 0–1 pass test: the criteria can be continuous.

The validity of a model can be defined as the degree of homomorphism between a certain system (the model) and another system that it purportedly represents (the real-world system).

Model validation can be defined along different dimensions.

First of all, no model exists without an underlying theory. A first dimension of validation therefore is concept validation, i.e., the validation of the model relative to the theory: is the model consistent with the theory on which it is based? This is common to both analytical and computational models. The latter, however, need an additional level of validation (Stanislaw, 1986): program validation, i.e., the validation of the simulator (the code that simulates the model) relative to the model itself.

Second, models can be evaluated against real data. This is empirical validation. The aim of this chapter is to introduce the reader to the techniques of empirical validation of ABMs in economics. It requires (i) the choice of the relevant empirical indicators (so that the theoretical framework can be validated relative to its indicators) and (ii) the validation of the empirical true value relative to its indicator.

Empirical validation is often the basis for theory validation – the validation of the theory relative to the simuland (the real-world system).

Empirically validating an ABM means, broadly speaking, “taking the model to the data,” in the form of empirical and/or experimental data, historical evidence or even anecdotal knowledge.

Empirical validation may concern the model inputs and/or outputs. Input validation refers to the realism of the assumptions. There are two classes of inputs of an ABM. The first one consists of structural assumptions concerning the behavior of the agents or the pattern of their interactions. Examples include a particular bounded-rationality rule that we assume agents follow (e.g., a mark-up price-setting rule), or a peculiar type of network (for instance, a small-world) governing the interactions among agents.