Conclusion

Summary

It is a fundamental challenge of empiricist science that we must assess the merits of our theories and models without any purely objective means of doing so. The set of lenses through which we theorize about and model our subject matter are the same lenses through which we identify what will count as “data” of/about the subject matter. If these lenses fundamentally distort or render imperceptible important aspects of the subject matter, then our theories and models will in a sense be detached – we will be exploring the implications of a version of the subject matter that comports with our conceptual map, but we will not know the extent to which that conceptual map reproduces that of the subject matter in the relevant respects. In order to ensure that such detachment has not occurred (or, retrospectively, to detect when it has), we must, in a sense, go “behind the model” – that is, we must use some other mode of access to the subject to ascertain whether important information about the latter has indeed been distorted or rendered imperceptible by the model.

The central difficulty in doing so is that we do not know a priori precisely what kind of information we will need to accomplish this task, as the subject matter under study may turn out to be constituted in a manner that we had not anticipated. As such, it is crucial that we develop and maintain the capacity to be sensitive to relevant information about the subject matter in whatever form it may become manifest, and that we incorporate this sensitivity into our methods.

The financial crisis of 2008–09 was an object lesson in both the importance and the difficulty of this task. It demonstrated that even apparently extensively vetted models that enjoy wide acceptance on the basis of their performance in econometric testing can possess substantial and inherent blind spots. And it demonstrated, further, how a failure to remain sufficiently attuned to the particular circumstances of the target subject matter – especially those circumstances that may lie outside the conceptual map of our models – allows such blind spots to persist.

The testimony of Sidney Winter at the 2010 Congressional hearing on the failures of DSGE modeling in the lead-up to the crisis helpfully illustrates this point.