4 - Large Worlds

Summary

Let us imagine to ourselves the case of a person just brought forth into this world, and left to collect from his observation of the order and course of events what powers and causes take place in it. The Sun would, probably, be the first object that would engage his attention: but after losing sight of it the first night he would be entirely ignorant whether he would ever see it again. He would therefore be in the condition of a person making a first experiment entirely unknown to him.

Richard Price Appendix to Bayes’ Essay

So far, we have considered learning models that operate within fixed conceptual frameworks. Fictitious play assumes that states, acts, and outcomes are known; reinforcement learning assumes the same for acts and outcomes. In many situations this is implausible, as Bayes’ friend and curator Richard Price has observed in his reflection on “a person just brought forth into this world.” One may not know the basic constituents of a new environment; accordingly, there may be no fixed conceptual framework for learning.

While this observation is not a knockout criticism of the learning models studied in previous chapters, it does put the spotlight on one of their inherent limitations. In this chapter, we will investigate some ways to overcome that limitation. To set the stage, I will give some context to the question of how much knowledge a learning model presupposes by discussing Savage's distinction between small worlds and large worlds in the context of learning. In a small world, the structure of an epistemic situation is fully known; in a large world, one does not know or anticipate every aspect of the epistemic situation that might be relevant. The distinction between large and small worlds will add yet another layer to our running discussion of bounded rationality.

The second section of this chapter looks at models based on flexible conceptual frameworks that take the epistemic incompleteness of a learning situation into account. They do so by keeping the learning procedure open to conceptual changes.