6 - Computational issues arising in Bayesian nonparametric hierarchical models

Summary

Chapter 5 highlights the rich dependence structures that can be captured via Bayesian nonparametric models using multistage (nonparametric) hierarchies illustrated graphically in Figure 5.2. The applications they present are impressive and we can see real operational benefits provided by the careful specification of additional layers of dependences. In this companion chapter we examine in detail some of the related key issues, including computational challenges and the use of de Finetti's representation theorem.

Introduction

Hierarchical models have played a central role in Bayesian inference since the time of Good (1965) with Lindley and Smith (1972) a key milestone providing the first comprehensive treatment of hierarchical priors for the parametric Bayes linear model. The Bayesian hierarchies have proved so popular because they provide a natural framework for “borrowing of strength” (a term apparently due to Tukey) or sharing partial information across components through the hierarchical structure. The Bayesian construction also provides a clear distinction from frequentist models in that the dependence structures do not have to be directly related to population random effects.

In this chapter we will look a little more closely at a couple of key issues emerging from the work of Teh and Jordan. First, we briefly step back a little and consider the fundamental notion played by de Finetti's representation theorem, reiterating the operational focus of Bayesian modeling on finite-dimensional specification of joint distributions on observables.