Optimal foraging of a reproducing animal as a discounted reward problem

Abstract

A model in which a foraging animal can reproduce if its energy reserves reach a critical level is presented. The animal's reserves are modelled as a finite-state Markov chain. The animal has the choice between foraging options which may have the same mean net gain but differ in their variances. In addition to starvation, animals are subject to death because of predation, bad weather, and so on. We focus on the case where the rate of mortality due to these sources is the same under all options. We investigate policies that maximise the expected lifetime reproductive success. It is found that the optimal value function is concave at low reserves (low-variance action region) and convex at high reserves (high-variance action region). The value function under the low-variance action has also the same shape and the same inflexion point. This result allows us to compute optimal policies just by looking at the low-variance value function. The result is also used to show that increasing the mortality rate increases the high-variance region under the optimal policy. The pattern of risk-sensitive behaviour predicted by this model is in contrast to that predicted by a similar model in which no reproduction occurs and the optimality criterion is to minimise the probability of death (McNamara (1990)).