Role of Serotonin in Punishment and Reward: New Vistas

Abstract

The role of forebrain serotonin (or 5-hydroxytryptamine, 5-HT) in the chemical neuromodulation of mammalian behaviour remains unclear. Two prominent hypotheses are that it is implicated primarily in (i) behavioural inhibition; or ii) aversive punishment, although distinguishing between these hypotheses is difficult because of intrinsic asymmetries in how these systems control behavior. Previous, apparently paradoxical, findings appear to show that 5-HT depletion both reduces and enhances the effects of aversive reinforcers in humans and rodents, suggesting independent modulations of dissociable neural systems. Evidence will be surveyed from experimental animals, using intra-cerebral treatments with a selective 5-HT neurotoxin, 5,7 dihydroxytryptamine, or microinfusions of selective 5-HT receptor agents, and from human volunteers using acute dietary depletion of tryptophan (ATD) or systemic treatments with serotonin re-uptake inhibitors, in order to address these distinctions. Central manipulations of 5-HT generally disrupt behavioural inhibition, including both impulsive and compulsive behaviour, although often with no effect on sensitive measures of response inhibition in the stop-signal reaction time task. Some evidence in humans using ATD suggests that the behavioral inhibition is greater for aversively motivated than appetitively motivated behavior. However, in direct comparisons of effects of local depletions of 5-HT in the orbitofrontal cortex and amygdala in the marmoset monkey, both forms of motivated behavior are impaired, especially in the anticipatory phase of behavioural response to impending appetitive or aversive reinforcers. These findings are related to a variety of neuropsychiatric disorders in which 5-HT dysregulation has been postulated, including depression, drug addiction and obsessive-compulsive disorder.