Email updates

Keep up to date with the latest news and content from Behavioral and Brain Functions and BioMed Central.

Open Access Highly Accessed Short paper

Dopamine, uncertainty and TD learning

Yael Niv12*, Michael O Duff2 and Peter Dayan2

Author Affiliations

1 Interdisciplinary Center for Neural Computation, Hebrew University, Jerusalem, Israel

2 Gatsby Computational Neuroscience Unit, University College London, London, UK

For all author emails, please log on.

Behavioral and Brain Functions 2005, 1:6  doi:10.1186/1744-9081-1-6

Published: 4 May 2005

Abstract

Substantial evidence suggests that the phasic activities of dopaminergic neurons in the primate midbrain represent a temporal difference (TD) error in predictions of future reward, with increases above and decreases below baseline consequent on positive and negative prediction errors, respectively. However, dopamine cells have very low baseline activity, which implies that the representation of these two sorts of error is asymmetric. We explore the implications of this seemingly innocuous asymmetry for the interpretation of dopaminergic firing patterns in experiments with probabilistic rewards which bring about persistent prediction errors. In particular, we show that when averaging the non-stationary prediction errors across trials, a ramping in the activity of the dopamine neurons should be apparent, whose magnitude is dependent on the learning rate. This exact phenomenon was observed in a recent experiment, though being interpreted there in antipodal terms as a within-trial encoding of uncertainty.