Short paper

Dopamine, uncertainty and TD learning

Yael Niv^1,2 , Michael O Duff² and Peter Dayan²

¹  Interdisciplinary Center for Neural Computation, Hebrew University, Jerusalem, Israel

²  Gatsby Computational Neuroscience Unit, University College London, London, UK

author email corresponding author email

Behavioral and Brain Functions 2005, 1:6doi:10.1186/1744-9081-1-6

Published:	4 May 2005

Abstract

Substantial evidence suggests that the phasic activities of dopaminergic neurons in the primate midbrain represent a temporal difference (TD) error in predictions of future reward, with increases above and decreases below baseline consequent on positive and negative prediction errors, respectively. However, dopamine cells have very low baseline activity, which implies that the representation of these two sorts of error is asymmetric. We explore the implications of this seemingly innocuous asymmetry for the interpretation of dopaminergic firing patterns in experiments with probabilistic rewards which bring about persistent prediction errors. In particular, we show that when averaging the non-stationary prediction errors across trials, a ramping in the activity of the dopamine neurons should be apparent, whose magnitude is dependent on the learning rate. This exact phenomenon was observed in a recent experiment, though being interpreted there in antipodal terms as a within-trial encoding of uncertainty.