Striatal and Tegmental Neurons Code Critical Signals for Temporal-Difference Learning of State Value in Domestic Chicks

Published on Nov 8, 2016in Frontiers in Neuroscience3.65
· DOI :10.3389/fnins.2016.00476
Chentao Wen1
Estimated H-index: 1
(Hokkaido University),
Yukiko Ogura5
Estimated H-index: 5
(Hokkaido University),
Toshiya Matsushima23
Estimated H-index: 23
(Hokkaido University)
To ensure survival, animals must update the internal representations of their environment in a trial-and-error fashion. Psychological studies of associative learning and neurophysiological analyses of dopaminergic neurons have suggested that this updating process involves the temporal-difference (TD) method in the basal ganglia network. However, the way in which the component variables of the TD method are implemented at the neuronal level is unclear. To investigate the underlying neural mechanisms, we trained domestic chicks to associate color cues with food rewards. We recorded neuronal activities from the medial striatum or tegmentum in a freely behaving condition and examined how reward omission changed neuronal firing. To compare neuronal activities with the signals assumed in the TD method, we simulated the behavioral task in the form of a finite sequence composed of discrete steps of time. The three signals assumed in the simulated task were the prediction signal, the target signal for updating, and the TD-error signal. In both the medial striatum and tegmentum, the majority of recorded neurons were categorized into three types according to their fitness for three models, though these neurons tended to form a continuum spectrum without distinct differences in the firing rate. Specifically, two types of striatal neurons successfully mimicked the target signal and the prediction signal. A linear summation of these two types of striatum neurons was a good fit for the activity of one type of tegmental neurons mimicking the TD-error signal. The present study thus demonstrates that the striatum and tegmentum can convey the signals critically required for the TD method. Based on the theoretical and neurophysiological studies, together with tract-tracing data, we propose a novel model to explain how the convergence of signals represented in the striatum could lead to the computation of TD error in tegmental dopaminergic neurons.
  • References (59)
  • Citations (15)
#1Ju Tian (Harvard University)H-Index: 5
#2Ryan Huang (Harvard University)H-Index: 1
Last.Mitsuko Watabe-Uchida (Harvard University)H-Index: 13
view all 9 authors...
#1Mitsuko Watabe-Uchida (Harvard University)H-Index: 13
#2Lisa C. Zhu (Harvard University)H-Index: 1
Last.Naoshige Uchida (Harvard University)H-Index: 33
view all 5 authors...
#1Jeremiah Y. Cohen (Harvard University)H-Index: 15
#2Sebastian Haesler (Harvard University)H-Index: 7
Last.Naoshige Uchida (Harvard University)H-Index: 33
view all 5 authors...
Cited By15
#1María Eugenia Alzugaray (CONICET: National Scientific and Technical Research Council)H-Index: 4
#2María Cecilia Bruno (CONICET: National Scientific and Technical Research Council)H-Index: 3
Last.Jorge Rafael Ronderos (UNLP: National University of La Plata)H-Index: 8
view all 0 authors...
#1Wing-Kin Tam (UMN: University of Minnesota)H-Index: 5
#2Tong Wu (UMN: University of Minnesota)H-Index: 5
Last.Zhi Yang (UMN: University of Minnesota)H-Index: 15
view all 5 authors...
#1Emilio Trigili (Sant'Anna School of Advanced Studies)H-Index: 3
#2Lorenzo Grazi (Sant'Anna School of Advanced Studies)H-Index: 2
Last.Alessandro Panarese (Sant'Anna School of Advanced Studies)H-Index: 10
view all 8 authors...
#1Regina H. Reynolds (UCL: University College London)H-Index: 1
#2Juan A. Botía (UCL: University College London)H-Index: 15
Last.Mina Ryten (UCL: University College London)H-Index: 34
view all 6 authors...
#1Sandhya Chengaiyan (SSN: Sri Sivasubramaniya Nadar College of Engineering)H-Index: 1
#2Anandha Sree Retnapandian (SSN: Sri Sivasubramaniya Nadar College of Engineering)
Last.Kavitha Anandan (SSN: Sri Sivasubramaniya Nadar College of Engineering)H-Index: 2
view all 0 authors...
#2Xin ZhangH-Index: 6
Last.Lexin Li (University of California, Berkeley)H-Index: 25
view all 3 authors...
View next paperInformation encoding and pathways in basal ganglia neurons