Striatal and Tegmental Neurons Code Critical Signals for Temporal-Difference Learning of State Value in Domestic Chicks

Published on Nov 8, 2016in Frontiers in Neuroscience3.648
· DOI :10.3389/fnins.2016.00476
Chentao Wen1
Estimated H-index: 1
(Hokkaido University),
Yukiko Ogura5
Estimated H-index: 5
(Hokkaido University),
Toshiya Matsushima24
Estimated H-index: 24
(Hokkaido University)
To ensure survival, animals must update the internal representations of their environment in a trial-and-error fashion. Psychological studies of associative learning and neurophysiological analyses of dopaminergic neurons have suggested that this updating process involves the temporal-difference (TD) method in the basal ganglia network. However, the way in which the component variables of the TD method are implemented at the neuronal level is unclear. To investigate the underlying neural mechanisms, we trained domestic chicks to associate color cues with food rewards. We recorded neuronal activities from the medial striatum or tegmentum in a freely behaving condition and examined how reward omission changed neuronal firing. To compare neuronal activities with the signals assumed in the TD method, we simulated the behavioral task in the form of a finite sequence composed of discrete steps of time. The three signals assumed in the simulated task were the prediction signal, the target signal for updating, and the TD-error signal. In both the medial striatum and tegmentum, the majority of recorded neurons were categorized into three types according to their fitness for three models, though these neurons tended to form a continuum spectrum without distinct differences in the firing rate. Specifically, two types of striatal neurons successfully mimicked the target signal and the prediction signal. A linear summation of these two types of striatum neurons was a good fit for the activity of one type of tegmental neurons mimicking the TD-error signal. The present study thus demonstrates that the striatum and tegmentum can convey the signals critically required for the TD method. Based on the theoretical and neurophysiological studies, together with tract-tracing data, we propose a novel model to explain how the convergence of signals represented in the striatum could lead to the computation of TD error in tegmental dopaminergic neurons.
  • References (57)
  • Citations (130)
📖 Papers frequently viewed together
98 Citations
12 Citations
5 Authors (André Garenne, ..., Thomas Boraud)
78% of Scinapse members use related papers. After signing in, all features are FREE.
#1Ju Tian (Harvard University)H-Index: 5
#2Ryan Huang (Harvard University)H-Index: 1
Last. Mitsuko Watabe-Uchida (Harvard University)H-Index: 13
view all 9 authors...
Summary Dopamine neurons encode the difference between actual and predicted reward, or reward prediction error (RPE). Although many models have been proposed to account for this computation, it has been difficult to test these models experimentally. Here we established an awake electrophysiological recording system, combined with rabies virus and optogenetic cell-type identification, to characterize the firing patterns of monosynaptic inputs to dopamine neurons while mice performed classical con...
67 CitationsSource
#1David Silver (Google)H-Index: 51
#2Aja Huang (Google)H-Index: 5
Last. Demis Hassabis (Google)H-Index: 44
view all 20 authors...
The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-pl...
3,766 CitationsSource
#1Yukiko Ogura (Hokkaido University)H-Index: 5
#2Takeshi Izumi (Hokkaido University)H-Index: 23
Last. Toshiya Matsushima (Hokkaido University)H-Index: 24
view all 4 authors...
The frequency or intensity of behavior is often facilitated by the presence of others. This social facilitation has been reported in a variety of animals, including birds and humans. Based on Zajonc’s “drive theory,” we hypothesized that facilitation and drive have shared neural mechanisms, and that dopaminergic projections from the midbrain to striatum are involved. As the ascending dopaminergic projections include the mesolimbic and nigrostriatal pathways, we targeted our lesions at the medial...
11 CitationsSource
#1Kei Oyama (Tohoku University)H-Index: 5
#2Yukina Tateyama (Tohoku University)H-Index: 1
Last. Ken-Ichiro Tsutsui (Tohoku University)H-Index: 17
view all 6 authors...
To investigate how the striatum integrates sensory information with reward information for behavioral guidance, we recorded single-unit activity in the dorsal striatum of head-fixed rats participating in a probabilistic Pavlovian conditioning task with auditory conditioned stimuli (CSs) in which reward probability was fixed for each CS but parametrically varied across CSs. We found that the activity of many neurons was linearly correlated with the reward probability indicated by the CSs. The rec...
13 CitationsSource
#1Neir EshelH-Index: 13
#2Michael BukwichH-Index: 2
Last. Naoshige UchidaH-Index: 39
view all 6 authors...
Dopamine neurons in the ventral tegmental area calculate reward prediction error by subtracting input from neighbouring GABA neurons.
149 CitationsSource
#1Wolfram Schultz (University of Cambridge)H-Index: 84
Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neurona...
310 CitationsSource
#1Hidetoshi Amita (Hokkaido University)H-Index: 7
#2Toshiya Matsushima (Hokkaido University)H-Index: 24
To investigate the role of social contexts in controlling the neuronal representation of food reward, we recorded single neuron activity in the medial striatum/nucleus accumbens of domestic chicks and examined whether activities differed between two blocks with different contexts. Chicks were trained in an operant task to associate light-emitting diode color cues with three trial types that differed in the type of food reward: no reward (S−), a small reward/short-delay option (SS), and a large r...
9 CitationsSource
#1Christina Bocklisch (University of Geneva)H-Index: 2
#2Vincent Pascoli (University of Geneva)H-Index: 19
Last. Christian Lüscher (Geneva College)H-Index: 50
view all 8 authors...
Drug-evoked synaptic plasticity in the mesolimbic system reshapes circuit function and drives drug-adaptive behavior. Much research has focused on excitatory transmission in the ventral tegmental area (VTA) and the nucleus accumbens (NAc). How drug-evoked synaptic plasticity of inhibitory transmission affects circuit adaptations remains unknown. We found that medium spiny neurons expressing dopamine (DA) receptor type 1 (D1R-MSNs) of the NAc project to the VTA, strongly preferring the GABA neuro...
140 CitationsSource
#1Mitsuko Watabe-Uchida (Harvard University)H-Index: 13
#2Lisa Zhu (Harvard University)H-Index: 1
Last. Naoshige Uchida (Harvard University)H-Index: 39
view all 5 authors...
Summary Recent studies indicate that dopamine neurons in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) convey distinct signals. To explore this difference, we comprehensively identified each area's monosynaptic inputs using the rabies virus. We show that dopamine neurons in both areas integrate inputs from a more diverse collection of areas than previously thought, including autonomic, motor, and somatosensory areas. SNc and VTA dopamine neurons receive contrasting ex...
572 CitationsSource
#1Jeremiah Y. Cohen (Harvard University)H-Index: 15
#2Sebastian Haesler (Harvard University)H-Index: 7
Last. Naoshige Uchida (Harvard University)H-Index: 39
view all 5 authors...
Dopaminergic neurons in the mouse ventral tegmental area signal the difference between received and expected reward, whereas GABAergic neurons signal expected reward.
622 CitationsSource
Cited By130
#1Julien Bryois (KI: Karolinska Institutet)H-Index: 16
#2Nathan SkeneH-Index: 16
Last. Ernest Arenas (KI: Karolinska Institutet)H-Index: 66
view all 12 authors...
Genome-wide association studies have discovered hundreds of loci associated with complex brain disorders, but it remains unclear in which cell types these loci are active. Here we integrate genome-wide association study results with single-cell transcriptomic data from the entire mouse nervous system to systematically identify cell types underlying brain complex traits. We show that psychiatric disorders are predominantly associated with projecting excitatory and inhibitory neurons. Neurological...
1 CitationsSource
#1Fakhereh Movahedian Attar (MPG: Max Planck Society)
#2Evgeniya Kirilina (FU: Free University of Berlin)H-Index: 13
Last. Nikolaus Weiskopf (MPG: Max Planck Society)H-Index: 6
view all 7 authors...
Short association fibers (U-fibers) connect proximal cortical areas and constitute the majority of white matter connections in the human brain. U-fibers play an important role in brain development, function, and pathology but are underrepresented in current descriptions of the human brain connectome, primarily due to methodological challenges in diffusion magnetic resonance imaging (dMRI) of these fibers. High spatial resolution and dedicated fiber and tractography models are required to reliabl...
1 CitationsSource
#1João Jorge (EPFL: École Polytechnique Fédérale de Lausanne)H-Index: 8
#2Frédéric Gretsch (EPFL: École Polytechnique Fédérale de Lausanne)H-Index: 3
Last. M. Bach Cuadra (EPFL: École Polytechnique Fédérale de Lausanne)H-Index: 18
view all 9 authors...
PurposeThe thalamus is an important brain structure and neurosurgical target, but its constituting nuclei are challenging to image non‐invasively. Recently, susceptibility‐weighted imaging (SWI) at ultra‐high field has shown promising capabilities for thalamic nuclei mapping. In this work, several methodological improvements were explored to enhance SWI quality and contrast, and specifically its ability for thalamic imaging.MethodsHigh‐resolution SWI was performed at 7T in healthy participants, ...
#1Ferenc Honbolygó (ELTE: Eötvös Loránd University)H-Index: 12
#2Andrea KóborH-Index: 5
Last. Valéria Csépe (University of Pannonia)H-Index: 27
view all 4 authors...
Understanding speech at the basic levels entails the simultaneous and independent processing of phonemic and prosodic features. While it is well-established that phoneme perception relies on language-specific long-term traces, it is unclear if the processing of prosodic features similarly involves language-specific representations. In the present study, we investigated the processing of a specific prosodic feature, word stress, using the method of event-related brain potentials (ERPs) employing ...
#1Sandhya Chengaiyan (SSN: Sri Sivasubramaniya Nadar College of Engineering)H-Index: 1
#2Anandha Sree Retnapandian (SSN: Sri Sivasubramaniya Nadar College of Engineering)H-Index: 1
Last. Kavitha Anandan (SSN: Sri Sivasubramaniya Nadar College of Engineering)H-Index: 2
view all 3 authors...
Retrieval of unintelligible speech is a basic need for speech impaired and is under research for several decades. But retrieval of random words from thoughts needs a substantial and consistent approach. This work focuses on the preliminary steps of retrieving vowels from Electroencephalography (EEG) signals acquired while speaking and imagining of speaking a consonant–vowel–consonant (CVC) word. The process, referred to as Speech imagery is imagining of speaking to oneself silently in the mind. ...
1 CitationsSource
#1Damla Arslan-Acaroz (Afyon Kocatepe University)H-Index: 5
#2Nalan Baysu-Sozbilir (Afyon Kocatepe University)
Formaldehyde (HCHO) is a reactive agent and the most essential common carcinogenic environmental pollutant. The present study investigated the protective and ameliorative effects of boric acid (BA) against formaldehyde-induced oxidative stress in A549 cell lines. The first group served as a control, the second group was treated with only 100 μM formaldehyde, and the third, fourth, and fifth groups were treated with 2.5, 5, and 10 mM BA, respectively. The sixth, seventh, and eighth groups were tr...
#1In Soo RyuH-Index: 3
#2Seong Shoon YoonH-Index: 10
Last. Joung-Wook SeoH-Index: 5
view all 15 authors...
3-fluoromethamphetamine (3-FMA), a derivative of methamphetamine (METH), produces behavioral impairment and deficits in dopaminergic transmission in the striatum of mice. The abuse potential of 3-FMA has not been fully characterized. The aim of this study was to evaluate the effects of 3-FMA on locomotor activity as well as its rewarding and reinforcing properties in the conditioned place preference (CPP) and self-administration procedures. Intravenous (i.v.) administration of 3-FMA (0.5 and 1.0...
#1J. Brendan Ritchie (Katholieke Universiteit Leuven)H-Index: 10
#2Hans Op de Beeck (Katholieke Universiteit Leuven)H-Index: 19
Last. Hans P. Op de Beeck (Katholieke Universiteit Leuven)H-Index: 22
view all 2 authors...
A large number of neuroimaging studies have shown that information about object category can be decoded from regions of the ventral visual pathway. One question is how this information might be functionally exploited in the brain. In an attempt to help answer this question, some studies have adopted a neural distance-to-bound approach, and shown that distance to a classifier decision boundary through neural activation space can be used to predict reaction times (RT) on animacy categorization tas...
4 CitationsSource