Match!

Sequence Transfer Learning for Neural Decoding

Published on Feb 15, 2018in bioRxiv
· DOI :10.1101/210732
Venkatesh Elango1
Estimated H-index: 1
(UCSD: University of California, San Diego),
Aashish N. Patel1
Estimated H-index: 1
(UCSD: University of California, San Diego)
+ 1 AuthorsVikash Gilja21
Estimated H-index: 21
(UCSD: University of California, San Diego)
Cite
Abstract
A fundamental challenge in designing brain-computer interfaces (BCIs) is decoding behavior from time-varying neural oscillations. In typical applications, decoders are constructed for individual subjects and with limited data leading to restrictions on the types of models that can be utilized. Currently, the best performing decoders are typically linear models capable of utilizing rigid timing constraints with limited training data. Here we demonstrate the use of Long Short-Term Memory (LSTM) networks to take advantage of the temporal information present in sequential neural data collected from subjects implanted with electrocorticographic (ECoG) electrode arrays performing a finger flexion task. Our constructed models are capable of achieving accuracies that are comparable to existing techniques while also being robust to variation in sample data size. Moreover, we utilize the LSTM networks and an affine transformation layer to construct a novel architecture for transfer learning. We demonstrate that in scenarios where only the affine transform is learned for a new subject, it is possible to achieve results comparable to existing state-of-the-art techniques. The notable advantage is the increased stability of the model during training on novel subjects. Relaxing the constraint of only training the affine transformation, we establish our model as capable of exceeding performance of current models across all training data sizes. Overall, this work demonstrates that LSTMs are a versatile model that can accurately capture temporal patterns in neural data and can provide a foundation for transfer learning in neural decoding.
  • References (0)
  • Citations (3)
Cite
References0
Newest
Published on Aug 26, 2019in Nature Human Behaviour
Kai J. Miller39
Estimated H-index: 39
(Stanford University)
Electrophysiological data from implanted electrodes in the human brain are rare, and therefore scientific access to such data has remained somewhat exclusive. Here we present a freely available curated library of implanted electrocorticographic data and analyses for 16 behavioural experiments, with 204 individual datasets from 34 patients recorded with the same amplifiers and at the same settings. For each dataset, electrode positions were carefully registered to brain anatomy. A large set of fu...
Published on Feb 21, 2017in eLife7.55
Chethan Pandarinath13
Estimated H-index: 13
,
Paul Nuyujukian22
Estimated H-index: 22
+ 6 AuthorsJaimie M. Henderson39
Estimated H-index: 39
(Stanford University)
People with various forms paralysis not only have difficulties getting around, but also are less able to use many communication technologies including computers. In particular, strokes, neurological injuries, or diseases such as ALS can lead to severe paralysis and make it very difficult to communicate. In rare instances, these disorders can result in a condition called locked-in syndrome, in which the affected person is aware but completely unable to move or speak. Several researchers are looki...
Published on Dec 1, 2016in Nature Communications11.88
David Sussillo18
Estimated H-index: 18
,
Sergey D. Stavisky11
Estimated H-index: 11
+ 2 AuthorsKrishna V. Shenoy52
Estimated H-index: 52
Brain-machine interfaces (BMI) depend on algorithms to decode neural signals, but these decoders cope poorly with signal variability. Here, authors report a BMI decoder which circumvents these problems by using a large and perturbed training dataset to improve performance with variable neural signals.
Published on Nov 24, 2016in The New England Journal of Medicine70.67
Mariska J. Vansteensel20
Estimated H-index: 20
,
Elmar Pels2
Estimated H-index: 2
+ 10 AuthorsMax van den Boom1
Estimated H-index: 1
Options for people with severe paralysis who have lost the ability to communicate orally are limited. We describe a method for communication in a patient with late-stage amyotrophic lateral sclerosis (ALS), involving a fully implanted brain–computer interface that consists of subdural electrodes placed over the motor cortex and a transmitter placed subcutaneously in the left side of the thorax. By attempting to move the hand on the side opposite the implanted electrodes, the patient accurately a...
Published on Apr 1, 2016in Journal of Neural Engineering4.55
Guy Hotson6
Estimated H-index: 6
(Johns Hopkins University),
David P. McMullen7
Estimated H-index: 7
(Johns Hopkins University)
+ 8 AuthorsBrock A. Wester9
Estimated H-index: 9
Objective. We used native sensorimotor representations of fingers in a brain–machine interface (BMI) to achieve immediate online control of individual prosthetic fingers. Approach. Using high gamma responses recorded with a high-density electrocorticography (ECoG) array, we rapidly mapped the functional anatomy of cued finger movements. We used these cortical maps to select ECoG electrodes for a hierarchical linear discriminant analysis classification scheme to predict: (1) if any finger was mov...
Published on May 1, 2015in NeuroImage5.81
Hiroshi Morioka6
Estimated H-index: 6
(Kyoto University),
Atsunori Kanemura5
Estimated H-index: 5
(AIST: National Institute of Advanced Industrial Science and Technology)
+ 5 AuthorsShin Ishii30
Estimated H-index: 30
(Kyoto University)
Abstract Brain signals measured over a series of experiments have inherent variability because of different physical and mental conditions among multiple subjects and sessions. Such variability complicates the analysis of data from multiple subjects and sessions in a consistent way, and degrades the performance of subject-transfer decoding in a brain–machine interface (BMI). To accommodate the variability in brain signals, we propose 1) a method for extracting spatial bases (or a dictionary) sha...
Published on Jan 1, 2015in arXiv: Neural and Evolutionary Computing
Wojciech Zaremba23
Estimated H-index: 23
,
Ilya Sutskever40
Estimated H-index: 40
Recurrent Neural Networks (RNNs) with Long Short-Term Memory units (LSTM) are widely used because they are expressive and are easy to train. Our interest lies in empirically evaluating the expressiveness and the learnability of LSTMs in the sequence-to-sequence regime by training them to evaluate short computer programs, a domain that has traditionally been seen as too complex for neural networks. We consider a simple class of programs that can be evaluated with a single left-to-right pass using...
Published on Jan 1, 2015 in ICML (International Conference on Machine Learning)
Rafal Jozefowicz9
Estimated H-index: 9
(Google),
Wojciech Zaremba23
Estimated H-index: 23
(NYU: New York University),
Ilya Sutskever40
Estimated H-index: 40
(Google)
The Recurrent Neural Network (RNN) is an extremely powerful sequence model that is often difficult to train. The Long Short-Term Memory (LSTM) is a specific RNN architecture whose design makes it much easier to train. While wildly successful in practice, the LSTM's architecture appears to be ad-hoc so it is not clear if it is optimal, and the significance of its individual components is unclear. In this work, we aim to determine whether the LSTM architecture is optimal or whether much better arc...
Published on Jan 1, 2015 in ICML (International Conference on Machine Learning)
Nitish Srivastava13
Estimated H-index: 13
(U of T: University of Toronto),
Elman Mansimov5
Estimated H-index: 5
(U of T: University of Toronto),
Ruslan Salakhudinov3
Estimated H-index: 3
(U of T: University of Toronto)
We use Long Short Term Memory (LSTM) networks to learn representations of video sequences. Our model uses an encoder LSTM to map an input sequence into a fixed length representation. This representation is decoded using single or multiple decoder LSTMs to perform different tasks, such as reconstructing the input sequence, or predicting the future sequence. We experiment with two kinds of input sequences - patches of image pixels and high-level representations ("percepts") of video frames extract...
Ilya Sutskever40
Estimated H-index: 40
,
Oriol Vinyals39
Estimated H-index: 39
,
Quoc V. Le62
Estimated H-index: 62
Deep Neural Networks (DNNs) are powerful models that have achieved excellent performance on difficult learning tasks. Although DNNs work well whenever large labeled training sets are available, they cannot be used to map sequences to sequences. In this paper, we present a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure. Our method uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensiona...
Cited By3
Newest
Published on Jun 1, 2019in Agricultural Water Management3.54
G.I. Ezenne (Rhodes University), Louise Jupp + 1 AuthorsJ.L. Tanner (Rhodes University)
Abstract In order to feed growing populations under scare water resources, a suitable technology that improves crop water productivity (CWP) is crucial. Precision agriculture that utilizes digital techniques such as unmanned aerial systems (UAS) can play a significant role in improving CWP. CWP is an important indicator that quantifies the effect of agricultural water management. To improve CWP, implementation of suitable methods for early detection of crop water stress before irreversible damag...
Published on Aug 27, 2018in Frontiers in Neuroscience3.65
Gang Pan28
Estimated H-index: 28
,
Jia-Jun Li1
Estimated H-index: 1
+ 5 AuthorsShaomin Zhang18
Estimated H-index: 18