A Convolutional Neural Network for the Detection of Asynchronous Steady State Motion Visual Evoked Potential

Published on Jun 1, 2019in IEEE Transactions on Neural Systems and Rehabilitation Engineering3.478
· DOI :10.1109/TNSRE.2019.2914904
Xin Zhang1
Estimated H-index: 1
(Xi'an Jiaotong University),
Guanghua Xu16
Estimated H-index: 16
(Xi'an Jiaotong University)
+ 4 AuthorsNing Jiang29
Estimated H-index: 29
(UW: University of Waterloo)
A key issue in brain-computer interface (BCI) is the detection of intentional control (IC) states and non-intentional control (NC) states in an asynchronous manner. Furthermore, for steady-state visual evoked potential (SSVEP) BCI systems, multiple states (sub-states) exist within the IC state. Existing recognition methods rely on a threshold technique, which is difficult to realize high accuracy, i.e., simultaneously high true positive rate and low false positive rate. To address this issue, we proposed a novel convolutional neural network (CNN) to detect IC and NC states in a SSVEP-BCI system for the first time. Specifically, the steady-state motion visual evoked potentials (SSMVEP) paradigm, which has been shown to induce less visual discomfort, was chosen as the experimental paradigm. Two processing pipelines were proposed for the detection of IC and NC states. The first one was using CNN as a multi-class classifier to discriminate between all the states in IC and NC state (FFT-CNN). The second one was using CNN to discriminate between IC and NC states, and using canonical correlation analysis (CCA) to perform classification tasks within the IC (FFT-CNN-CCA). We demonstrated that both pipelines achieved a significant increase in accuracy for low-performance healthy participants when traditional algorithms such as CCA threshold were used. Furthermore, the FFT-CNN-CCA pipeline achieved better performance than the FFT-CNN pipeline based on the stroke patients’ data. In summary, we showed that CNN can be used for robust detection in an asynchronous SSMVEP-BCI with great potential for out-of-lab BCI applications.
  • References (32)
  • Citations (1)
📖 Papers frequently viewed together
2018SMC: Systems, Man and Cybernetics
20172.78PLOS ONE
3 Authors (No Sang Kwak, ..., Sl Whan)
30 Citations
2019EMBC: International Conference of the IEEE Engineering in Medicine and Biology Society
4 Authors (Aravind Ravi, ..., Ning Jiang)
78% of Scinapse members use related papers. After signing in, all features are FREE.
#1Kaori Suefusa (H.I., S.I.: University of Agriculture, Faisalabad)H-Index: 3
#2Tanaka Toshihisa (TUAT: Tokyo University of Agriculture and Technology)H-Index: 21
Objective: One of the challenges in the area of brain–computer interfacing (BCI) is to develop an asynchronous BCI or a self-paced BCI that detects whether a user intends to pass messages. This paper proposes a novel asynchronous BCI that uses mixed frequency and phase-coded visual stimuli, which can provide high-speed and accurate command entries. Methods: The mixed-coded visual stimuli were presented as flickers with a following blank interval to synchronize the recorder of electroencephalogra...
9 CitationsSource
#1Wenqiang YanH-Index: 3
#2Guanghua Xu (Xi'an Jiaotong University)H-Index: 16
Last. Ziyan DanH-Index: 1
view all 5 authors...
Objective : The purpose of this paper was to study the applicability of paradigms with motion forms for use in a brain–computer interface (BCI). We examined the performances of different paradigms and evaluated the stimulus effects. Methods : We designed four novel stimulus paradigms based on basic motion modes: swing, rotation, spiral, and radial contraction–expansion. Canonical correlation analysis (CCA) was used to analyze the accuracy. Additionally, we optimized CCA template signal harmonic ...
9 CitationsSource
Deep learning has demonstrated tremendous success in variety of application domains in the past few years. This new field of machine learning has been growing rapidly and applied in most of the application domains with some new modalities of applications, which helps to open new opportunity. There are different methods have been proposed on different category of learning approaches, which includes supervised, semi-supervised and un-supervised learning. The experimental results show state-of-the-...
52 Citations
#1Mingfei Liu (SCUT: South China University of Technology)H-Index: 1
#2Wei Wu (SCUT: South China University of Technology)H-Index: 10
Last. Yuanqing Li (SCUT: South China University of Technology)H-Index: 30
view all 6 authors...
Detecting P300 signals from electroencephalography (EEG) is the key to establishing a P300 speller, which is a type of braincomputer interface (BCI) system based on the oddball paradigm that allows users to type messages simply by controlling eye-gazes. The convolutional neural network (CNN) is an approach that has achieved good P300 detection performances. However, the standard CNN may be prone to overfitting and the convergence may be slow. To address these issues, we develop a novel CNN, term...
24 CitationsSource
#1Xin Zhang (Xi'an Jiaotong University)H-Index: 3
#2Guanghua Xu (Xi'an Jiaotong University)H-Index: 16
Last. Xun Zhang (Xi'an Jiaotong University)H-Index: 1
view all 4 authors...
Steady state visual evoked potential (SSVEP)-based brain computer interface (BCI) has advantages of high information transfer rate (ITR), less electrodes and little training. So it has been widely investigated. However, the available stimulus frequencies are limited by brain responses. Simultaneous modulation of stimulus luminance is a novel method to resolve this problem. In this study, three experiments were devised to gain a deeper understanding of the brain response to the stimulation using ...
5 CitationsSource
#1Marina TaniH-Index: 1
#2Yumie Ono (Meiji University)H-Index: 13
Last. Takanori TominagaH-Index: 5
view all 7 authors...
Abstract Motor imagery (MI) is a mental practice that reproduces the visual- and/or kinesthetic-modality brain activations accompanying movement. It is a useful rehabilitation technique as the affected motor cortex can be stimulated in patients with stroke and hemiplegia. However, most patients with stroke have difficulty with MI owing to advanced age and/or higher-cognitive dysfunction, thus impairing their ability to internally simulate the action. We therefore investigated whether action obse...
12 CitationsSource
#1Shenghong He (SCUT: South China University of Technology)H-Index: 4
#2Rui Zhang (SCUT: South China University of Technology)H-Index: 6
Last. Yuanqing Li (SCUT: South China University of Technology)H-Index: 4
view all 9 authors...
The key issue of electroencephalography (EEG)-based brain switches is to detect the control and idle states in an asynchronous manner. Most existing methods rely on a threshold. However, it is often time consuming to select a satisfactory threshold, and the chosen threshold might be inappropriate over a long period of time due to the variability of the EEG signals. This paper presents a new P300-based threshold-free brain switch. Specifically, one target button and three pseudo buttons, which ar...
18 CitationsSource
#1No Sang Kwak (KU: Korea University)H-Index: 2
#2Klaus-Robert Müller (Technical University of Berlin)H-Index: 92
Last. Sl Whan (KU: Korea University)H-Index: 42
view all 3 authors...
30 CitationsSource
Jun 1, 2016 in CVPR (Computer Vision and Pattern Recognition)
#1Kaiming He (Microsoft)H-Index: 57
#2Xiangyu Zhang (Xi'an Jiaotong University)H-Index: 25
Last. Jian Sun (Microsoft)H-Index: 88
view all 4 authors...
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the...
14k CitationsSource
#1No Sang Kwak (KU: Korea University)H-Index: 2
#2Klaus-Robert Müller (Technical University of Berlin)H-Index: 92
Last. Sl Whan (KU: Korea University)H-Index: 42
view all 3 authors...
Objective. We have developed an asynchronous brain–machine interface (BMI)-based lower limb exoskeleton control system based on steady-state visual evoked potentials (SSVEPs). Approach. By decoding electroencephalography signals in real-time, users are able to walk forward, turn right, turn left, sit, and stand while wearing the exoskeleton. SSVEP stimulation is implemented with a visual stimulation unit, consisting of five light emitting diodes fixed to the exoskeleton. A canonical correlation ...
65 CitationsSource
Cited By1
OBJECTIVE: We presented a comparative study on the training methodologies of Convolutional Neural Network (CNN) for detection of steady-state visual evoked potentials (SSVEP). Two training scenarios were also compared: user-independent (UI) training and user-dependent (UD) training. APPROACH: The CNN was trained in both UD and UI scenarios on two types of features for SSVEP classification: magnitude spectrum features (M-CNN) and complex spectrum features (C-CNN). And the Canonical Correlation An...
#1Li Wang (GU: Guangzhou University)
#2Weijian Huang (GU: Guangzhou University)
Last. Chun Zhang (SEU: Southeast University)
view all 4 authors...
Abstract With the help of brain-computer interface (BCI) systems, the electroencephalography (EEG) signals can be translated into control commands. It is rare to extract temporal-spatial-frequency features of the EEG signals at the same time by conventional deep neural networks. In this study, two types of series and parallel structures are proposed by combining convolutional neural network (CNN) and long short term memory (LSTM). The frequency and spatial features of EEG are extracted by CNN, a...
#1Phairot AutthasanH-Index: 1
#2Xiangqian Du (TITech: Tokyo Institute of Technology)
view all 9 authors...
Brain-Computer interfaces (BCIs) play a significant role in easing neuromuscular patients on controlling computers and prosthetics. Due to their high signal-to-noise ratio, steadystate visually evoked potentials (SSVEPs) has been widely used to build BCIs. However, currently developed algorithms do not predict the modulation of SSVEP amplitude, which is known to change as a function of stimulus luminance contrast. In this study, we aim to develop an integrated approach to simultaneously estimate...
2 CitationsSource
Abstract The construction of smart cities in our country has received extensive attention. Under the situation that smart cities are vigorously promoted nowadays, compared with traditional construction and operation and maintenance methods, building information model (BIM) technology is more suitable to serve as an important foundation for intelligent management in the whole process of construction projects. BIM is an abbreviation for building information model. BIM relies on a variety of digita...
The steady-state visual-evoked potential-based brain-computer interface (SSVEP-BCI) is a typically recognized visual stimulus frequency from brain responses. Each frequency represents one command to control a machine. For example, multiple target stimuli with different frequencies can be used to control the moving speeds of a robot. Each target stimulus frequency corresponds to a speed level. Such a conventional SSVEP-BCI is choice selection paradigm with discrete information, allowing users to ...