Match!

Effects of Confidence-Based Rejection on Usability and Error in Pattern Recognition-Based Myoelectric Control

Published on Sep 1, 2019in IEEE Journal of Biomedical and Health Informatics4.217
· DOI :10.1109/JBHI.2018.2878907
Jason W. Robertson2
Estimated H-index: 2
(UNB: University of New Brunswick),
Kevin Englehart24
Estimated H-index: 24
(UNB: University of New Brunswick),
Erik Scheme24
Estimated H-index: 24
(UNB: University of New Brunswick)
Abstract
Rejection of movements based on the confidence in the classification decision has previously been demonstrated to improve the usability of pattern recognition based myoelectric control. To this point, however, the optimal rejection threshold has been determined heuristically, and it is not known how different thresholds affect the tradeoff between error mitigation and false rejections in real-time closed-loop control. To answer this question, 24 able-bodied subjects completed a real-time Fitts’ law-style virtual cursor control task using a support vector machine classifier. It was found that rejection improved information throughput at all thresholds, with the best performance coming at thresholds between 0.60 and 0.75. Two fundamental types of error were defined and identified: operator error (identifiable, repeatable behaviors, directly attributable to the user), and systemic error (other errors attributable to misclassification or noise). The incidence of both operator and systemic errors were found to decrease as rejection threshold increased. Moreover, while the incidence of all error types correlated strongly with path efficiency, only systemic errors correlated strongly with throughput and trial completion rate. Interestingly, more experienced users were found to commit as many errors as novice users, despite performing better in the Fitts' task, suggesting that there is more to usability than error prevention alone. Nevertheless, these results demonstrate the usability gains possible with rejection across a range of thresholds for both novice and experienced users alike.
  • References (42)
  • Citations (8)
📖 Papers frequently viewed together
968 Citations
104 Citations
243 Citations
78% of Scinapse members use related papers. After signing in, all features are FREE.
References42
Newest
#1Levi J. HargroveH-Index: 36
#2Laura A. Miller (NU: Northwestern University)H-Index: 16
Last. Todd A. Kuiken (NU: Northwestern University)H-Index: 45
view all 4 authors...
Background Advances such as targeted muscle reinnervation and pattern recognition control may provide improved control of upper limb myoelectric prostheses, but evaluating user function remains challenging. Virtual environments are cost-effective and immersive tools that are increasingly used to provide practice and evaluate prosthesis control, but the relationship between virtual and physical outcomes—i.e., whether practice in a virtual environment translates to improved physical performance—is...
4 CitationsSource
Jul 1, 2018 in EMBC (International Conference of the IEEE Engineering in Medicine and Biology Society)
#1Jason W. Robertson (UNB: University of New Brunswick)H-Index: 2
#2Kevin Englehart (UNB: University of New Brunswick)H-Index: 24
Last. Erik Scheme (UNB: University of New Brunswick)H-Index: 24
view all 3 authors...
: In myoelectric pattern-recognition control, the rejection of movement decisions based on confidence - the likelihood of a correct classification - has been shown to improve system usability, however it is not known to what extent this is due directly to error mitigation, and to what extent this is due to users having opportunities to change the way they contract. To understand this, 24 subjects participated in a real-time pattern recognition control task with rejection at seven different confi...
3 CitationsSource
#1Ahmed W. Shehata (UNB: University of New Brunswick)H-Index: 4
#2Erik Scheme (UNB: University of New Brunswick)H-Index: 24
Last. Jonathon W. Sensinger (UNB: University of New Brunswick)H-Index: 4
view all 3 authors...
On-going developments in myoelectric prosthesis control have provided prosthesis users with an assortment of control strategies that vary in reliability and performance. Many studies have focused on improving performance by providing feedback to the user but have overlooked the effect of this feedback on internal model development, which is key to improve long-term performance. In this paper, the strength of internal models developed for two commonly used myoelectric control strategies: raw cont...
17 CitationsSource
#1Joseph L. Betthauser (Johns Hopkins University)H-Index: 5
#2Christopher L. Hunt (Johns Hopkins University)H-Index: 3
Last. Nitish V. Thakor (Johns Hopkins University)H-Index: 66
view all 7 authors...
Myoelectric signals can be used to predict the intended movements of an amputee for prosthesis control. However, untrained effects like limb position changes influence myoelectric signal characteristics, hindering the ability of pattern recognition algorithms to discriminate among motion classes. Despite frequent and long training sessions, these deleterious conditional influences may result in poor performance and device abandonment. Goal: We present a robust sparsity-based adaptive classificat...
37 CitationsSource
#1Agamemnon Krasoulis (Edin.: University of Edinburgh)H-Index: 6
#2Iris Kyranou (Edin.: University of Edinburgh)H-Index: 3
Last. Sethu Vijayakumar (Edin.: University of Edinburgh)H-Index: 35
view all 5 authors...
Myoelectric pattern recognition systems can decode movement intention to drive upper-limb prostheses. Despite recent advances in academic research, the commercial adoption of such systems remains low. This limitation is mainly due to the lack of classification robustness and a simultaneous requirement for a large number of electromyogram (EMG) electrodes. We propose to address these two issues by using a multi-modal approach which combines surface electromyography (sEMG) with inertial measuremen...
46 CitationsSource
#1Janne M. HahneH-Index: 10
#2Marko MarkovicH-Index: 14
Last. Dario Farina (Imperial College London)H-Index: 80
view all 3 authors...
State of the art clinical hand prostheses are controlled in a simple and limited way that allows the activation of one function at a time. More advanced laboratory approaches, based on machine learning, offer a significant increase in functionality, but their clinical impact is limited, mainly due to lack of reliability. In this study, we analyse two conceptually different machine learning approaches, focusing on their robustness and performance in a closed loop application. A classification (fi...
41 CitationsSource
#1Adenike A. Adewuyi (NU: Northwestern University)H-Index: 5
#2Levi J. Hargrove (NU: Northwestern University)H-Index: 36
Last. Todd A. KuikenH-Index: 45
view all 3 authors...
Background The use of pattern recognition-based methods to control myoelectric upper-limb prostheses has been well studied in individuals with high-level amputations but few studies have demonstrated that it is suitable for partial-hand amputees, who often possess a functional wrist. This study’s objective was to evaluate strategies that allow partial-hand amputees to control a prosthetic hand while allowing retain wrist function.
10 CitationsSource
#1Robert J. Beaulieu (Johns Hopkins University)H-Index: 11
#2Matthew R. Masters (Johns Hopkins University)H-Index: 5
Last. Alcimar Barbosa Soares (UFU: Federal University of Uberlandia)H-Index: 11
view all 7 authors...
5 CitationsSource
#1Reva E. Johnson (Valpo: Valparaiso University)H-Index: 4
#2Konrad P. Kording (Rehabilitation Institute of Chicago)H-Index: 46
Last. Jonathon W. Sensinger (UNB: University of New Brunswick)H-Index: 4
view all 4 authors...
The objective of this study was to understand how people adapt to errors when using a myoelectric control interface. We compared adaptation across 1) non-amputee subjects using joint angle, joint torque, and myoelectric control interfaces, and 2) amputee subjects using myoelectric control interfaces with residual and intact limbs (five total control interface conditions). We measured trial-by-trial adaptation to self-generated errors and random perturbations during a virtual, single degree-of-fr...
18 CitationsSource
#1Dapeng Yang (HIT: Harbin Institute of Technology)H-Index: 13
#2Yikun Gu (HIT: Harbin Institute of Technology)H-Index: 8
Last. Hong Liu (HIT: Harbin Institute of Technology)H-Index: 28
view all 5 authors...
Abstract In pattern recognition (PR)-based myoelectric control schemes, the classifier is generally trained in ideal laboratory conditions, due to which the classification accuracy might be affected by confounding factors such as force variations, limb positions, and inadvertent electromyography (EMG) activation. Many endeavors have been put forward to mitigate this effect by adopting new training protocols that consider only quite a few independent factors. In this note, we propose a dynamic pr...
12 CitationsSource
Cited By8
Newest
#1Ziyi Yang (Kagawa University)H-Index: 1
#2Guo Shuxiang (BIT: Beijing Institute of Technology)H-Index: 2
Last. Takashi Tamiya (Kagawa University)H-Index: 34
view all 5 authors...
Bilateral rehabilitation training robotic systems have potential to promote the upper limb motor recovery of post-stroke hemiparesis patients through providing the synchronization motion between the impaired limb and contralateral limb. The active rehabilitation training based on patients’ intention can also promote the recovery effect by stimulating the activity of the ipsilateral hemisphere and contralateral hemisphere. In this paper, a novel intention-based bilateral training system using bio...
Source
#1Joseph L. Betthauser (Johns Hopkins University)H-Index: 5
#2John T. Krall (Johns Hopkins University)H-Index: 1
Last. Nitish V. Thakor (Johns Hopkins University)H-Index: 66
view all 7 authors...
Prediction of movement intentions from electromyographic (EMG) signals is typically performed with a pattern recognition approach, wherein a short dataframe of raw EMG is compressed into an instantaneous feature-encoding that is meaningful for classification. However, EMG signals are time-varying, implying that a frame-wise approach may not sufficiently incorporate temporal context into predictions, leading to erratic and unstable prediction behavior. Goal: We demonstrate that sequential predict...
Source
#1Evan Campbell (UNB: University of New Brunswick)H-Index: 3
#2Angkoon Phinyomark (UNB: University of New Brunswick)H-Index: 23
Last. Erik Scheme (UNB: University of New Brunswick)H-Index: 24
view all 3 authors...
This manuscript presents a hybrid study of a comprehensive review and a systematic(research) analysis. Myoelectric control is the cornerstone ofmany assistive technologies used in clinicalpractice, such as prosthetics and orthoses, and human-computer interaction, such as virtual reality control.Although the classification accuracy of such devices exceeds 90% in a controlled laboratory setting,myoelectric devices still face challenges in robustness to variability of daily living conditions.The in...
4 CitationsSource
#1Agamemnon Krasoulis (Edin.: University of Edinburgh)H-Index: 6
#2Sethu Vijayakumar (Newcastle University)H-Index: 35
Last. Kianoush Nazarpour (Newcastle University)H-Index: 18
view all 3 authors...
In the field of upper-limb myoelectric prosthesis control, the use of statistical and machine learning methods has been long proposed as a means of enabling intuitive grip selection and actuation. Recently, this paradigm has found its way toward commercial adoption. Machine learning-based prosthesis control typically relies on the use of a large number of electrodes. Here, we propose an end-to-end strategy for multi-grip, classification-based prosthesis control using only two sensors, comprising...
1 CitationsSource
#1Agamemnon Krasoulis (Newcastle University)H-Index: 6
#2Sethu Vijayakumar (Edin.: University of Edinburgh)H-Index: 35
Last. Kianoush Nazarpour (Newcastle University)H-Index: 18
view all 3 authors...
In the field of upper-limb myoelectric prosthesis control, the use of statistical and machine learning methods has been long proposed as a means of enabling intuitive grip selection and activation; yet, clinical adoption remains rather limited. One of the main causes hindering clinical translation of machine learning-based prosthesis control is the requirement for a large number of electromyography (EMG) sensors. Here, we propose an end-to-end strategy for multi-grip, classification-based prosth...
1 CitationsSource
#1Tianzhe Bao (University of Leeds)H-Index: 1
#2Syed Ali Raza Zaidi (University of Leeds)H-Index: 13
Last. Zhiqiang Zhang (Xidian University)
view all 5 authors...
Convolutional neural network (CNN) has been widely exploited for simultaneous and proportional myoelectric control due to its capability of deriving informative, representative and transferable features from surface electromyography (sEMG). However, muscle contractions have strong temporal dependencies but conventional CNN can only exploit spatial correlations. Considering that long short-term memory neural network (LSTM) is able to capture long-term and non-linear dynamics of time-series data, ...
#1Evan Campbell (UNB: University of New Brunswick)H-Index: 3
#2Angkoon Phinyomark (UNB: University of New Brunswick)H-Index: 23
Last. Erik Scheme (UNB: University of New Brunswick)H-Index: 24
view all 3 authors...
The linear discriminant analysis (LDA) classifier remains a standard in myoelectric control due to its simplicity, ease of implementation, and robustness. Despite this, challenges associated with the temporal evolution of the myoelectric signal may require flexibility beyond the capabilities of standard LDA. Recently proposed approaches have leveraged more complex systems, such as adaptive window framing or temporal convolutional neural networks to incorporate temporal structure. In this work, w...
3 CitationsSource
#1Lizhi Pan (NCSU: North Carolina State University)H-Index: 5
#2Dustin L. Crouch (NCSU: North Carolina State University)H-Index: 9
Last. He Huang (NCSU: North Carolina State University)H-Index: 25
view all 3 authors...
Electromyography (EMG)-based interfaces are trending toward continuous, simultaneous control with multiple degrees of freedom. Emerging methods range from data-driven approaches to biomechanical model-based methods. However, there has been no direct comparison between these two types of continuous EMG-based interfaces. The aim of this study was to compare a musculoskeletal model (MM) with two data-driven approaches, linear regression (LR) and artificial neural network (ANN), for predicting conti...
2 CitationsSource