Intent inferral on a hand orthosis for stroke patients is challenging due to the difficulty of data collection from impaired subjects. Additionally, EMG signals exhibit significant variations across different conditions, sessions, and subjects, making it hard for classifiers to generalize. Traditional approaches require a large labeled dataset from the new condition, session, or subject to train intent classifiers; however, this data collection process is burdensome and time-consuming. In this paper, we propose ChatEMG, an autoregressive generative model that can generate synthetic EMG signals conditioned on prompts (i.e., a given sequence of EMG signals). ChatEMG enables us to collect only a small dataset from the new condition, session, or subject and expand it with synthetic samples conditioned on prompts from this new context. ChatEMG leverages a vast repository of previous data via generative training while still remaining context-specific via prompting. Our experiments show that these synthetic samples are classifier-agnostic and can improve intent inferral accuracy for different types of classifiers. We demonstrate that our complete approach can be integrated into a single patient session, including the use of the classifier for functional orthosis-assisted tasks. To the best of our knowledge, this is the first time an intent classifier trained partially on synthetic data has been deployed for functional control of an orthosis by a stroke survivor.
@article{xu2024chatemg,
title={ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke},
author={Xu, Jingxi and Wang, Runsheng and Shang, Siqi and Chen, Ava and Winterbottom, Lauren and Hsu, To-Liang and Chen, Wenxi and Ahmed, Khondoker and La Rotta, Pedro Leandro and Zhu, Xinyue and others},
journal={arXiv preprint arXiv:2406.12123},
year={2024}
}Our orthosis is an exotendon device consisting of a forearm splint and fingertip components. These two parts are connected through an exotendon network. The device is underactuated, moving all of the fingers at once. When the motor retracts the tendons, the subject’s fingers extend. When the motor releases the tendons, the subject uses their own grip strength to close their hand. Additional details can be found here.
ChatEMG operates on the “language” of EMG data, and it generates synthetic EMG signals in response to EMG prompts. This approach is similar in concept to language models such as ChatGPT, which is capable of generating text in response to a text prompt, hence its chosen moniker.
The Fugl-Meyer assessment is a stroke-specific, clinically validated, performance-based index for motor impairment. It is split into upper and lower extremity components, and we use the upper extremity subscale FM-UE.
The stroke participants in our study can fully close their hands but are unable to completely extend their fingers without assistance. The passive range of motion in the fingers is within functional limits.
We visualize synthetic samples for all five stroke patients across three intents. We also present their gender, age, and Fugl-Meyer scores for the upper extremity (FM-UE) in parentheses. The first plot shows the 150-step (1.5s) prompts provided to ChatEMG. The second and third plots show the real and synthetic data, respectively. The vertical dashed line indicates when the prompt ends. In the synthetic plot, the model generates the synthetic data after the vertical line. In the real plot, the entire data (both before and after the vertical line) is real.
This work was supported in part by the National Institutes of Health (R01NS115652, F31HD111301) and the CU Data Science Institute.