融合影像与时域特征的多维注意力机制增强的跨模态sEMG手势识别方法*

doi:10.16731/j.cnki.1671-3133.2026.08.017

现代制造工程 ›› 2026, Vol. 551 ›› Issue (8): 122-131.doi: 10.16731/j.cnki.1671-3133.2026.08.017

• 仪器仪表/检测/监控 • 上一篇下一篇

融合影像与时域特征的多维注意力机制增强的跨模态sEMG手势识别方法^*

黄斌浩¹, 吕健¹, 强利刚²

1 贵州大学现代制造技术教育部重点实验室,贵阳 550025;
2 贵州航天控制技术有限公司,贵阳 116026

收稿日期:2025-04-08 出版日期:2026-08-18 发布日期:2026-09-01
通讯作者: 吕健,副教授,硕士生导师,博士,主要研究方向为人机交互、智能设计。 E-mail:jlv@gzu.edu.cn
作者简介:黄斌浩,硕士研究生,主要研究方向为外骨骼机器人设计、智能交互。
基金资助:
^*贵州省科技厅科学支撑计划项目(黔科合支撑[2022]一般197)

Cross-modal sEMG gesture recognition method enhanced by multi-dimensional attention mechanism integrating image and temporal features

HUANG Binhao¹, LÜ Jian¹, QIANG Ligang²

1 Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education,Guizhou University,Guiyang 550025,China;
2 Guizhou Aerospace Control Technology Co.,Ltd.,Guiyang 116026,China

Received:2025-04-08 Online:2026-08-18 Published:2026-09-01

摘要/Abstract

摘要： 现有表面肌电(surface Electromyography,sEMG)信号手势识别方法在时空特征提取和多模态信息融合方面存在局限,传统二维卷积神经网络(2D Convolutional Neural Network,2D CNN)和长短期记忆网络(Long Short-Term Memory,LSTM)难以精准分类复杂手势。为解决这一问题,提出了一种基于sEMG灰度图和多维注意力机制的跨模态手势识别方法。通过将sEMG信号转化为灰度特征图和时域特征表格,并结合3D ResNet-18网络进行时空特征提取,同时引入TabAttention模块和CSRA残差注意力机制,以增强特征融合和空间细节捕捉。实验结果表明,所提方法在6种手势识别中达到98.13 %的准确率,优于2D CNN、LSTM、AlexNet和VGGNet模型,为sEMG手势识别提供了更高效的解决方案。

关键词: 表面肌电, 3D卷积神经网络, 时空特征提取, 跨模态信息融合, 手势分类

Abstract: Existing surface Electromyography (sEMG) gesture recognition methods exhibit certain limitations in spatiotemporal feature extraction and multimodal information fusion,with traditional 2D Convolutional Neural Network (3D CNN) and Long Short-Term Memory (LSTM) models struggling to accurately distinguish complex gestures. To address this issue,it proposes a cross-modal sEMG gesture recognition method based on sEMG grayscale images and a multi-dimensional attention mechanism. The proposed method transforms sEMG signals into grayscale feature maps and tabular time-domain features,utilizing a 3D ResNet-18 model for spatiotemporal feature extraction. Additionally,TabAttention and CSRA residual attention mechanisms are introduced to enhance feature fusion and optimize spatial detail extraction. Experimental results demonstrate that the proposed approach achieves an accuracy of 98.13 % in recognizing six types of gestures,significantly outperforming 2D CNN,LSTM,AlexNet,and VGGNet,thus providing a more superior solution for sEMG gesture recognition.

Key words: surface Electromyography (sEMG), 3D Convolutional Neural Network (3D CNN), spatiotemporal feature extraction, cross-modal information fusion, gesture classification

中图分类号:

TP391.4

黄斌浩, 吕健, 强利刚. 融合影像与时域特征的多维注意力机制增强的跨模态sEMG手势识别方法^*[J]. 现代制造工程, 2026, 551(8): 122-131.

HUANG Binhao, LÜ Jian, QIANG Ligang. Cross-modal sEMG gesture recognition method enhanced by multi-dimensional attention mechanism integrating image and temporal features[J]. Modern Manufacturing Engineering, 2026, 551(8): 122-131.

参考文献

[1] PYUN K R,KWON K,YOO M J,et al. Machine-learned wearable sensors for real-time hand-motion recognition:toward practical applications[J]. National Science Review,2024,11(2):1-33.
[2] HUANG H,ZHANG S,CHEN Y,et al. Portable sEMG-Controlled Hand Electrical Stimulation System[J]. IEEE Sensors Journal,2024,25(2):2839-2850.
[3] SHARMA S,FAISAL K N,SHARMA R R. Automated Grasp Recognition using sEMG:Recent Advances,Challenges and Future Developments[J]. IEEE Transactions on Instrumentation and Measurement,2024,74:1-17.
[4] WEI Z,ZHANG Z Q,XIE S Q. Continuous motion intention prediction using sEMG for upper-limb rehabilitation:a systematic review of model-based and model-free approaches[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering,2024,32:1487-1504.
[5] 李振江,魏德健,冯妍妍,等. 表面肌电手部动作识别的研究进展[J]. 计算机工程与应用,2024,60(3):29-43.
[6] NARAYAN Y. SEMG signal classification using KNN classifier with FD and TFD features[J]. Materials Today:Proceedings,2021,37:3219-3225.
[7] ANTUVAN C W,MASIA L. An LDA-based approach for real-time simultaneous classification of movements using surface electromyography[J]. IEEE transactions on neural systems and rehabilitation engineering,2019,27(3):552-561.
[8] NARAYAN Y. Comparative analysis of SVM and Naive Bayes classifier for the SEMG signal classification[J]. Materials Today:Proceedings,2021,37:3241-3245.
[9] 周小波,邹任玲,卢旭华,等. 基于人工神经网络的表面肌电信号分类器研究进展[J]. 电子科技,2021,34(2):62-67.
[10] 卫文韬,李亚军. 基于双流卷积神经网络的肌电信号手势识别方法[J]. 计算机集成制造系统,2022,28(1):124-131.
[11] 骆俊锦,王万良,王铮,等. 基于时序二维化和卷积特征融合的表面肌电信号分类方法[J]. 模式识别与人工智能,2020,33(7):588-599.
[12] 许留凯,张克勤,徐兆红,等.基于表面肌电信号能量核相图的卷积神经网络人体手势识别算法[J].生物医学工程学杂志,2021,38(4):621-629.
[13] 周国良,张道辉,郭小萍.基于TiCNN-DRSN模型的sEMG手势识别算法的研究[J].电子测量技术,2024,47(6):190-196.
[14] LIU X,HU L,TIE L,et al. Integration of Convolutional Neural Network and Vision Transformer for gesture recognition using sEMG[J]. Biomedical Signal Processing and Control,2024,98:106686.
[15] LIU T,BAI D,MA L,et al. Complex Surface Electromyography Signal Gesture Recognition Based on Multi-Pathway Featured Scale Convolutional Neural Network[J]. IEEE Transactions on Instrumentation and Measurement,2024,73:1-11.
[16] QINGZHENG C,QING T,MUCHAO Z,et al. CNN-based gesture recognition using raw numerical gray-scale images of surface electromyography[J]. Biomedical Signal Processing and Control,2025,101:107176.
[17] KINGMA D P,BA J. Adam:A method for stochastic optimization[J]. arXiv preprint arXiv,2014,14(12):1-15.
[18] CHUA L O. CNN:A vision of complexity[J]. International Journal of Bifurcation and Chaos,1997,7(10):2219-2425.
[19] GREFF K,SRIVASTAVA R K,KOUTNÍK J,et al. LSTM:A search space odyssey[J]. IEEE transactions on neural networks and learning systems,2016,28(10):2222-2232.
[20] ALOM M Z,TAHA T M,YAKOPCIC C,et al. The history began from alexnet:A comprehensive survey on deep learning approaches[J]. arXiv preprint arXiv,2018,18(3):1-39.
[21] WANG L,GUO S,HUANG W,et al. Places205-vggnet models for scene recognition[J]. arXiv preprint arXiv,2015,15(8):1-2.

融合影像与时域特征的多维注意力机制增强的跨模态sEMG手势识别方法^*

Cross-modal sEMG gesture recognition method enhanced by multi-dimensional attention mechanism integrating image and temporal features

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics