Research on Multimodal Interaction System of Lower Limb Exoskeleton Oriented to Competitive Training

doi:10.3901/JME.2023.05.029

Abstract

Abstract: Currently, there is a lack of research on wearable devices that focus on assisting lower limbs to improve physical fitness training. The synergistic research of exoskeleton and virtual reality is an emerging direction in recent years. The exoskeleton system with multimodal information fusion can optimize the training effect and experience. Therefore, a multimodal interaction system of the lower limb exoskeleton for athletic training is developed. Which not only provides a wide variety of standardized training but also realizes a multisensory immersion experience. Firstly, a multi-degree-of-freedom step-by-step action planning method is designed to reproduce training actions based on lower limb movement action points. Second, a multimodal interaction system for the lower limb exoskeleton is proposed to realize the presentation and indication of normative movements in information space through a virtual reality athletic training simulator. The action data set and the lower limb exoskeleton robot are constructed to realize the assistance and correction of basic training in physical space. The system is based on a multimodal interaction execution strategy to give trainers visual, auditory, and tactile multisensory feedback in real-time to achieve an immersive experience that meets safety norms. Finally, the system is evaluated through two experiments on system functionality and user experience. The functional experiment proves that the system can improve the action accuracy rate by 27.62% on average compared with the original traditional training, and has certain generality. The experimental results of user experience show that the system’s function meets the design expectations, and the comfort compared to other indicators needs to be improved.

Key words: lower limb exoskeleton robot, virtual reality, action data set, multimodal interaction

CLC Number:

TG802

QI Wenqian, SUN Shouqian, CHEN Chao, ZHANG Yebingqing, ZHAO Dongwei. Research on Multimodal Interaction System of Lower Limb Exoskeleton Oriented to Competitive Training[J]. Journal of Mechanical Engineering, 2023, 59(5): 29-40.

References

[1] 卢形. 助力型下肢外骨骼人体意图识别和运动控制研究[D]. 杭州:浙江大学,2019. LU Xing. Research on the human intention control for the lower recognition and motion exoskeleton robot[D]. Hangzhou:Zhejiang University,2019.
[2] WU T M,WANG S Y,CHEN D Z. Design of an exoskeleton for strengthening the upper limb muscle for overextension injury prevention[J]. Mechanism and Machine Theory,2011, 46(12):1825-1839.
[3] CHIYOHARA S,FURUKAWA J,NODA T,et al. Passive training with upper extremity exoskeleton robot affects proprioceptive acuity and performance of motor learning[J]. Scientific Reports,2020,10(1):1-9.
[4] BÜSCHING I,SEHLE A,STÜRNER J,et al. Using an upper extremity exoskeleton for semi-autonomous exercise during inpatient neurological rehabilitation-a pilot study[J]. Journal of Neuroengineering and Rehabilitation,2018, 15(1):1-7.
[5] KOVACS M S,PRITCHETT R,WICKWIREP J,et al. Physical performance changes after unsupervised training during the autumn/spring semester break in competitive tennis players[J]. British Journal of Sports Medicine, 2007,41(11):705-710.
[6] KLAMROTH-MARGANSKA V,BAUR K,NAGLE A,et al. Strength training of the upper extremity in virtual reality with an exoskeleton robot[C]//2017 International Conference on Virtual Rehabilitation (ICVR). IEEE, 2017:1-6.
[7] 刘鹏. 面向康复的上肢外骨骼机器人训练系统范式设计及其VR实现[D]. 成都:电子科技大学,2020. LIU Peng. Paradigm design and VR Implementation of rehabilitation-oriented upper extremity robot training system[D]. Chengdu:University of Electronic Science and Technology of China,2020.
[8] 张艳松. 提升儿童运动能力的外骨骼系统开发及试验研究[D]. 杭州:浙江大学,2013. ZHANG Yansong. Development of an exoskeleton system for children's exercise capacity promotion and its experimental study[D]. Hangzhou:Zhejiang University,2013.
[9] LÜ M,CHEN W H,DING X,et al. Development of an EMG-controlled knee exoskeleton to assist home rehabilitation in a game context[J]. Frontiers in Neurorobotics,2019,13:67.
[10] KARVOUNIARI A,MICHALOS G,DIMITROPOULOS N,et al. An approach for exoskeleton integration in manufacturing lines using virtual Reality techniques[J]. Procedia CIRP,2018,78:103-108.
[11] SECCO E L,TADESSE A M. A wearable exoskeleton for hand kinesthetic feedback in virtual reality[C]//International Conference on Wireless Mobile Communication and Healthcare. Springer,Cham,2019:186-200.
[12] 王文东,肖孟涵,孔德智,等. 基于人机耦合模型的上肢康复外骨骼闭环PD迭代控制方法[J]. 机械工程学报,2021,57(21):11-21. WANG Wendong,XIAO Menghan,KONG Dezhi,et al. Closed-loop PD iterative control method for upper limb rehabilitation exoskeleton based on human-robot coupling model[J]. Journal of Mechanical Engineering, 2021,57(21):11-21.
[13] MARINI F,TIMMONS M J,TALLITSCH R B,et al. Human anatomy[M]. San Francisco,CA:Pearson/Benjamin Cummings,2006.
[14] 秦仕昊. 下肢矢状面运动的穿戴式软传感测量系统研究[D]. 哈尔滨:哈尔滨工业大学,2019. QIN Shihao. Research on wearable soft sensing measurement system for sagittal motion of lower[D]. Harbin:Harbin Institute of Technology,2019.
[15] 杨金江. 助力型下肢外骨骼机器人多信号融合感知系统研究[D]. 杭州:浙江大学,2017. YANG Jinjiang. Research on multi-signal fusion perception system of power-assisted lower limb exoskeleton robot[D]. Hangzhou:Zhejiang University,2017.
[16] 刘建辉. 基于穿戴舒适性的下肢外骨骼机器人研究[D]. 上海:东华大学,2021. LIU Jianhui. Research on lower limb exoskeleton robot based on wearing comfort[D]. Shanghai:Donghua University,2021.
[17] 董海军. 中小学生功能性动作评价研究[J]. 西安体育学院学报,2017,34(2):226-236. DONG Haijun. On functional activities evaluation of primary and middle school students[J]. Journal of Xi'an Physical Education University,2017,34(2):226-236.
[18] 王存金,董林杰,李杰,等. 基于人行走能耗分析的踝关节外骨骼设计[J]. 机械工程学报,2021,57(19):79-92. WANG Cunjin,DONG Linjie,LI Jie,et al. Design of ankle exoskeleton based on analysis on energy cost of human walking[J]. Journal of Mechanical Engineering, 2021,57(19):79-92.
[19] Springer. Springer handbook of robotics[M]. Springer,2016.
[20] 王柏村,薛塬,延建林,等. 以人为本的智能制造:理念、技术与应用[J]. 中国工程科学,2020,22(4):139-146. WANG Baicun,XUE Yuan,YAN Jianlin,et al. Human-centered intelligent manufacturing:overview and perspectives[J]. Strategic Study of CAE,2020, 22(4):139-146.
[21] WANG B,LI X,FREIHEIT T,et al. Learning and intelligence in human-cyber-physical systems:framework and perspective[C]//2020 Second International Conference on Transdisciplinary AI (TransAI). IEEE,2020:142-145.
[22] 肖祥武,王丰,王晓辉,等. 面向工业互联网的智慧电厂仿生体系架构及信息物理系统[J]. 电工技术学报,2020,35(23):4898-4911. XIAO Xiangwu,WANG Feng,WANG Xiaohui,et al. Bionic structure and cyber-physical system for intelligent power plant oriented to the industrial internet[J]. Transactions of China Electrotechnical Society,2020,35(23):4898-4911.
[23] SUN Z,BAO G,LI X. Experimental research of force feedback dataglove based on pneumatic artificial muscle[J]. Chinese Journal of Mechanical Engineering,2009(4):489.
[24] SUZUKI S,AMEMIYA Y,SATO M. Enhancement of gross-motor action recognition for children by CNN with OpenPose[C]//IECON 2019-45th Annual Conference of the IEEE Industrial Electronics Society. IEEE,2019:5382-5387.
[25] 国家技术监督局. GB 10000-88中国成年人人体尺寸[S]. 北京:中国标准出版社,1989. General Administration of Quality Supervision,Inspection and Quarantine of the People's Republic of China. GB 10000-88 Chinese adult body size[S]. Beijing:Standards Press of China,1989.
[26] 刘建辉. 基于穿戴舒适性的下肢外骨骼机器人研究[D]. 上海:东华大学,2021. LIU Jianhui. Research on lower limb exoskeleton robot based on wearing comfort[D]. Shanghai:Donghua University,2021.
[27] 李剑锋,刘钧辉,张雷雨,等. 人机相容型肩关节康复外骨骼机构的运动学与灵活性分析[J]. 机械工程学报,2018,54(3):46-54. LI Jianfeng,LIU Junhui,ZHANG Leiyu,et al. Kinematics and dexterity analysis of the human-machine compatible exoskeleton mechanism for shoulder joint rehabilitation[J]. Journal of Mechanical Engineering,2018,54(3):46-54.
[28] CHEN H R,LIN Y S. An examination of digital game-based situated learning applied to Chinese language poetry education[J]. Technology,Pedagogy and Education,2016,25(2):171-186.
[29] 曲晨. 我国校园足球教练员专业知识结构特征的研究[J]. 中国学校体育(高等教育),2018,5(8):65-69. QU Chen. Research on the professional knowledge structure characteristics of football coaches on campus in China[J]. China School Physical Education,2018,5(8):65-69.