Summary of the Research on the Interaction of Tightly Coupled Physical Human-machine System

doi:10.3901/JME.2021.17.010

Abstract

Abstract: Tightly coupled physical man-machine systems are used to enhance and reconstruct human motion functions, and have been widely used in fields such as helping the elderly and the disabled, rehabilitation medicine, and national defense and military. The paper surveys on the interaction of tightly coupled physical man-machine systems. Through functional division, the system is divided into three aspects:functional replacement, functional rehabilitation and functional enhancement, and the research status of each type of system is introduced in detail. Then, the status quo of human-computer interaction interface is explained from two aspects of implanted sensing and external sensing. Then the perceptual intention recognition algorithm of tightly coupled human-computer interaction are analyzed. Finally, the future development is prospected and summarized.

Key words: tightly coupled physical human-machine system, functional replacement, functional rehabilitation, human-computer interaction interface, intention recognition

CLC Number:

TG156

FANG Bin, SUN Baixin, Cheng Guang, Dai Quanmin, Liu Weifeng, SUN Fuchun. Summary of the Research on the Interaction of Tightly Coupled Physical Human-machine System[J]. Journal of Mechanical Engineering, 2021, 57(17): 10-20.

References

[1] AMOR H,NEUMANN G,KAMTHE S,et al. Interaction primitives for human-robot cooperation tasks[C]//2014 IEEE International Conference on Robotics and Automation (ICRA),Hong Kong,China,2014:2831-2837.
[2] 雷永林,朱一凡,谭跃进,等. 模型驱动的复杂人机系统过程建模仿真方法[J]. 系统工程与电子技术,2016,38(1):223-231. LEI Yonglin,ZHU Yifan,TAN Yuejin,et al. Model driven process modeling and simulation of complex man-machine systems[J]. Systems Engineering and Electronics,2016,38(1):223-231.
[3] 王朝立. 人机共融安全技术发展趋势展望[J]. 自动化仪表,2020,41(3):1-5,10. WANG Chaoli. Prospect for the development trend of manmachine integration safety technology[J]. Automation Instrumentation,2020,41(3):1-5,10.
[4] HUU-TOAN T,HONG C,LIN Xichuan,et al. The relationship between physical human-exoskeleton interaction and dynamic factors:Using a learning approach for control applications[J]. Computers,Networks & Communications,2014,57(12):1-13.
[5] 程洪,黄瑞,邱静,等. 人机智能技术及系统研究进展综述[J]. 智能系统学报,2020,15(2):386-398. CHENG Hong,HUANG Rui,QIU Jing,et al. Overview of human-machine intelligence technology and system research progress[J]. Journal of Intelligent Systems,2020,15(2):386-398.
[6] BILLARD A,CALINON S,DILLMANN R,et al. Survey:Robot programming by demonstration in handbook of robotics[M]. MIT Press,2008.
[7] LASCHI C,MAZZOLAI B,CIANCHETTI M. Soft robotics:Technologies and systems pushing the boundaries of robot abilities[J]. Science Robotics,2016,1(1):eaah3690.
[8] BALASUBRAMANIAN S,WARD J,SUGAR T,et al. Characterization of the dynamic properties of pneumatic muscle actuators[C]//ICORR 2007. IEEE 10th International Conference on Rehabilitation Robotics,2007:764-770.
[9] RUS D,TOLLEY M. Design fabrication and control of soft robots[J]. Nature,2015,521(7553):467-475.
[10] LEE C,KIM M,KIM Y J,et al. Soft robot review[J]. International Journal of Control Automation & Systems,2017,15(1):3-15.
[11] 梁旭,王卫群,侯增广,等. 康复机器人的人机交互控制方法[J]. 中国科学:信息科学,2018,48(1):24-46. LIANG Xu,WANG Weiqun,HOU Zengguang et al.Human-machine interactive control method of rehabilitation robot[J]. Science in China:Information Science,2018,48(1):24-46.
[12] 黄骐云. 多模态人机接口及其残疾人辅助应用研究[D].广州:华南理工大学,2019. HUANG Qiyun. Research on multi-modal human-machine interface and its assistance application for the disabled[D]. Guangzhou:South China University of Technology,2019.
[13] 易金花,喻洪流,张颖,等. 中央驱动式上肢康复机器人运动学建模与分析[J]. 生物医学工程学杂志,2015,32(6):1196-1201. YI Jinhua,YU Hongliu,ZHANG Ying,et al. Kinematics modeling and analysis of central driven upper limb rehabilitation robot[J]. Journal of Biomedical Engineering,2015,32(6):1196-1201.
[14] 张泰略,李晋川,邱越,等. 基于脑-肌电信号的新型假肢控制模式的实验研究[J]. 医疗卫生装备,2020,41(7):36-40. ZHANG Tailue,LI Jinchuan,QIU Yue,et al. Experimental study of a new prosthetic control mode based on brain-electromyography signals[J]. Medical and Medical Equipment,2020,41(7):36-40.
[15] CHALODHORN R,GRIMES D,GROCHOW K. Learning to walk through imitation[C]//Proceedings of the 20th International Joint Conference on Artifical Intelligence,2007:2084-2090.
[16] GRIMES D,CHALODHORN R,RAO R. Dynamic imitation in a humanoid robot through nonparametric probabilistic inference[C]//Proceedings of Robotics:Science and Systems,2006:1.
[17] MACIEJASZ P,ESCHWEILER J,Gerlach-Hahn K,et al. A survey on robotic devices for upper limb rehabilitation[J]. Journal of Neuroengineering and Rehabilitation,2014,11:10.
[18] 闫继宏,石培沛,张新彬,等. 软体机械臂仿生机理、驱动及建模控制研究发展综述[J]. 机械工程学报,2018,54(15):1-14. YAN Jihong,SHI Peipei,ZHANG Xinbin,et al. Review of biomimetic mechanism,actuation,modeling and control in soft manipulators[J]. Journal of Mechanical Engineering,2018,54(15):1-14.
[19] 黄海. 基于人机交互力的下肢外骨骼控制技术研究[D]. 哈尔滨:哈尔滨工业大学,2019. HUANG Hai. Research on lower limb exoskeleton control technology based on human-computer interaction[D]. Harbin:Harbin Institute of Technology,2019.
[20] 潘泉,于昕,程咏梅. 信息融合理论的基本方法与进展[J]. 自动化学报,2003(4):599-615. PAN Quan,YU Xin,CHENG Yongmei. Basic methods and progress of information fusion theory[J]. Acta Automatica Sinica,2003(4):599-615.
[21] HUO W,MOHAMMED S,MORENO J C,et al. Lower limb wearable robots for assistance and rehabilitation:A state of the art[J]. IEEE Systems Journal,2014,1:1-14,2014.
[22] 张晓玉. 我国智能辅助器具科技创新的现状与发展[J].中国康复理论与实践,2013,19(5):401-403. ZHANG Xiaoyu. The status quo and development of technological innovation of intelligent assistive devices in my country[J]. Chinese Rehabilitation Theory and Practice,2013,19(5):401-403.
[23] AMIGO L E,FERNANDEZ Q,GIRALT X. Study of patient-orthosis interaction forces in rehabilitation therapies[C]//2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.
[24] BLAYA J A,HERR H. Adaptive control of a variable-impedance ankle foot orthosis to assist drop-foot gait[J]. IEEE Transactions Neural System and Rehabilitation Engineering,2014,12(1):24-31.
[25] BARTON G J,VANRENTERGHEM J,LEES A. A method for manipulating a movable platform's axes of rotation:A novel use of the CAREN system[J]. Gait and Posture,2006,24:510-514.
[26] JAMWAL P K,XIE S Q,HUSSAIN S,et al. An adaptive wearable parallel robot for the treatment of ankle injuries,[C]//IEEE/ASME Transactions on Mechatronics,2014,19:64-75.
[27] BELLMANN M,SCHMALZ T,BLUMENTRITT S. Comparative biomechanical analysis of current microprocessor-controlled prosthetic knee joints[J]. Arch Phys Med Rehabil,2010,91:644-652.
[28] ROSSI D,VITIELLO N,LENZI T. Sensing pressure distribution on a lower-limb exoskeleton physical human-machine interface[J]. Sensors,2011,11:207-227.
[29] COLLINS S H,WIGGIN M B,SAWICKI G S,Reducing the energy cost of human walking using an unpowered exoskeleton[J]. Nature,2015,522:212-215.
[30] DARVISH K,BRUNO B,SIMETTI E,et al. An adaptive human-robot cooperation framework for assemblylike tasks[C]//Proceedings of International Conference of the Italian Association for Artificial Intelligence. Genova,Italy,2016:45-50.
[31] OSBOR N,LUKE E. Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain[J]. Science Robotics,2018,3:eaat3818.
[32] BEKIROGLU Y,LAAKSONEN J,JORGENSEN J A,et al. Assessing grasp stability based on learning and haptic data[J]. IEEE Transactions on Robotics,2011,27(3):616-629.
[33] HUANG H,LI T,BRUSCHINI C,et al. EMG pattern recognition using decomposition techniques for constructing multi class classifiers[C]//IEEE International Conference on Biomedical Robotics and Biomechatronics,Singapore,2016:1296-1301.
[34] KADROLKAR A,IV F. Intent recognition of torso motion using wavelet transform feature extraction and linear discriminant analysis ensemble classification[J]. Biomedical Signal Processing & Control,2017,38(1):250-264.
[35] YAGJOUB D,SIAMAK N,MOHAMMAD R,et al. A prosthetic knee joint inspired from musculoskeletal system David Moser[J]. Biocybernetics and Biomedical Engineering,2013,33:118-124.
[36] 王启宁,郑恩昊,许东方. 基于非接触式电容传感的人体运动意图识别[J]. 机械工程学报,2019,55(11):19-27. WANG Qining,ZHENG Endong,XU Dongfang. Human motion intention recognition based on non-contact capacitive sensing[J]. Journal of Mechanical Engineering,2019,55(11):19-27.
[37] 赵晓东,刘作军,陈玲玲,等. 下肢假肢穿戴者跑动步态识别方法[J]. 浙江大学学报,2018,52(10):1980-1988. ZHAO Xiaodong,LIU Zuojun,CHEN Lingling,et al. Recognition method for running gait of lower limb prosthetic wearers[J]. Journal of Zhejiang University,2018,52(10):1980-1988.
[38] SU B Y,WANG J,LIU S Q,et al. A CNN-based method for intent recognition using inertial measurement units and intelligent lower limb prosthesis[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering,2019,27(5):1032-1042.
[39] 马跃. 下肢外骨骼机器人人机协同控制策略研究[D].中国科学院大学(中国科学院深圳先进技术研究院),2020. MA Yue. Research on human-machine cooperative control strategy of lower limb exoskeleton robot[D]. University of Chinese Academy of Sciences (Shenzhen Institute of Advanced Technology,Chinese Academy of Sciences),2020.
[40] HANG T. Symbiotic man-machine interactions in wearable exoskeletons to enhance mobility for paraplegics[J]. European Union,Seventh Framework Programme,2013,10:661626.
[41] SHOKUR S,GALLO S,MOIOLI R C,et al. Assimilation of virtual legs and perception of floor texture by complete paraplegic patients receiving artificial tactile feedback[J]. Sci. Rep.,2016,6:3229.
[42] ROSE C G,O'MALLEY M K. Hybrid rigid-soft hand exoskeleton to assist functional dexterity[J]. IEEE Robotics and Automation Letters,2019,4(1):73-80.
[43] DALY J,WOLPAW J. Brain-computer interfaces in neurologicalrehabilitation[J]. Lancet Neurology,2008,7(11):1032-1043.
[44] BUNDY D,SOUDERS L,BARANYAI K,et al. Contralesional brain-computerinterface control of a powered exoskeleton for motor recovery in chronic strokesurvivors[J]. Stroke,2017,48(7):1908-1915.
[45] VÁRKUTI B,GUAN C,PAN Y,et al. Resting state changes in functionalconnectivity correlate with movement recovery for BCI and robot-assistedupper-extremity training after stroke[J]. Neurorehabilitation and NeuralRepair,2013,27(1):53-62.
[46] LI L,JING W,XU G,et al. The study of object-oriented motor imagerybased on EEG suppression[J]. Plos One,2015,10(12):e0144256.
[47] WHYTE J,DIJKERS MP,HART T,et al. The importance of voluntarybehavior in rehabilitation treatment and outcomes[J]. Archives of Physicalmedicine and Rehabilitation,2019,100(1):156-163.
[48] ZEILIG G,WEINGARDEN H,ZWECKER M. Safety and tolerance of the ReWalkTM exoskeleton suit for ambulation by people with complete spinal cord injury:a pilot study[J]. Journal of Spinal Cord Medicine,2012,35(2):96-101.
[49] PAN Y T,LAMB Z,MACIEVICH J. A vibrotactile feedback device for balance rehabilitation in the EksoGTTM robotic exoskeleton[C]//IEEE International Conference on Biomedical Robotics and Biomechatronics. Enschede,Netherlands,2018:569-576.
[50] CHEN S,LI J,SHUAI Mei. First multicenter clinical trial of China's domestically designed powered exoskeleton assisted walking in patients with paraplegia[J]. Annals of Physical and Rehabilitation Medicine,2018,61:e495.
[51] HUANG Rui,CHENG Hong,QIU Jing,et al. Learning physical human-robot interaction with coupled cooperative primitives for a lower exoskeleton[J]. IEEE Transactions on Automation Science and Engineering,2019(1):1-9.
[52] CHEN Qiming,CHENG Hong,YUE Chunfeng,et al. Dynamic balance gait for walking assistance exoskeleton[J]. Applied Bionics and Biomechanics,2018(1):doi.org/10.1155/2018/7847014.
[53] MAKIN T R,De VIGNEMONT F,Faisal A A. Neurocognitive barriers to the embodiment of technology[J]. Nature Biomedical Engineering,2017,1(1):14.
[54] SARAH S. Youbionic goes beyond augmented reality to augmented human with new 3D printed double hand device[J]. 3D Printing Robotics,2017,2:48-57.
[55] PENALOZA C I,NISHIO S. BMI control of a third arm for multitasking[J]. Science Robotics,2018,3(20):eaat1228.
[56] LESSARD S,PANSODTEE P,ROBBINS A,et al. A soft exosuit for flexible upper-extremity rehabilitation[J]. IEEE Trans. Neur. Syst. Reh.,2018,26(8):1604-1617.
[57] JULIAN W,HOWIE C. Task-specifific manipulator design and trajectory synthesis[J]. IEEE Robotics and AutomationLetters,2018,1:2890206.
[58] CATHERINE V,JEFF D,LOUIS-PHILIPPE L,et al. Multifunctional 3-DOF wearable supernumerary robotic arm based on magnetorheological clutches[J]. ICRA 2020(1):3211-3223.
[59] MOHAMMED A,THOMAS H,MOHAMED K,et al. Orochi:Investigating requirements and expectations for multipurpose daily used supernumerary robotic limbs[C]//AH2019:Augmented Human International Conference,2019Reims,France. 2019:231-242.
[60] GEORGE P,MINAS V,LIAROKAPIS G. Open-source,anthropomorphic,underactuated robot hands with a selectively lockable differential mechanism:Towards affordable prostheses[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),Hamburg,Germany,2015:661-672.
[61] SALMINGER S. Long-term implantof intramuscular sensors and nerve transfers for wireless control of roboticarms in above-elbow amputees[J]. Science Robotics,2019(4):eaaw6306.
[62] GEORGE J A,KLUGER D T,DAVIS T S,et al. Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand[J]. Science Robotics,2019,4(32):eaax2352.
[63] PHILIP P. A regenerative peripheral nerve interface allows real-time control of an artificial hand inupper limb amputees[J]. Science Translational Medicine,2020,12:533.
[64] WANG T,FENG Z,WANG C,et al. Real-time investigation of interactions between nanoparticles and cell membrane model[J]. Science Robotics,2018,164:70-77.
[65] MAURICE P,CAMERNIK J,GORJAN D,et al. Objective and subjective effects of a passive exoskeleton on overhead work[J]. Science Robotics,2020,28(1):152-164.
[66] CHRIS I,CHAR P,BECHL I. Task-specific grasp selection for underactuated hands[C]//IEEE International Conference on Robotics and Automation,2014,32(2):66-75.
[67] Merad M. Assessment of an automatic prosthetic elbow control strategy using residual limb motion for transhumeral amputated individuals with socket or osseointegrated prostheses[J]. IEEE Transactions on Medical Robotics and Bionics,2020,2(1):38-49.
[68] HUA Q,SUN J,LIU H,et al. Skin-inspired highly stretchable and conformable matrix networks for multifunctional sensing[J]. Nat Commun,2018,9:244.
[69] SUN B,DAI Q,CHENG G. Electronic skin design methods applied to capsule robot[C]//2019 IEEE International Conference on Robotics and Biomimetics (ROBIO),Dali,China,2019:252-256.
[70] 侯增广,赵新刚,程龙,等. 康复机器人与智能辅助系统的研究进展[J]. 自动化学报,2016,42(12):1765-1779. HOU Zengguang,ZHAO Xingang,CHENG Long,et al. Research progress of rehabilitation robots and intelligent auxiliary systems[J]. Acta Automatica Sinica,2016,42(12):1765-1779.
[71] TANG Z C,YU H N,CANG S. Impact of load variation on joint angle estimation from surface EMG signals[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering,2016,24(12):1342-1350.
[72] ALEXANDER O,NEBOJS A M,ANDERS B. Exploiting the intertemporal structure of the upper-limb sEMG:Comparisons between an LSTM network and cross-sectional myoelectric pattern recognition methods[C]//201941st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC),2019:6611-6615.
[73] ROMERO-HERNANDEZ P,ASIAIN J D L,MANUEL GRAÑA. Deep learning prediction of gait based on inertial measurements[M]. Springer,2019.
[74] FERRARI A,BERGAMINI L,GUERZONI G. Gait-based diplegia classification using LSMT networks[J]. Journal of Healthcare Engineering,2019,20:213-221.
[75] DHAWAN A S,MUKHERJEE B,PATWARDHAN S,et al. Proprioceptive sonomyographic control:A novel method for intuitive and proportional control of multiple degrees-of-freedom for individuals with upper extremity limb loss[J]. Sci. Rep.,2019(1):7.
[76] HWANGBO J,LEE J,DOSOVITSKIY A,et al. Learning agile and dynamic motor skills for legged robots[J]. Science Robotics,2019,4(26):1.
[77] YOUNG A J,SIMON A M,FEY N P,et al. Intent recognition in a powered lower limb prosthesis using time history information[J]. Annals of Biomedical Engineering,2014,42(3):631-641.
[78] FERRER G,SANFELIU A. Bayesian human motion intentionality prediction in urban environments[J]. Pattern Recognition Letters,2014,44:134-140.
[79] OLGA V D,REINDER H A,BONGERS R M. The i-LIMB hand and the DMC plus hand compared:A case report[J]. Prosthetics and Orthotics International,2010,34(2):216-220.
[80] HUA Y,FAN J,LIU G,et al. A novel weight-bearing lower limb exoskeleton based on motion intention prediction and locomotion state identification[J]. IEEE Access,2019(7):37620-37638.