Method of Dexterous Finger Unit Full-pose Modeling Based on Magnetic-inertial Sensor Fusion

doi:10.3901/JME.2022.18.133

Abstract

Abstract: As an important natural medium for human-environment-interaction, human hand takes advantages of high degrees of flexibility and complexity, and efficient and accurate hand motion and gesture recognition is of great significance for realizing hand-based human-machine-environment integration. A method for constructing a personalized dexterous finger unit full-pose information based on the magnetic-inertial sensor information fusion technology is proposed, in order to provide technical support for accurate human-machine-environment perception and inclusiveness. Combined with the biological structural constraints of the phalanx of the hand, a finger unit kinematics model was established. Through the structured magnetic field marking technology, a magnetic-inertial information fusion finger unit full-pose deriving system is developed, and the construction of a personal parameterized finger unit model is completed. The continuous path comparison test and single-point positioning accuracy verification test of different volunteer finger units proved that the proposed method for personalized finger unit full-pose information derivation based on magnetic-inertial sensor information fusion technology can achieve a positioning accuracy between [1.43, 2.81] mm.

Key words: dexterous hand, finger unit, magnetic-inertial sensing, information fusion, full-pose inversion

CLC Number:

TP241

SHEN Huimin, GE Ruikang, GU Xiaowei, CAI Xiaotong, GAN Yi, YANG Geng. Method of Dexterous Finger Unit Full-pose Modeling Based on Magnetic-inertial Sensor Fusion[J]. Journal of Mechanical Engineering, 2022, 58(18): 133-140.

References

[1] 王柏村，易兵，刘振宇，等. HCPS视角下智能制造的发展与研究[J/OL].计算机集成制造系统:1-19 [2021-10-26]. http://kns.cnki.net/kcms/detail/11.5946.tp.20201007.1905.002.html. WANG Baicun，YI Bing，LIU Zhenyu，et al. The development and research of intelligent manufacturing from the perspective of HCPS[J/OL]. Computer Integrated Manufacturing System:1-19 [2021-10-26]. http://kns.cnki.net/kcms/detail/11.5946.tp.20201007.1905.002.html.
[2] 陶飞，刘蔚然，刘检华，等. 数字孪生及其应用探索[J]. 计算机集成制造系统，2018，24(1):4-21.TAO Fei，LIU Weiran，LIU Jianhua，et al. Digital twin and its application exploration[J]. Computer Integrated Manufacturing System，2018，24(1):4-21.
[3] 王柏村，彭晨，易兵，等. 智能时代的人机共融体:技术驱动、以人为本——《The Humachine:Humankind，Machines，and the Future of Enterprise》导读[J]. 中国机械工程，2021，32(19):2390-2393.WANG Baicun，PENG Chen，YI Bing，et al. The human-machine integration in the intelligent era:Technology driven，people-oriented——guide to "The Humachine:Humankind，Machines，and the Future of Enterprise"[J]. China Machinery Engineering，2021，32(19):2390-2393.
[4] 蔡世波，陶志成，万伟伟，等. 机器人多指灵巧手的研究现状、趋势与挑战[J]. 机械工程学报，2021，57(15):1-14.CAI Shibo，TAO Zhicheng，WAN Weiwei，et al. Research status，trends and challenges of robotic multi-finger dexterous hands[J]. Journal of Mechanical Engineerin，2021，57(15):1-14.
[5] GUO J，LIU C，POIGNET P. A scaled bilateral teleoperation system for robotic-assisted surgery with time delay[J]. J. Intell. Robot. Syst.，2019，95:165-192.
[6] NAGATANI K，KIRIBAYASHI S，OKADA Y，et al. Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots[J]. J. Field Robot，2013，30:44-63.
[7] BOLOPION A，RÉGNIER S. A review of haptic feedback teleoperation systems for micromanipulation and microassembly[J]. IEEE T rans. Autom. Sci. Eng.， 2013，10:496-502.
[8] COLASANTO L，SUAREZ R，ROSELL J. Hybrid mapping for the assistance of teleoperated grasping tasks[J]. IEEE Transactions on Systems，Man，and Cybernetics:Systems，2013，43(2):390-401.
[9] CHEN C，YU H，RADWIN R G. A motion tracking system for hand activity assessment[C]//IEEE China Summit & International Conference on Signal & Information Processing. IEEE，2014:320-324.
[10] KORTIER H G，ANTONSSON J，SCHEPERS H M，et al. Hand pose estimation by fusion of inertial and magnetic sensing aided by a permanent magnet[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering，2015，23(5):796-806.
[11] PACCHIEROTTI C，SINCLAIR S，SOLAZZI M，et al. Wearable haptic systems for the fingertip and the hand:Taxonomy，review，and perspectives[J]. IEEE Trans. Haptics，2017，10(4):580-600.
[12] 唐纳德·A·诺伊曼. 骨骼肌肉功能解剖学[M]. 2版. 北京:人民军医出版社，2014.DONALD A NEUMANN. Functional anatomy of skeletal muscles[M]. 2nd ed. Beijing:People’s Military Medical Publishing House，2014.
[13] CHEN J，ZHU H，ZENG Z，et al. Motion tracking of both hands with occasional mutual occlusion using RGB-D camera and IMU[C/CD]//2017 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE，2017.
[14] WERNER W，OLAF K，HUBERT M，et al. Magnetic marker monitoring:An application of biomagnetic measurement instrumentation and principles for the determination of the gastrointestinal behavior of magnetically marked solid dosage forms[J]. Advanced Drug Delivery Reviews，2005，57(8):1210-1222.
[15] FEIX T，ROMERO J，SCHMIEDMAYER H B，et al. The GRASP taxonomy of human grasp types[J]. IEEE Transactions on Human-Machine Systems，2016，46(1):66-77.
[16] 刘明进. 人手抓握运动规律及其在机器人仿人手设计中的应用[D]. 武汉:华中科技大学，2017.LIU Mingjin. The law of grasping motion of human hand and its application in the design of robot imitating human hand[D]. Wuhan:Huazhong University of Science and Technology，2017.
[17] FANG B，SUN F，LIU H，et al. A novel data glove using inertial and magnetic sensors for motion capture and robotic arm-hand teleoperation[J]. Industrial Robot，2017，44(2):155-165.
[18] YANG Z，YAN S，BEIJNUM B J，et al. Hand-finger pose estimation using inertial sensors，magnetic sensors and a magnet[J]. IEEE Sensors Journal，2021，21(16):18115-18122.
[19] SALCHOW-HÖMMEN C，CALLIES L，LAIDIG D，et al. A tangible solution for hand motion tracking in clinical applications[J]. Sensors，2019，19(1):208-235.