Research on Operation Skill Transfer and Control Oriented to Complex Force Interaction Tasks

doi:10.3901/JME.2022.18.116

Abstract

Abstract: The problem that has been explored for years in robotics is how to make robots have the same strong perception capabilities as humans and perform complex operations, especially with force interaction. Solving the problem can help the robot to truly transfer from “device” to “assistant”. The transfer and control of operational skills for complex force interaction tasks is one of the frontiers of the human-robot skills transfer research field. The core of this research lies in the learning of multi-modal skill data, demonstrated in the force interaction operations process by skilled operators. And through reasonable design skill models, combined with advanced control theory and robot perception capabilities, the robot can autonomously perform complex force interaction tasks, so that robots can truly assist or even replace humans in performing common complex tasks in life. This article summarizes three important issues in this field: ① Multimodal information fusion demonstration method; ② Skill learning for force interaction tasks; ③ Skill control based on robot compliance control and intelligent skill switching based on robot perception. At last, the research status in this field will be discussed and analyzed.

Key words: skill transfer, variable impedance, multimodal information fusion, force interaction

CLC Number:

TG242

ZHAO Jie, WU Rui, ZHANG He, ZHU Yanhe, ZANG Xizhe. Research on Operation Skill Transfer and Control Oriented to Complex Force Interaction Tasks[J]. Journal of Mechanical Engineering, 2022, 58(18): 116-132.

References

[1] 曾超，杨辰光，李强，等. 人-机器人技能传递研究进展[J]. 自动化学报，2019，45(10):1813-1828. Zeng Chao，Yang Chenguang，Li Qiang，et al. Research progress on human-robot skill transfer[J]. Acta Automatica Sinica，2019，45(10):1813-1828.
[2] Billard A，Kragic D. Trends and challenges in robot manipulation[J]. Science，2019，364(6446):eaat8414.
[3] Rozo L，Calinon S，Caldwell D，et al. Learning collaborative impedance-based robot behaviors[J]. AAAI Conference on Artificial Intelligence，2013:1422-1428.
[4] Rozo L，Calinon S，Caldwell D G，et al. Learning physical collaborative robot behaviors from human demonstrations[J]. IEEE Transactions on Robotics，2016，32(3):513-527.
[5] Kronander K，Billard A. Learning compliant manipulation through kinesthetic and tactile human-robot interaction[C]//IEEE Transactions on Haptics. 2014，7(3):367-380.
[6] Khansari-Zadeh S M，Khatib O. Learning potential functions from human demonstrations with encapsulated dynamic and compliant behaviors[J]. Autonomous Robots，Springer US，2017，41(1):45-69.
[7] Peternel L，Petrič T，Oztop E，et al. Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach[J]. Autonomous Robots，2014，36(1-2):123-136.
[8] Wu R，Zhang H，Peng T，et al. Human-robot interaction and demonstration learning mode based on electromyogram signal and variable impedance control[J]. Mathematical Problems in Engineering，Hindawi，2018，2018.
[9] Sheridan T B. Telerobotics[J]. Automatica，1989，25(4):487-507.
[10] Calinon S，Billard A. Incremental learning of gestures by imitation in a humanoid robot[C]//Proceeding of the ACM/IEEE International Conference on Human-robot Interaction - HRI ’07. New York，New York，USA:ACM Press，2007:255.
[11] Kitagawa M，Okamura A M，Bethea B T，et al. Analysis of suture manipulation forces for teleoperation with force feedback[J]. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)，2002，2488:155-162.
[12] Rank M，Hirche S. Cutaneous haptic feedback in robotic teleoperation(Springer series on touch and haptic systems)[J]. Multisensory Softness，2014(8):147-165.
[13] Wu R，Zhang H，Peng T，et al. Variable impedance interaction and demonstration interface design based on measurement of arm muscle co-activation for demonstration learning[J]. Biomedical Signal Processing and Control，2019，51:8-18.
[14] Zahedi E，Khosravian F，Wang W，et al. Towards skill transfer via learning-based guidance in human-robot interaction:An application to orthopaedic surgical drilling skill[J]. Journal of Intelligent and Robotic Systems:Theory and Applications，Journal of Intelligent & Robotic Systems，2020，98(3-4):667-678.
[15] Zahedi E，Dargahi J，Kia M，et al. Gesture-based adaptive haptic guidance:A comparison of discriminative and generative modeling approaches[J]. IEEE Robotics and Automation Letters，2017，2(2):1015-1022.
[16] Coles T R，Meglan D，John N W. The role of haptics in medical training simulators:A survey of the state of the art[J]. IEEE Transactions on Haptics，2011，4(1):51-66.
[17] Wu R，Billard A. Learning from demonstration and interactive control of variable-impedance to cut soft tissues[J/OL]. [2000-01-08]. https://www.epfl.ch/labs/lasa/sahr/publications/
[18] Bernardino A，Henriques M，Hendrich N，et al. Precision grasp synergies for dexterous robotic hands[C]//2013 IEEE International Conference on Robotics and Biomimetics，ROBIO 2013. 2013:62-67.
[19] Ureche L P，Billard A. Constraints extraction from asymmetrical bimanual tasks and their use in coordinated behavior[J]. Robotics and Autonomous Systems，2018，103:222-235.
[20] Yao K，Billard A. An inverse optimization approach to understand human acquisition of kinematic coordination in bimanual fine manipulation tasks[J]. Biological Cybernetics，Springer Berlin Heidelberg，2020，114(1):63-82.
[21] Yao K，Sternad D，Billard A，et al. Effect of task conditions on human hand pose selection strategies in bimanual fine manipulation rask[C/CD]//Neural Control of Movement 2022 Annual Meeting. Society for Neural Control of Movement,2022.
[22] Howard I S，Ingram J N，Körding K P，et al. Statistics of natural movements are reflected in motor errors[J]. Journal of Neurophysiology，2009，102(3):1902-1910.
[23] MARTIN-MARTIN R, LEE M A, GARDNER R, et al. Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks[C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2019:1010-1017.
[24] YU X, LI Y, ZHANG S, et al. Estimation of human impedance and motion intention for constrained human-robot interaction[J]. Neurocomputing, 2020, 390: 268-279.
[25] Silverio J，Huang Y，Abu-Dakka F J，et al. Uncertainty-aware imitation learning using kernelized movement primitives[C]//IEEE International Conference on Intelligent Robots and Systems，2019:90-97.
[26] Burdet E，Osu R，Franklin D W，et al. The central nervous system stabilizes unstable dynamics by learning optimal impedance[J]. Nature，2001，414(6862):446-449.
[27] Trumbower R D，Krutky M A，Yang B S，et al. Use of self-selected postures to regulate multi-joint stiffness during unconstrained tasks[J]. PLoS ONE，2009，4(5):e5411.
[28] Shin E C，Ryu J H，Yang G H. Estimation of human arm impedance in accordance with the master device types and gripping posture[C]//IEEE/ASME International Conference on Advanced Intelligent Mechatronics，AIM，2015，2015-Augus:1744-1748.
[29] Franklin D W，Liaw G，Milner T E，et al. Endpoint stiffness of the arm is directionally tuned to instability in the environment[J]. Journal of Neuroscience，2007，27(29):7705-7716.
[30] Erden M S，Billard A. Hand impedance measurements during interactive manual welding with a robot[J]. IEEE Transactions on Robotics，2015，31(1):168-179.
[31] Flash T，Mussa-Ivaldi F. Human arm stiffness characteristics during the maintenance of posture[J]. Experimental Brain Research，1990，82(2):315-326.
[32] Gomi H，Kawato M. Human arm stiffness and equilibrium-point trajectory during multi-joint movement[J]. Biological Cybernetics，1997，76(3):163-171.
[33] Tee K P，Burdet E，Chew C M，et al. A model of force and impedance in human arm movements[J]. Biological Cybernetics，2004，90(5):368-375.
[34] Lloyd D G，Besier T F. An EMG-driven musculoskeletal model to estimate muscle forces and knee joint moments in vivo[J]. Journal of Biomechanics，2003，36(6):765-776.
[35] YAO K，STERNAD D，BILARD A，et al. A hybrid BMI-based exoskeleton for paresis:EMG control for assisting arm movements[J]. Journal of Neural Engineering，2017，14(1):1741-2552.
[36] Ajoudani A，Fang C，Tsagarakis N，et al. Reduced-complexity representation of the human arm active endpoint stiffness for supervisory control of remote manipulation[J]. The International Journal of Robotics Research，2017:027836491774403.
[37] Ajoudani A，Tsagarakis N，Bicchi A. Teleimpedance:teleoperation with impedance regulation using a body-machine interface[J]. ijrr，2012，110:19-31.
[38] Ajoudani A，Fang C，Tsagarakis N G，et al. A reduced-complexity description of arm endpoint stiffness with applications to teleimpedance control[C]//IEEE International Conference on Intelligent Robots and Systems，2015，2015-Decem:1017-1023.
[39] Wu R，Zhang H，Zhao J. Robot variable impedance skill transfer and learning framework based on a simplified human arm impedance model[J]. IEEE Access，2020，8:225627-225638.
[40] Billard A，Calinon S. Handbook of robotics chapter 59 :Robot programming by demonstration[J]. Robotics，2007，48(21):1371-1394.
[41] Khansari-Zadeh S M，Billard A. Learning stable nonlinear dynamical systems with Gaussian mixture models[J]. IEEE Transactions on Robotics，2011，27(5):943-957.
[42] Ijspeert A J，Nakanishi J，Schaal S. Movement imitation with nonlinear dynamical systems in humanoid robots[C]//Proceedings - IEEE International Conference on Robotics and Automation. 2002，2:1398-1403.
[43] Pastor P，Hoffmann H，Asfour T，et al. Learning and generalization of motor skills by learning from demonstration[C]//Proceedings - IEEE International Conference on Robotics and Automation. 2009:763-768.
[44] Ude A，Gams A，Asfour T，et al. Task-specific generalization of discrete and periodic dynamic movement primitives[J]. IEEE Transactions on Robotics，2010，26(5):800-815.
[45] Righetti L，Buchli J，Ijspeert A J. Dynamic Hebbian learning in adaptive frequency oscillators[J]. Physica D:Nonlinear Phenomena，2006，216(2):269-281.
[46] Degallier S，Righetti L，Gay S，et al. Toward simple control for complex，autonomous robotic applications:Combining discrete and rhythmic motor primitives[J]. Autonomous Robots，2011，31(2-3):155-181.
[47] Calinon S，Sauser E L，Billard A G，et al. Evaluation of a probabilistic approach to learn and reproduce gestures by imitation[C]//2010 IEEE International Conference on Robotics and Automation. IEEE，2010:2671-2676.
[48] Kober J，MUlling K，KrOmer O，et al. Movement templates for learning of hitting and batting[C]//2010 IEEE International Conference on Robotics and Automation. IEEE，2010:853-858.
[49] Ijspeert A J. Central pattern generators for locomotion control in animals and robots:A review[J]. Neural Networks，2008，21(4):642-653.
[50] Grillner S. Neurobiological bases of rhythmic motor acts in vertebrates[J]. Science，1985，228(4696):143-149.
[51] Bullock D，Bongers R M，Lankhorst M，et al. A vector-integration-to-endpoint model for performance of viapoint movements[J]. Neural Networks，1999，12(1):1-29.
[52] Shon A P，Grochow K，Rao R P N. Robotic imitation for human motion capture using gaussian processes[C]//5th IEEE-RAS International Conference on Humanoid Robots，2005. IEEE，2005:129-134.
[53] Calinon S，Guenter F，Billard A. On learning，representing，and generalizing a task in a humanoid robot[J]. IEEE Transactions on Systems，Man，and Cybernetics，Part B:Cybernetics，2007，37(2):286-298.
[54] Billard A G，Calinon S，Guenter F. Discriminative and adaptive imitation in uni-manual and bi-manual tasks[J]. Robotics and Autonomous Systems，2006，54(5):370-384.
[55] Paik J K，Katsaggelos A K. Image restoration using a modified Hopfield network[J]. IEEE Transactions on Image Processing，1992，1(1):49-63.
[56] Amin M H，Andriyash E，Rolfe J，et al. Quantum Boltzmann machine[J]. Physical Review X，American Physical Society，2018，8(2):21050.
[57] Atiya A F，Parlos A G. New results on recurrent network training:unifying the algorithms and accelerating convergence[J]. IEEE Transactions on Neural Networks，2000，11(3):697-709.
[58] Yang C，Zeng C，Fang C，et al. A DMPs-based framework for robot learning and generalization of humanlike variable impedance skills[J]. IEEE/ASME Transactions on Mechatronics，IEEE，2018，23(3):1193-1203.
[59] Shaal S，Kotosaka S，Sternad D. Nonlinear dynamical systems as movement primitives[C]// IEEE Intemational Conference on Humanoid Robotics，2001.
[60] Ijspeert A J，Nakanishi J，Shibata T，et al. Nonlinear dynamical systems for imitation with humanoid robots[C]//Proceedings of the IEEE International Conference on Humanoid Robots，2001:219-226.
[61] Nakanishi J，Morimoto J，Endo G，et al. Learning from demonstration and adaptation of biped locomotion[J]. Robotics and Autonomous Systems，2004，47(2-3):79-91.
[62] Kulvicius T，Ning K，Tamosiunaite M，et al. Joining movement sequences:Modified dynamic movement primitives for robotics applications exemplified on handwriting[J]. IEEE Transactions on Robotics，2012，28(1):145-157.
[63] Khansari-Zadeh S M，Billard A. A dynamical system approach to realtime obstacle avoidance[J]. Autonomous Robots，2012，32(4):433-454.
[64] Khansari-Zadeh S M，Billard A. Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions[J]. Robotics and Autonomous Systems，Elsevier，2014，62(6):752-765.
[65] Shavit Y，Figueroa N，Salehian S S M，et al. Learning augmented joint-space task-oriented dynamical systems:A linear parameter varying and synergetic control approach[J]. IEEE Robotics and Automation Letters，IEEE，2018，3(3):2718-2725.
[66] Figueroa N，Billard A. Modeling compositions of impedance-based primitives via dynamical systems[C/CD]//Proceedings of the Workshop on Cognitive Whole-Body Control for Compliant Robot Manipulation (COWB-COMP)，2018.
[67] Kronander K，Khansari M，Billard A. Incremental motion learning with locally modulated dynamical systems[J]. Robotics and Autonomous Systems，2015，70:52-62.
[68] Perrin N，Schlehuber-Caissier P. Fast diffeomorphic matching to learn globally asymptotically stable nonlinear dynamical systems[J]. Systems and Control Letters，Elsevier B.V.，2016，96:51-59.
[69] Amanhoud W，Khoramshahi M，Billard A. A dynamical system approach to motion and force generation in contacttasks[C/CD]//Robotics: Science and Systems，2019.
[70] Amanhoud W，Khoramshahi M，Bonnesoeur M，et al. Force adaptation in contact tasks with dynamical systems[J]. Proceedings of 2020 IEEE International Conference on Robotics and Automation，2020(March):6841-6847.
[71] Salehian S S M，Billard A. A dynamical system based approach for controlling robotic manipulators during non-contact / contact transitions[J]. IEEE Robotics and Automation Letters，2018，3(4):2738-2745.
[72] 付乐，武睿，赵杰. 协作机器人安全规范:ISO/TS 15066 的演变与启示[J]. 机器人，2017 (4):532-540.Fu Le，Wu Rui，Zhao Jie. The evolution and enlightenment of safety specification of cooperativerobots:ISO/TS 15066[J]. Robot，2017，39(4):532-540.
[73] Ott C，Mukherjee R，Nakamura Y. Unified impedance and admittance control[C]//Robotics and Automation (ICRA)，2010 IEEE International Conference on，2010:554-561.
[74] ARAI H，TACHI S. Position control of a manipulator with passive joints using coupled dynamics[J]. Transactions of the Society of Instrument and Control Engineers，1989，25(9):1012-1017.
[75] Van der Linde R Q，Lammertse P. HapticMaster -A generic force controlled robot for human interaction[J]. Industrial Robot:An International Journal，2003，30(6):515-524.
[76] Chiaverini S，Sciavicco L. The parallel approach to force/position control of robotic manipulators[J]. IEEE Transactions on Robotics and Automation，1993，9(4):361-373.
[77] Franklin D W，Burdet E，Peng Tee K，et al. CNS learns stable，accurate，and efficient movements using a simple algorithm[J]. Journal of Neuroscience，2008，28(44):11165-11173.
[78] Hogan N. Impedance control:An approach to manipulation 1[J]. Techniques，1985，107:304-313.
[79] Yu W，Rosen J，Li X. PID admittance control for an upper limb exoskeleton[C]//Proceedings of the 2011 American Control Conference，2011:1124-1129.
[80] Albu-Schaffer A，Eiberger O，Grebenstein M，et al. Soft robotics[J]. IEEE Robotics & Automation Magazine，2008，15(3):20-30.
[81] Wolf S，Hirzinger G. A new variable stiffness design:Matching requirements of the next robot generation[J]. Proceedings-IEEE International Conference on Robotics and Automation，2008:1741-1746.
[82] LE TIEN L，ALBU-SCHÄFFER A，DE LUCA A，et al. Friction observer and compensation for control of robots with joint torque measurement[C]//2008 IEEE/RSJ International Conference on Intelligent Robots and Systems，IROS，2008:3789-3795.
[83] Lin C，Chang P，Luh S J. Formulation and optimization of cubic polynomial joint trajectories for mechanical manipulators[C]//1982 21st IEEE Conference on Decision and Control. IEEE，1982:330-335.
[84] Shamma J S，Athans M. Gain scheduling:Potential hazards and possible remedies[J]. IEEE Control Systems，1992，12(3):101-107.
[85] Ferraguti F，Secchi C，Fantuzzi C. A tank-based approach to impedance control with variable stiffness[C]// Proceedings - IEEE International Conference on Robotics and Automation，IEEE，2013:4948-4953.
[86] Yang C，Ganesh G，Haddadin S，et al. Human-like adaptation of force and impedance in stable and unstable interactions[J]. IEEE Transactions on Robotics，2011，27(5):918-930.
[87] Kronander K，Billard A. Stability considerations for variable impedance control[J]. IEEE Transactions on Robotics，2016(99):1298-1305.
[88] Duindam V，Stramigioli S，Scherpen J M A. Passive compensation of nonlinear robot dynamics[J]. IEEE Transactions on Robotics and Automation，2004，20(3):480-487.
[89] Li P Y，Horowitz R. Passive velocity field control of mechanical manipulators[J]. IEEE Transactions on Robotics and Automation，1999，15(4):751-763.
[90] Kronander K，Billard A. Passive interaction control with dynamical systems[J]. IEEE Robotics and Automation Letters，2016，1(1):106-113.
[91] Mitsioni I，Karayiannidis Y，Kragic D. Modelling and learning dynamics for robotic food-cutting[J/OL]. [2022-01-22]. https://doi.org/10.48550/arXiv.2003.09179.
[92] Gemici M C，Saxena A. Learning haptic representation for manipulating deformable food objects[C]//IEEE International Conference on Intelligent Robots and Systems，IEEE，2014:638-645.