Research and System Implementation of Quadruped Robot Following Strategy Based on Deep Reinforcement Learning

doi:10.3901/JME.2023.13.079

Abstract

Abstract: The target following strategy is an important part of the target following system of the quadruped robot. Aiming at the problems of many random factors of target motion, complex system decision-making and insufficient robustness of real-world deployment in the following process, firstly, a target following strategy based on deep reinforcement learning is proposed, which is based on the spatial position information of the input target relative to the robot, output the following action command to realize the robot's following decision to the random moving target. Then, the robot is trained using a deep reinforcement learning algorithm based on the Actor-Critic framework, and observation noise is added to obtain a more robust following strategy and a correction factor is introduced to reduce the speed deviation of the robot in the simulated environment and the real environment. The initial verification is carried out on the simulation platform, and finally the following strategy is deployed on the quadruped robot for experimental verification. The results show that the system has good following performance and meets the needs of most application scenarios.

Key words: quadruped, object following system, deep reinforcement learning, object following strategy

CLC Number:

TG156

ZHONG Peicheng, LUO Deyuan, PANG Mingjun. Research and System Implementation of Quadruped Robot Following Strategy Based on Deep Reinforcement Learning[J]. Journal of Mechanical Engineering, 2023, 59(13): 79-88.

References

[1] RAIBERT M,BLANKESPOOR K,NELSON G,et al. BigDog,the rough-terrain quadruped robot[C]//Proceedings of the 17th World Congress. The International Federation of Automatic Control,Seoul:IFAC,2008:10822-10825.
[2] HUTTER M,GEHRING C,JUD D,et al. ANYmal-A highly mobile and dynamic quadruped robot[C]//2016 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),Daejeon:IEEE,2016:38-44.
[3] HUTTER M,GEHRING C,BLOESCH M,et al. StarlETH:A compliant quadrupedal robot for fast,efficient,and versatile locomotion[C]//International Conference on Climbing and Walking Robots (CLAWAR). Adaptive Mobile Robotics,2012:483-490.
[4] KIM D,CARLO J D,KATZ B,et al. Highly dynamic quadruped locomotion via whole-body impulse control and model predictive control[J/OL]. arXiv.org,2019.
[5] BLEDT G,POWELL M J,KATZ B,et al. MIT Cheetah 3:Design and control of a robust,dynamic quadruped robot[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS). IEEE,2018:2245-2252.
[6] PERDOCH M,BRADLEY D M,CHANG J K,et al. Leader tracking for a walking logistics robot[C]//2015 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),Hamburg:IEEE,2015:2994-3001.
[7] PANG L,CAO Zhiqiang,YU Junzhi,et al. A visual leader-following approach with a T-D-R framework for quadruped robots[J]. IEEE Transactions on Systems,Man and Cybernetics:Systems,2019,51(4):2342-2354.
[8] PARK J H,RYU H,CHOI Y H,et al. User following strategy for mobile robots[C]//International Conference on Ubiquitous Robots & Ambient Intelligence,Jeju:IEEE,2013:717-719.
[9] JIA S,WANG L,SHUANG W,et al. Fuzzy-based intelligent control strategy for a person following robot[C]//2013 IEEE International Conference on Robotics and Biomimetics (ROBIO),Shenzhen:IEEE,2013:2408-2413.
[10] CHOU K Y,CHEN Y T,LIN J K,et al. Q-learning based tracking control and slope climbing strategy design of autonomous mobile robot and flatbed vehicle[C]//2021 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW),Penghu:IEEE,2021:1-2.
[11] CHANDRA K D,JUN M. Integrating multiple policies for person-following robot training using deep reinforcement learning[J]. IEEE Access,2021,9:75526-75541.
[12] LUO W,SUN P,ZHONG F,et al. End-to-end active object tracking and its real-world deployment via reinforcement learning[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence,2019,42(6):1317-1332.
[13] LI Z,LI B,LIANG Q,et al. Research and realization of target following and autonomous obstacle avoidance algorithm of quadruped robot[C]//2021 40th Chinese Control Conference(CCC),Shanghai:IEEE,2021:3984-3989.
[14] ZHANG H,LIU H,DENG L,et al. Leader recognition and tracking for quadruped robots[C]//2018 IEEE International Conference on Information and Automation (ICIA),Wuyishan:IEEE,2018:1438-1443.
[15] ZHANG Z,YAN J,KONG X,et al. Efficient motion planning based on kinodynamic model for quadruped robots following persons in confined spaces[J]. IEEE/ASME Transactions on Mechatronics,2021,26(4):1997-2006.
[16] REDMON J,DIVVALA S,GIRSHICK R,et al. You only look once:Unified,real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR),Las Vegas:IEEE,2016:779-788.
[17] BOCHKOVSKIY A,WANG C Y,LIAO H. YOLOv4:Optimal speed and accuracy of object detection[J]. arXiv,2020,2004:10934.
[18] WOJKE N,BEWLEY A,PAULUS D. Simple online and realtime tracking with a deep association metric[C]//2017 IEEE International Conference on Image Processing(ICIP),Beijing:IEEE,2017:3645-3649.
[19] BEWLEY A,GE Z,OTT L,et al. Simple online and realtime tracking[C]//2016 IEEE International Conference on Image Processing(ICIP),Phoenix:IEEE,2016:3464-3468.
[20] SCHULMAN J,WOLSKI F,DHARIWAL P,et al. Proximal policy optimization algorithms[J]. arXiv.org,2017.
[21] SCHULMAN J,LEVINE S,MORITZ P,et al. Trust region policy optimization[C]//Proceedings of the 31th International Conference on Machine Learning,Lille:JMLR:W&CP,2015:1889-1897.