Intelligent Posture Control of Humanoid Robot in Variable Environment

doi:10.3901/JME.2020.03.064

Abstract

Abstract: To solve the problems of motion instability of humanoid robots in variable uncertain, unstructured terrain and the low accuracy motion control, intelligent posture motion control algorithm is proposed. The deep reinforcement learning based continuous motion and continuous state space is applied to posture control, and the humanoid robot motion intelligent posture controller is established. Aiming at the problems of less sample and low efficiency of physical prototype training, the identification robot model is present to perform offline pre-training of the posture controller as a prior knowledge for continuous learning and in the real physical environment, improve the training efficiency in the later stage. The optimized robot posture controller is applied to the motion control of the robot. Compared with the robot motion with PID controller, MPC controller and PID+MPC controller, the standard deviation of the upper body pitch posture trajectory tracking error of the robot is reduced by 60.97%, 46.36%, 23.98% in the environmental transitional walking test, respectively. In the walking test of ground obstacles, the standard deviations of the trajectory tracking errors of the robot's upper body pitching posture are reduced by 60.38%, 26.38% and 9.52%, respectively.

Key words: bipedal walking, deep reinforcement learning, motion control

CLC Number:

TG156

SHI Qun, Lü Lei, XIE Jiajun. Intelligent Posture Control of Humanoid Robot in Variable Environment[J]. Journal of Mechanical Engineering, 2020, 56(3): 64-72.

References

[1] REHER J,COUSINEAU E A,HEREID A,et al. Realizing dynamic and efficient bipedal locomotion on the humanoid robot DURUS[C]//IEEE International Conference on Robotics and Automation,IEEE,2016:1794-1801.
[2] 丁长涛. 随机不确定扰动下平面双足机器人动力学响应特性研究[D]. 杭州:浙江大学,2016. DING Changtao. Study on dynamic response characteristics of planar biped robot under randomly uncertain disturbance[D]. Hangzhou:Zhejiang University,2016.
[3] ZHOU Shihua,SONG Guiqiu,REN Zhaohui,et al. Nonlinear dynamic analysis of coupled gear-rotor-bearing system with the effect of internal and external excitations[J]. Chinese Journal of Mechanical Engineering,2016,30(2):281-292.
[4] 陈奇石. 强化学习在仿人机器人行走稳定控制上的研究及实现[D]. 广州:华南理工大学,2016. CHEN Qishi. Research and implementation of reinforcement learning on walking stability control of humanoid robots[D]. Guangzhou:South China University of Technology,2016.
[5] 王文玺,肖世德,孟祥印,等. 基于Agent的递阶强化学习模型与体系结构[J]. 机械工程学报,2010,46(2):76-82. WANG Wenxi,XIAO Shide,MENG Xiangyin,et al. Agent-based hierarchical reinforcement learning model and architecture[J]. Journal of Mechanical Engineering,2010,46(2):76-82.
[6] SHAO Shiyu,SUN Wenjun,YAN Ruqiang,et al. A deep learning approach for fault diagnosis of induction motors in manufacturing[J]. Chinese Journal of Mechanical Engineering,2017,30(6):1347-1356.
[7] 侯文擎,叶鸣,李巍华. 基于改进堆叠降噪自编码的滚动轴承故障分类[J]. 机械工程学报,2018,54(7):87-96. HOU Wenqing,YE Ming,LI Weihua. Fault classification of rolling bearings based on improved stack noise reduction self-coding[J]. Journal of Mechanical Engineering,2018,54(7):87-96.
[8] HWANG K S,LIN J L,LI J S. Biped balance control by reinforcement learning[J]. Journal of Information Science and Engineering,2016,32(4):1041-1060.
[9] SILVA I J,PERICO D H,HOMEM T P D,et al. Using reinforcement learning to improve the stability of a humanoid robot:Walking on sloped terrain[C]//Robotics Symposium (LARS) and 2015 3rd Brazilian Symposium on Robotics (LARS-SBR),2015 12th Latin American. IEEE,2015:210-215.
[10] WU W,GAO L. Posture self-stabilizer of a biped robot based on training platform and reinforcement learning[J]. Robotics and Autonomous Systems,2017,98:42-55.
[11] WANG S,BRAAKSMA J,BABUSKA R,et al. Reinforcement learning control for biped robot walking on uneven surfaces[C]//International Joint Conference on Neural Networks. July 16-21,2006, Sheraton Vancouver Wall Center Hotel,Vancouver,BC,Canada. IEEE,2006:4173-4178.
[12] 刘道远. 基于Q学习的欠驱动双足机器人行走控制研究[D]. 杭州:浙江大学,2013. LIU Daoyuan. Research on walking control of under-actuated biped robot based on Q-learning[D]. Hangzhou:Zhejiang University,2013.
[13] XIE Z. Feedback control for Cassie with deep reinforcement learning[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Oct 1-5,2018,Madrid Spain. IEEE,2018:1241-1246.
[14] UYANIK I,ANKARALI M M,COWAN N J,et al. Identification of a vertical hopping robot model via harmonic transfer functions[J]. Transactions of the Institute of Measurement and Control,2016,38(5):501-511.
[15] ZHANG Junyuan,JIN Yang,XIE Lisha,et al. Establishment and validation for the theoretical model of the vehicle airbag[J]. Chinese Journal of Mechanical Engineering,2015,29(3):27-34.
[16] DIAS J E A,PEREIRA G A S,PALHARES R M. Longitudinal model identification and velocity control of an autonomous car[J]. IEEE Transactions on Intelligent Transportation Systems,2015,16(2):776-786.
[17] QIN S J,LIN W,LJUNG L. A novel subspace identification approach with enforced causal models[J]. Automatica,2005,41(12):2043-2053.
[18] STIMAC G,BRAUT S. Comparative analysis of PSO algorithms for PID controller tuning[J]. Chinese Journal of Mechanical Engineering,2014,28(5):928-936.
[19] VAZQUEZ S,RODRIGUEZ J,RIVERA M,et al. Model predictive control for power converters and drives:Advances and trends[J]. IEEE Transactions on Industrial Electronics,2017,64(2):935-947.
[20] PINSKER J E,LEE J B,DASSAU E,et al. Response to comment on Pinsker et al. Randomized crossover comparison of personalized MPC and PID control algorithms for the artificial pancreas[J]. Diabetes Care,2016,39(1):1135-1142.