Whole-body Motion Skills Generation for Humanoid Robots Based on Multi-critic Actor Imitation Learning

doi:10.3901/JME.260393

Abstract

Abstract: Whole-body motion skills are fundamental for humanoid robots to achieve complex behaviors and perform diverse tasks. Although existing motion control methods perform well in simulation, transferring them to the real world remains challenging. Humanoid robots typically have high degrees of freedom and strong dynamic coupling. As a result, accurate motion imitation and stable sim-to-real transfer are difficult to achieve under dynamics discrepancies and sensor noise. To address these issues, this research proposes a humanoid motion skill learning method based on multi-critic actor imitation learning. First, a difficulty-aware curriculum learning strategy based on a motion complexity metric is designed, enabling the robot to progressively learn complex motion skills from diverse motion datasets. Then, a multi-critic actor network architecture is introduced to improve the fidelity of imitation to motion capture data while balancing motion imitation accuracy and sim-to-real transfer capability. Experimental results show that the proposed method achieves high-precision motion imitation on the Unitree G1 humanoid robot. Motion skills learned in simulation are successfully transferred to the real world in a zero-shot setting. This demonstrates the effectiveness and generalization capability of the proposed method in real-world scenarios.

Key words: humanoid robots, multi-critic actor imitation learning, motion control

CLC Number:

TG242

SHAO Shibo, LINChengran, DU Guofeng, LI Junnian, CAO Zhengcai. Whole-body Motion Skills Generation for Humanoid Robots Based on Multi-critic Actor Imitation Learning[J]. Journal of Mechanical Engineering, 2026, 62(11): 191-200.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: http://www.cjmenet.com.cn/EN/10.3901/JME.260393

http://www.cjmenet.com.cn/EN/Y2026/V62/I11/191

References

[1] SFERRAZZA C，HUAND D，LIN X，et al. Humanoid-bench：Simulated humanoid benchmark for whole-body locomotion and manipulation[EB/OL]. arXiv，2024. arXiv：2403.10506. https：//arxiv.org/ abs/2403.10506.
[2] CHEN Z，JI M，CHENG X，et al. Gmt：General motion tracking for humanoid whole-body control[EB/OL]. arXiv，2025. arXiv：2506.14770. https：//arxiv.org/abs/ 2506.14770.
[3] XU M，SHI Y，YIN K K，et al. Parc：Physics-based augmentation with reinforcement learning for character controllers[C]// Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference. New York，NY，United States：Association for Computing Machinery. 2025：1-11.
[4] YUAN M，YU T，GE W，et al. A Survey of behavior foundation model：Next-generation whole-body control system of humanoid robots[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2025(1)：1-20.
[5] ZE Y，CHEN Z，ARAUJO J P，et al. TWIST：Teleoperated whole-body imitation system[C]// Proceedings of the Conference on Robot Learning. Seoul，Korea：PMLR，2025：2143-2154.
[6] BARREIROS J A，ÖNOL A Ö，ZHANG M，et al. Learning contact-rich whole-body manipulation with example-guided reinforcement learning[J]. Science Robotics，2025，10(105)：eads6790.
[7] JI M，PENG X，LIU F，et al. Exbody2：Advanced expressive humanoid whole-body control[EB/OL]. arXiv，2024.arXiv：2412.13196. https：//arxiv.org/abs/2412. 13196.
[8] RAMOS J，KIM S. Dynamic locomotion synchronization of bipedal robot and human operator via bilateral feedback teleoperation[J]. Science Robotics，2019，4(35)：eaav4282.
[9] DALLARD A，BENALLEGUE M，KANEHIRO F，et al. Synchronized human-humanoid motion imitation[J]. IEEE Robotics and Automation Letters，2023，8(7)：4155-4162.
[10] KOSEKI S，MOHSENI O，OWAKI D，et al. Concerted control：Modulating joint stiffness using GRF for gait generation at different speeds[J]. IEEE Robotics and Automation Letters，2025，10(4)：3446-3453.
[11] 杜国锋，邵士博，李尚霖，等. 融合视觉语言模型与近端策略优化算法的人形机器人步态切换方法[J]. 机械工程学报，2025，61(21)：204-212. DU Guofeng，SHAO Shibo，LI Shanglin，et al. Gait switching method for humanoid robot integrating vision-language[J]. Journal of Mechanical Engineering，2025，61(21)：204-212.
[12] 钟沛成，骆德渊，庞明君. 基于深度强化学习的四足机器人跟随策略研究及系统实现[J]. 机械工程学报，2023，59(13)：79-88. ZHONG Peicheng，LUO Deyuan，PANG Mingjun. Research and system implementation of quadruped robot following strategy based on deep reinforcement learning[J]. Journal of Mechanical Engineering，2023，59(13)：79-88.
[13] 陈词，余纪宇，李超，等. 基于预训练―微调框架的四足机器人结构―控制协同设计[J]. 机器人，2025，47(5)：625-635. CHEN Ci，YU Jiyu，LI Chao，et al. Structure-control co-design of quadruped robots based on pre-training- fine-tuning framework[J]. Robot，2025，47(5)：625-635.
[14] LI Y，ZHENG L，WANG Y，et al. Impedance learning-based adaptive force tracking for robot on unknown terrains[J]. IEEE Transactions on Robotics，2025，41：1404-1420.
[15] WU F，NAL X，JANG J，et al. Learn to teach：Sample-efficient privileged learning for humanoid locomotion over real-world uneven terrain[J]. IEEE Robotics and Automation Letters，2025，10(9)：9048-9055.
[16] CAO Z，LI J，SHAO S，et al. Siamese adaptive network-based accurate and robust visual object tracking algorithm for quadrupedal robots[J]. IEEE Transactions on Cybernetics，2025，55(3)：1264-1276.
[17] MA Y，CRAMARIUC A，FARSHIDIAN F，et al. Learning coordinated badminton skills for legged manipulators[J]. Science robotics，2025，10(102)：eadu3922.
[18] HWANGBO J，LEE J，DOSOVITSKIY A，et al. Learning agile and dynamic motor skills for legged robots[J]. Science Robotics，2019，4(26)：eaau5872.
[19] MARGOLIS G B，YANG G，PAIGWAR K，et al. Rapid locomotion via reinforcement learning[J]. The International Journal of Robotics Research，2024，43(4)：572-587.
[20] Unitree. Unitree G1：Humanoid agent AI Avatar[EB/OL]. [2024]. https：//www.unitree.com/g1.
[21] MAHMOOD N，GHORBANI N，TROJE N F，et al. AMASS：Archive of motion capture as surface shapes[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Archive of Motion Capture as Surface Shapes，2019：5442-5451.
[22] SCHULMAN J，WOLSKI F，DHARIWAL P，et al. Proximal policy optimization algorithms[EB/OL]. arXiv，2017. arXiv：1707.06347，https：//arxiv.org/abs/1707. 06347.
[23] SCHULMAN J，MORITZ P，LEVINE S，et al. High-dimensional continuous control using generalized advantage estimation[EB/OL]. arXiv，2015. arXiv：1506.02438，https：//arxiv.org/abs/1506.02438.
[24] FU Y，XIE F，XU C，et al. DemoHLM：From one demonstration to generalizable humanoid loco- manipulation[J]. IEEE Robotics and Automation Letters，2026，11(4)：4393-4400.
[25] MITTAL M，YU C，YU Q，et al. Orbit：A unified simulation framework for interactive robot learning environments[J]. IEEE Robotics and Automation Letters，2023，8(6)：3740-3747.
[26] ZHANG Q，CUI P，YAN D，et al. Whole-body humanoid robot locomotion with human reference[C]// Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and System. NJ，USA：IEEE，2024：11225-11231.