面向智能汽车人机协同转向控制的强化学习变阻抗人机交互方法

doi:10.3901/JME.2022.18.141

机械工程学报 ›› 2022, Vol. 58 ›› Issue (18): 141-149.doi: 10.3901/JME.2022.18.141

扫码分享

面向智能汽车人机协同转向控制的强化学习变阻抗人机交互方法

韩嘉懿, 赵健, 朱冰

吉林大学汽车仿真与控制国家重点实验室长春 130022

收稿日期:2021-11-01 修回日期:2022-03-22 出版日期:2022-09-20 发布日期:2022-12-08
通讯作者: 朱冰(通信作者)，男，1982年出生，博士，教授，博士研究生导师。主要研究方向为智能汽车决策、规划与测试。E-mail：zhubing@jlu.edu.cn
作者简介:韩嘉懿,男,1992年出生,博士研究生。主要研究方向为智能汽车人机共驾;E-mail:jiayi.han@qq.com
基金资助:
国家自然科学基金资助项目(52172386)。

Variable Impedance-based Human-machine Interaction Method Using Reinforcement Learning for Shared Steering Control of Intelligent Vehicle

HAN Jiayi, ZHAO Jian, ZHU Bing

State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022

Received:2021-11-01 Revised:2022-03-22 Online:2022-09-20 Published:2022-12-08

摘要/Abstract

摘要： 人机交互已成为智能汽车设计的核心要素之一，针对人机协同转向控制问题，提出一种基于强化学习的智能汽车变阻抗人机交互方法。首先基于虚拟阻抗的思想提出针对转向控制的人机交互框架，用于描述控制权分配的连续过程；其次在此基础上，设计基于变阻抗的人机协同转向控制算法，通过改变虚拟阻抗动态调整控制权分配；再次开发基于深度确定性策略梯度(Deep deterministic policy gradient，DDPG)的阻抗调协策略，根据驾驶人操纵行为确定虚拟阻抗；最后进行驾驶人在环试验，试验结果表明，与常规方法相比，所提出的方法能够使自动驾驶系统根据驾驶人的操纵行为让渡一定的控制权给驾驶人，人机交互过程平稳、柔和，易于驾驶人适应，对驾驶人的影响更小，降低了驾驶人的操纵负荷，同时自动驾驶系统还能够生成适当大小的控制转矩向驾驶人表达自身的驾驶意图，实现有效的人机交互。

关键词: 智能汽车, 人机交互, 协同转向, 阻抗控制, 强化学习

Abstract: Human-Machine interaction has become one of the focuses of intelligent vehicle design. Aiming at the problem of human-machine shared steering control, a human-machine interaction method with variable impedance based on reinforcement learning is proposed. Firstly, A human-machine interaction framework for shared steering is built based on virtual impedance, which can describe the continuous process of control authority distribution. Secondly, on this basis, a variable impedance-based human-machine shared steering control method is designed, which can dynamically distribute control authority by changing virtual impedance. Thirdly, an impedance tuning strategy based on deep deterministic policy gradient (DDPG) is developed to determine the virtual impedance according to the driver's steering behavior. The driver-in-the-loop experiment shows that, compared with the conventional method, the method proposed can make the automation system yield a certain degree of control authority to the driver according to the driver’s steering behavior, and make the interaction process smooth and easy for the driver to adapt to. The method has a less bad influence on the driver and reduces the driver's steering load, and meanwhile, the automation system can also generate an appropriate control torque to express its intention to the driver, and realize the effective human-machine interaction.

Key words: intelligent vehicle, human-machine interaction, shared steering, impedance control, reinforcement learning

中图分类号:

U461

韩嘉懿, 赵健, 朱冰. 面向智能汽车人机协同转向控制的强化学习变阻抗人机交互方法[J]. 机械工程学报, 2022, 58(18): 141-149.

HAN Jiayi, ZHAO Jian, ZHU Bing. Variable Impedance-based Human-machine Interaction Method Using Reinforcement Learning for Shared Steering Control of Intelligent Vehicle[J]. Journal of Mechanical Engineering, 2022, 58(18): 141-149.

导出引用管理器 EndNote|Reference Manager|ProCite|BibTeX|RefWorks

链接本文: http://www.cjmenet.com.cn/CN/10.3901/JME.2022.18.141

http://www.cjmenet.com.cn/CN/Y2022/V58/I18/141

参考文献

[1] ESKANDARIAN A. Handbook of intelligent vehicles[M]. London:Springer，2012.
[2] 宗长富，代昌华，张东. 智能汽车的人机共驾技术研究现状和发展趋势[J]. 中国公路学报，2021，34(06):214-237.ZONG Changfu，DAI Changhua，ZHANG Dong. Human-robot interaction technology of autonomous vehicles:a review and perspectives[J]. China Journal of Highway and Transport，2021，34(06):214-237.
[3] NUNES A，REIMER B，COUGHLIN F. People must retain control of autonomous vehicles[J]. Nature，2018，556:169-171.
[4] 胡云峰，曲婷，刘俊，等. 智能汽车人机协同控制的研究现状与展望[J]. 自动化学报，2019，45(7):1261-1280.HU Yunfeng，QU Ting，LIU Jun，et al. Human-machine cooperative control of intelligent vehicle:Recent developments and future perspectives[J]. Acta Automatica Sinica，2019，45(7):1261-1280.
[5] FLEMISCH F，HEESEN M，HESSE T，et al. Towards a dynamic balance between humans and automation:authority，ability，responsibility and control in shared and cooperative control situations[J]. Cognition，Technology & Work，2012，14(1):3-18.
[6] MARCANO M，DÍAZ S，PÉREZ J，et al. A review of shared control for automated vehicles:theory and applications[J]. IEEE Transactions on Human-Machine Systems，2020，50(6):475-491.
[7] RUSSELL H，HARBOTT L，NISKY I，et al. Motor learning affects car-to-driver handover in automated vehicles[J]. Science Robotics，2016，1(1):1-9.
[8] 谢有浩，魏振亚，赵林峰，等. 基于μ综合方法的智能车辆人机共驾的鲁棒横向控制[J]. 机械工程学报，2020，56(4):104-114.XIE Youhao，WEI Zhenya，ZHAO Linfeng，et al. Robust lateral control of intelligent vehicle in the human-machine sharing based on μ-synthesis[J]. Journal of Mechanical Engineering，2020，56(4):104-114.
[9] JI X，YANG K，NA X，et al. Shared steering torque control for lane change assistance:A stochastic game-theoretic approach[J]. IEEE Transactions on Industrial Electronics，2018，66(4):3093-3105.
[10] NA X，COLE D. Game-theoretic modeling of the steering interaction between a human driver and a vehicle collision avoidance controller[J]. IEEE Transactions on Human-Machine Systems，2015，45(1):25-38.
[11] NGUYEN A，SENTOUH C，POPIEUL J. Driver-automation cooperative approach for shared steering control under multiple system constraints:Design and experiments[J]. IEEE Transactions on Industrial Electronics，2017，64(5):3819-3830.
[12] JEONG Y，CHUNG C. Nonlinear hybrid impedance control for steering control of rack-mounted electric power steering in autonomous vehicles[J]. IEEE Transactions on Intelligent Transportation Systems，2020，21(7):2956-2965.
[13] MA B，LIU Y，NA X. A shared steering controller design based on steer-by-wire system considering human-machine goal consistency[J]. Journal of the Franklin Institute，2019，356(8):4397-4419.
[14] 李海源，刘畅，严鲁涛，等. 上肢外骨骼机器人的阻抗控制与关节试验研究[J]. 机械工程学报，2019，56(19):200-209.LI Haiyuan，LIU Chang，YAN Lutao，et al. Research on impendence control of an upper limb exoskeleton robot and joint experiments[J]. Journal of Mechanical Engineering，2019，56(19):200-209.
[15] GHASEMI A，JAYAKUMAR P，GILLESPIE B. Shared control architectures for vehicle steering[J]. Cognition，Technology and Work，2019，21(4):699-709.
[16] 李超，张智，夏桂华，等. 基于强化学习的学习变阻抗控制[J]. 哈尔滨工程大学学报，2019，40(2):304-311.LI Chao，ZHANG Zhi，XIA Guihua，et al. Learning variable impedance control based on reinforcement learning[J]. Journal of Harbin Engineering University，2019，40(2):304-311.
[17] IZADI V，GHASEMI A. Modulation of control authority in adaptive haptic shared control paradigms[J]. Mechatronics，2020，78:1-12.
[18] 贺伊琳，宋若旸，马建. 基于强化学习DDPG的智能车辆轨迹跟踪控制[J]. 中国公路学报，2021，34(11):335-348. HE Yilin，SONG Ruoyang，MA Jian. Trajectory tracking control of intelligent vehicle based on ddpg method of reinforcement learning[J]. China Journal of Highuay and Transport，2021，34(11):335-348.
[19] LÜ Chen，WANG Huaji，CAO Dongpu，et al. Characterization of driver neuromuscular dynamics for human-automation collaboration design of automated vehicles[J]. IEEE/ASME Transactions on Mechatronics，2018，23(6):2558-2567

面向智能汽车人机协同转向控制的强化学习变阻抗人机交互方法

Variable Impedance-based Human-machine Interaction Method Using Reinforcement Learning for Shared Steering Control of Intelligent Vehicle

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	郭烈, 关龙新, 葛平淑, 吴晓建. 面向网络不可靠的智能汽车纵侧向运动控制研究综述[J]. 机械工程学报, 2026, 62(8): 298-316.
[2]	杨泽浩, 董威, 黄思翰, 阴艳超, 董李扬, 郑祖杰. 基于数字孪生和强化学习的智能生产线实时调度仿真优化方法[J]. 机械工程学报, 2026, 62(5): 12-25.
[3]	黄铭, 黄思翰, 陈建鹏, 董威, 王柏村, 阮兵, 高云鹏, 王国新, 阎艳. 基于多目标深度强化学习的不确定扰动下岛式装配线动态调度优化[J]. 机械工程学报, 2026, 62(5): 74-87.
[4]	张志勇, 姜志昊, 胡林, 黄彩霞. 面向连续弯道的改进人工势场建模与动态避障路径规划[J]. 机械工程学报, 2026, 62(2): 271-282.
[5]	俞滨, 何小龙, 王源, 佘进波, 巴凯先. 液压足式机器人关节底层力阻抗特性补偿控制[J]. 机械工程学报, 2025, 61(7): 294-300.
[6]	胡炳涛, 钟锐锐, 冯毅雄, 杨晨, 王天跃, 洪兆溪, 谭建荣. 人-信息-物理互联环境下数字车间制造能力建模与自适应调度[J]. 机械工程学报, 2025, 61(3): 23-39.
[7]	杜国锋, 邵士博, 李尚霖, 林诚然, 曹政才. 融合视觉语言模型与近端策略优化算法的人形机器人步态切换方法[J]. 机械工程学报, 2025, 61(21): 204-212.
[8]	唐小林, 甘炯鹏, 张振果. 横纵向耦合跟车场景下基于多智能体深度强化学习的混合动力车队协同能量管理研究[J]. 机械工程学报, 2025, 61(2): 236-246.
[9]	朱鹏兴, 胡建军, 李嘉佳, 彭航, 王鑫. 考虑驾驶风格不确定性的插电式混合动力汽车个性化生态自适应巡航协同优化方法[J]. 机械工程学报, 2025, 61(18): 214-229.
[10]	林歆悠, 代军, 曾松榕. 基于运动风险的强化学习换道决策方法研究[J]. 机械工程学报, 2025, 61(18): 252-266.
[11]	刘建军, 邱俊豪, 张忠明. 复杂制造系统人机交互调度与案例研究[J]. 机械工程学报, 2025, 61(15): 351-371.
[12]	曹延旭, 魏中宝, 宋若旸, 何洪文. 基于深度强化学习的跟驰场景中燃料电池混合动力汽车多目标能量管理研究[J]. 机械工程学报, 2025, 61(12): 191-203.
[13]	李明富, 王飞鸿, 朱凌枫, 李想, 雷高攀, 刘翊, 李林凌, 侯宇葵, 胡余良. 接触丰富的自动装配方法研究进展：机器人装配接触力控制技能学习[J]. 机械工程学报, 2025, 61(10): 395-413.
[14]	刘东博, 陈力. 基于模糊逻辑的双臂空间机器人在轨辅助对接操作变结构力/位控制[J]. 机械工程学报, 2025, 61(1): 60-70.
[15]	王无印, 黄子钊, 庄子龙, 方怀瑾, 秦威. 基于深度强化学习的自动化码头堆场场桥调度方法[J]. 机械工程学报, 2024, 60(6): 44-57.