Research on Safe Decision-making for Autonomous Driving at Unsignalized Intersections

doi:10.3901/JME.2024.10.147

Abstract

Abstract: Proposes a safe decision-making method for unsignalized intersections, focusing on driving risks caused by environmental occlusion and the random behaviors of traffic participants. Firstly, a fundamental decision-making policy is established based on the value-distributional reinforcement learning-fully parameterized quantile network(FPQN). Secondly, the cumulative reward distribution modeled by FPQN and conditional value at risk(CVaR) are integrated to construct a driving risk-aware decision-making policy. Thirdly, the ensemble learning theory is introduced to establish the decision-making uncertainty estimation framework based on ensemble FPQN(EFPQN), which can quantify decision risks in real-time. Meanwhile, to handle the driving risk induced by high decision-making uncertainty, a model predictive control-based backup strategy is designed. Finally, the proposed safe decision-making method is validated using the SUMO simulation platform in an unsignalized intersection scenario. Experimental results show that the proposed method effectively reduces driving risks caused by environmental occlusion and random behaviors of traffic participants compared with baseline methods.

Key words: unsignalized intersections, environmental occlusion, aleatoric uncertainty, unknown scenario, epistemic uncertainty

CLC Number:

U461

YANG Kai, TANG Xiaolin, ZHONG Guichuan, WANG Ming, LI Guofa, HU Xiaosong. Research on Safe Decision-making for Autonomous Driving at Unsignalized Intersections[J]. Journal of Mechanical Engineering, 2024, 60(10): 147-159.

References

[1] 李克强，戴一凡，李升波，等. 智能网联汽车(ICV)技术的发展现状及趋势[J]. 汽车安全与节能学报，2017，8(1):1-14. LI Keqiang，DAI Yifan，LI Shengbo，et al. State-of-the-artand technical trends of intelligent and connected vehicles[J]. Journal of Automotive Safety and Energy，2017，8(1):1-14.
[2] BOARD N T S. Collision between vehicle controlled by developmental automated driving system and pedestrian[EB/OL]. [2023-04-14]. https://www.ntsb.gov/investigations/AccidentReports/Reports/HAR1903.pdf.
[3] HO S S. Complementary and competitive framing of driverless cars:Framing effects，attitude volatility，or attitude resistance?[J]. International Journal of Public Opinion Research，2021，33(3):512-531.
[4] BRITO B，AGARWAL A，ALONSO-MORA J. Learning interaction-aware guidance for trajectory optimization in dense traffic scenarios[J]. IEEE Transactions on Intelligent Transportation Systems，2022，22(10):18808-18821.
[5] SCHWARTING W，ALONSO-MORA J，RUS D. Planning and decision-making for autonomous vehicles[J]. Annual Review of Control，Robotics，and Autonomous Systems，2018，1:187-210.
[6] ZHANG M，LI N，GIRARD A，et al. A finite state machine based automated driving controller and its stochastic optimization[C]//Dynamic Systems and Control Conference. Tysons，Virginia，USA:ASME，2017:V002T07A002.
[7] BUEHLER M，IAGNEMMA K，SINGH S. The DARPA urban challenge:Autonomous vehicles city traffic[M]. Berlin:Springer，2009.
[8] 阳鑫，唐小林，杨凯，等. 极限工况下无人驾驶车辆运动规划策略研究[J]. 机械工程学报，2022，58(22):349-359. YANG Xin，TANG Xiaolin，YANG Kai，et al. Research on the motion planning strategy for autonomous vehicles in extreme conditions[J]. Journal of Mechanical Engineering，2022，58(22):349-359.
[9] QIAN X，NAVARRO I，DE LA FORTELLE A，et al. Motion planning for urban autonomous driving using Bézier curves and MPC[C]//2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). Rio de Janeiro，Brazil:IEEE，2016:826-833.
[10] HUANG Y，WANG H，KHAJEPOUR A，et al. A novel local motion planning framework for autonomous vehicles based on resistance network and model predictive control[J]. IEEE Transactions on Vehicular Technology，2019，69(1):55-66.
[11] ZHANG C，STEINHAUSER F，HINZ G，et al. Traffic mirror-aware POMDP behavior planning for autonomous urban driving[C]//2022 IEEE Intelligent Vehicles Symposium (IV). Aachen:IEEE，2022:323-330.
[12] QIAO Z，MUELLING K，DOLAN J，et al. Pomdp and hierarchical options mdp with continuous actions for autonomous driving at intersections[C]//2018 21st International Conference on Intelligent Transportation Systems (ITSC). Maui，HI:IEEE，2018:2377-2382.
[13] BAI H，HSU D，LEE W S. Integrated perception and planning in the continuous space:A POMDP approach[J]. The International Journal of Robotics Research，2014，33(9):1288-1302.
[14] BRECHTEL S，GINDELE T，DILLMANN R. Probabilistic decision-making under uncertainty for autonomous driving using continuous POMDPs[C]//17th International IEEE Conference on Intelligent Transportation Systems (ITSC). Qingdao:IEEE，2014:392-399.
[15] ZHU Z，ZHAO H. A survey of deep RL and IL for autonomous driving policy learning[J]. IEEE Transactions on Intelligent Transportation Systems，2021，23(9):14043-14065.
[16] ISELE D，RAHIMI R，COSGUN A，et al. Navigating occluded intersections with autonomous vehicles using deep reinforcement learning[C]//2018 IEEE International Conference on Robotics and Automation (ICRA). Brisbane，QLD，Australia:IEEE，2018:2034-2039.
[17] TRAM T，JANSSON A，GRÖNBERG R，et al. Learning negotiating behavior between cars in intersections using deep Q-learning[C]//2018 21st International Conference on Intelligent Transportation Systems (ITSC). Maui，HI:IEEE，2018:3169-3174.
[18] TREIBER M，HENNECKE A，HELBING D. Congested traffic states in empirical observations and microscopic simulations[J]. Physical Review E，2000，62(2):1805.
[19] DABNEY W，OSTROVSKI G，SILVER D，et al. Implicit quantile networks for distributional reinforcement learning[C]//35th International Conference on Machine Learning (ICML). Stockholm:ACM，2018:1096-1105.
[20] YANG D，ZHAO L，LIN Z，et al. Fully parameterized quantile function for distributional reinforcement learning[J]. Advances in Neural Information Processing Systems，2019，32:6193-6202.
[21] VON NEUMANN J，MORGENSTERN O. Theory of games and economic behavior[M]. Princeton:Princefon Princeton University Press, 2007.
[22] CHAPMAN M P，BONALLI R，SMITH K M，et al. Risk-sensitive safety analysis using conditional value-at-risk[J]. IEEE Transactions on Automatic Control，2021，67(12):6521-6536.
[23] BERNHARD J，POLLOK S，KNOLL A. Addressing inherent uncertainty:Risk-sensitive behavior generation for automated driving using distributional reinforcement learning[C]//2019 IEEE Intelligent Vehicles Symposium (IV). Paris:IEEE，2019:2148-2155.
[24] HOEL C J，WOLFF K，LAINE L. Tactical decision-making in autonomous driving by reinforcement learning with uncertainty estimation[C]//2020 IEEE Intelligent Vehicles Symposium (IV). Las Vegas:IEEE，2020:1563-1569.
[25] HOEL C J，WOLFF K，LAINE L. Ensemble quantile networks:Uncertainty-aware reinforcement learning with applications in autonomous driving[J]. IEEE Transactions on Intelligent Transportation Systems，2023，24(6):6030-6041.
[26] ZANELLI A，DOMAHIDI A，JEREZ J，et al. FORCES NLP:An efficient implementation of interior-point methods for multistage nonlinear nonconvex programs[J]. International Journal of Control，2020，93(1):13-29.