基于视觉相机和激光雷达融合的无人车障碍物检测与跟踪研究

doi:10.3901/JME.2025.02.296

机械工程学报 ›› 2025, Vol. 61 ›› Issue (2): 296-309,320.doi: 10.3901/JME.2025.02.296

• 运载工程 • 上一篇

扫码分享

基于视觉相机和激光雷达融合的无人车障碍物检测与跟踪研究

魏超^1,2, 吴西涛¹, 朱耿霆¹, 舒用杰¹, 李路兴¹, 随淑鑫¹

1. 北京理工大学坦克传动国防科技重点实验室北京 100081;
2. 北京理工大学前沿技术研究院北京 100081

收稿日期:2024-01-10 修回日期:2024-08-24 发布日期:2025-02-26
作者简介:魏超，男，1980年出生，博士，教授，博士研究生导师。主要研究方向为无人驾驶车辆技术。E-mail：weichaobit@163.com;吴西涛(通信作者)，男，1991年出生，博士研究生。主要研究方向为无人驾驶车辆技术。E-mail：wu.xitao@qq.com
基金资助:
国家自然科学基金资助项目(51875039)。

Research on Obstacle Detection and Tracking of Autonomous Vehicles Based on the Fusion of Vision Camera and LiDAR

WEI Chao^1,2, WU Xitao¹, ZHU Gengting¹, SHU Yongjie¹, LI Luxing¹, SUI Shuxin¹

1. Science and Technology on Vehicle Transmission Laboratory, Beijing Institute of Technology, Beijing 100081;
2. Institute of Advanced Technology, Beijing Institute of Technology, Beijing 100081

Received:2024-01-10 Revised:2024-08-24 Published:2025-02-26

摘要/Abstract

摘要： 为提高无人车障碍物检测跟踪的精度和稳定性，首先针对YOLO v5(You only look once version 5，YOLO v5)网络存在的语义信息和候选框信息丢失的问题，引入深度可分离空洞空间金字塔结构与目标框加权融合算法完成对网络的优化；其次针对单阶段障碍物点云聚类精度低的问题，设计一种考虑点云距离与外轮廓连续性的两阶段障碍物点云聚类方法并完成三维包围盒的建立；最后将注意力机制引入MobileNet使网络更加聚焦于目标对象特有的视觉特征，并综合利用视觉特征和三维点云信息共同构建关联性度量指标，提高匹配精度。利用KITTI数据集对构建的障碍物目标检测、跟踪与测速算法进行仿真测试，并搭建实车平台进行真实环境试验，验证所提算法的有效性和真实环境可迁移性。

关键词: 视觉相机, 激光雷达, 目标检测, 多目标跟踪, 无人车

Abstract: To improve the accuracy and stability of obstacle detection and tracking, depthwise separable atrous spatial pyramid pooling(DASPP) layer and weighted boxes fusion(WBF) algorithm are firstly introduced into you only look once version 5(YOLO v5) to tackle the problems of loss of semantic information and candidate box information, respectively. Then, a two-stage point cloud clustering method considering the point cloud distance and the continuity of the outer contour is proposed and a bounding box is established to improve the clustering accuracy of each target while ensuring the recall rate of obstacle targets. Finally, the convolutional block attention module(CBAM) is added into MobileNet to effectively extract the visual features of the obstacle target, visual features and 3D information are combined to establish correlation metrics and thus to improve tracking precision. Tests based on KITTI dataset and real environments show the effectiveness and transferability of the proposed algorithm.

Key words: vision camera, LiDAR, object detection, multi-object tracking, autonomous vehicle

中图分类号:

U463

魏超, 吴西涛, 朱耿霆, 舒用杰, 李路兴, 随淑鑫. 基于视觉相机和激光雷达融合的无人车障碍物检测与跟踪研究[J]. 机械工程学报, 2025, 61(2): 296-309,320.

WEI Chao, WU Xitao, ZHU Gengting, SHU Yongjie, LI Luxing, SUI Shuxin. Research on Obstacle Detection and Tracking of Autonomous Vehicles Based on the Fusion of Vision Camera and LiDAR[J]. Journal of Mechanical Engineering, 2025, 61(2): 296-309,320.

参考文献

[1] BADUE C，GUIDOLINI R，CARNEIRO R V，et al. Self-driving cars：A survey[J]. Expert Systems with Application，2021，165：1-27.
[2] EVERINGHAM M，GOOL L V，WILLIAMS C K I，et al. The pascal visual object classes (voc) challenge[J]. International Journal of Computer Vision，2010，88(2)：303-338.
[3] RUSSAKOVSKY O，DENG Jia，SU Hao，et al. Imagenet large scale visual recognition challenge[J]. International Journal of Computer Vision，2015，115(3)：211-252.
[4] HINTON G E，SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science，2006，313(5786)：504-507.
[5] LECUN Y，BENGIO Y，HINTON G，et al. Deep learning[J]. Nature，2015：436-444.
[6] GIRSHICK R，DONAHUE J，DARRELL T，et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition，Columbus，New York：IEEE，2014：580-587.
[7] HE Kaiming，ZHANG Xiangyu，REN Shaoqing，et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37(9)：1904-1916.
[8] GIRSHICK R. Fast R-CNN[C]// Proceedings of the IEEE International Conference on Computer Vision，Santiago，Chile：IEEE，2015：1440-1448.
[9] REN Shaoqing，HE Kaiming，GIRSHICK R，et al. Faster r-cnn：Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39(6)：1137-1149.
[10] REDMON J，DIVVALA S，GIRSHICK R，et al. You only look once：Unified，real-time object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，NV，New York：IEEE，2016：779-788.
[11] REDMON J，FARHADI A. Yolo9000：Better，faster，stronger[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，New York： IEEE，2017：6517-6525.
[12] REDMON J，FARHADI A. Yolov3：An incremental improvement[J]. arXiv preprint，2018，arXiv:1804.02767.
[13] LIU Wei，ANGUELOV D，ERHAN D，et al. Ssd：single shot multibox detector[C]// European Conference on Computer Vision，Amsterdam，Netherlands：Springer International Publishing，2016：21-37.
[14] LIN T Y，GOYAL P，GIRSHICK R，et al. Focal loss for dense object detection[C]// IEEE International Conference on Computer Vision，Venice，Italy：IEEE，2017：2999-3007.
[15] GUHA S，RASTOGI R，SHIM K. Cure：An efficient clustering algorithm for large databases[J]. Information Systems，1998，26(1)：35-58.
[16] MACQUEEN J B. Some methods for classification and analysis of multivariate observations[C]// Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability，Berkeley，CA：University of California Press，1967：281-297.
[17] AGRAWAL R，FALOUTSOS C，SWAMI A N. Efficient similarity search in sequence databases[C]// International Conference on Foundations of Data Organization and Algorithms：4th International Conference，Chicago，Illinois，USA：Springer Berlin Heidelberg，1993：69-84.
[18] ESTER M，KRIEGEL H P，SANDER J，et al. A density-based algorithm for discovering clusters in large spatial databases with noise[C]// Proceedings of 2nd International Conference on knowledge Discovery and Data Mining，Portland，Oregon，USA，1996：226-231.
[19] AVIDAN S. Support vector tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2004，26(8)：1064-1072.
[20] XIANG Yu，ALAHI A，SAVARESE S. Learning to track：Online multi-object tracking by decision making[C]// IEEE International Conference on Computer Vision，Santiago，Chile：IEEE，2015：4705-4713.
[21] MILAN A，REZATOFIGHI H，DICK A，et al. Online multi-target tracking using recurrent neural networks[C]// Proceedings of the AAAI Conference on Artificial Intelligence，San Francisco，California，USA：AAAI，2017：4225-4232.
[22] BEWLEY A，GE Z，OTT L，et al. Simple online and realtime tracking[C]// 2016 IEEE International Conference on Image Processing，Phoenix，Arizona，New York：IEEE，2016：3464-3468.
[23] WOJKE N，BEWLEY A，PAULUS D. Simple online and realtime tracking with a deep association metric[C]// 2017 IEEE International Conference on Image Processing，Beijing，China：IEEE，2017：3645-3649.
[24] SON J，BAEK M，CHO M，et al. Multi-object tracking with quadruplet convolutional neural networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，New York：IEEE，2017：3786-3795.
[25] SCHEIDEGGER S，BENJAMINSSON J，ROSENBERG E，et al. Mono-camera 3d multi-object tracking using deep learning detections and pmbm filtering[C]// 2018 IEEE Intelligent Vehicles Symposium，Suzhou，China：IEEE，2018：433-440.
[26] YU F，KOLTUN V. Multi-scale context aggregation by dilated convolutions[C]// 2016 International Conference on Learning Representations，San Juan，Puerto Rico，2016：1678-1693.
[27] HOWARD A G，ZHU Menglong，CHEN Bo，et al. Mobilenets：Efficient convolutional neural networks for mobile vision applications[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，New York：IEEE，2017：3666-3695.
[28] SOLOVYEV R，WANG Weimin，GABRUSEVA T. Weighted boxes fusion：Ensembling boxes from different object detection models[J]. Image and Vision Computing，2021，107：104-117.
[29] KALMAN R E. A new approach to linear filtering and prediction problems[J]. Journal of Fluids Engineering，1959，82(1)：35-45.

基于视觉相机和激光雷达融合的无人车障碍物检测与跟踪研究

Research on Obstacle Detection and Tracking of Autonomous Vehicles Based on the Fusion of Vision Camera and LiDAR

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 13

编辑推荐

Metrics

本文评价

[1]	高凯, 罗攀, 谢进, 胡林, 陈彬, 杜荣华. 基于数据融合的混合动力汽车速度轮廓预测[J]. 机械工程学报, 2024, 60(6): 342-353.
[2]	李尚杰, 殷国栋, 耿可可, 刘帅鹏. 基于图像和点云实例匹配的智能车目标检测和跟踪[J]. 机械工程学报, 2024, 60(22): 302-310.
[3]	林晨, 何智成, 黄怡菲, 林智桂, 付广, 黄晋. 多级参数融合网络的驾驶场景目标检测方法研究[J]. 机械工程学报, 2024, 60(10): 64-75.
[4]	毛杨坤, 段现银, 林昕, 傅盈西, 朱锟鹏. 基于目标检测的选区激光熔融成形过程熔池与飞溅监测[J]. 机械工程学报, 2023, 59(9): 335-348.
[5]	彭湃, 耿可可, 王子威, 柳智超, 殷国栋. 智能汽车环境感知方法综述[J]. 机械工程学报, 2023, 59(20): 281-303.
[6]	柳晨光, 郭珏菡, 吴勇, 初秀民, 吴文祥, 雷超凡. 无人水面艇三维激光雷达目标实时识别系统[J]. 机械工程学报, 2022, 58(4): 202-211.
[7]	刘永刚, 于丰宁, 章新杰, 陈峥, 秦大同. 基于激光点云与图像融合的3D目标检测研究[J]. 机械工程学报, 2022, 58(24): 289-299.
[8]	赵子婧, 刘宏哲, 曹东璞. 基于Libra R-CNN改进的交通标志检测算法[J]. 机械工程学报, 2021, 57(22): 255-265.
[9]	彭湃, 耿可可, 殷国栋, 庄伟超, 刘帅鹏, 徐利伟. 基于传感器融合里程计的相机与激光雷达自动重标定方法[J]. 机械工程学报, 2021, 57(20): 206-214.
[10]	薛培林, 吴愿, 殷国栋, 刘帅鹏, 林乙蘅, 黄文涵, 张云. 基于信息融合的城市自主车辆实时目标识别[J]. 机械工程学报, 2020, 56(12): 165-173.
[11]	訾斌, 尹泽强, 李永昌, 赵涛. 基于YOLO模型的柔索并联机器人移动构件快速定位方法[J]. 机械工程学报, 2019, 55(3): 64-72.
[12]	吕杰, 罗芳颖, 袁泽剑. 目标搜索与识别的视觉注意网络与学习方法[J]. 机械工程学报, 2019, 55(11): 123-130.
[13]	郭景华;胡平;李琳辉;王荣本;张明恒;郭烈. 基于遗传优化的无人车横向模糊控制[J]. , 2012, 48(6): 76-82.