Research on Obstacle Detection and Tracking of Autonomous Vehicles Based on the Fusion of Vision Camera and LiDAR

doi:10.3901/JME.2025.02.296

Abstract

Abstract: To improve the accuracy and stability of obstacle detection and tracking, depthwise separable atrous spatial pyramid pooling(DASPP) layer and weighted boxes fusion(WBF) algorithm are firstly introduced into you only look once version 5(YOLO v5) to tackle the problems of loss of semantic information and candidate box information, respectively. Then, a two-stage point cloud clustering method considering the point cloud distance and the continuity of the outer contour is proposed and a bounding box is established to improve the clustering accuracy of each target while ensuring the recall rate of obstacle targets. Finally, the convolutional block attention module(CBAM) is added into MobileNet to effectively extract the visual features of the obstacle target, visual features and 3D information are combined to establish correlation metrics and thus to improve tracking precision. Tests based on KITTI dataset and real environments show the effectiveness and transferability of the proposed algorithm.

Key words: vision camera, LiDAR, object detection, multi-object tracking, autonomous vehicle

CLC Number:

U463

WEI Chao, WU Xitao, ZHU Gengting, SHU Yongjie, LI Luxing, SUI Shuxin. Research on Obstacle Detection and Tracking of Autonomous Vehicles Based on the Fusion of Vision Camera and LiDAR[J]. Journal of Mechanical Engineering, 2025, 61(2): 296-309,320.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: http://www.cjmenet.com.cn/EN/10.3901/JME.2025.02.296

http://www.cjmenet.com.cn/EN/Y2025/V61/I2/296

References

[1] BADUE C，GUIDOLINI R，CARNEIRO R V，et al. Self-driving cars：A survey[J]. Expert Systems with Application，2021，165：1-27.
[2] EVERINGHAM M，GOOL L V，WILLIAMS C K I，et al. The pascal visual object classes (voc) challenge[J]. International Journal of Computer Vision，2010，88(2)：303-338.
[3] RUSSAKOVSKY O，DENG Jia，SU Hao，et al. Imagenet large scale visual recognition challenge[J]. International Journal of Computer Vision，2015，115(3)：211-252.
[4] HINTON G E，SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science，2006，313(5786)：504-507.
[5] LECUN Y，BENGIO Y，HINTON G，et al. Deep learning[J]. Nature，2015：436-444.
[6] GIRSHICK R，DONAHUE J，DARRELL T，et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition，Columbus，New York：IEEE，2014：580-587.
[7] HE Kaiming，ZHANG Xiangyu，REN Shaoqing，et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37(9)：1904-1916.
[8] GIRSHICK R. Fast R-CNN[C]// Proceedings of the IEEE International Conference on Computer Vision，Santiago，Chile：IEEE，2015：1440-1448.
[9] REN Shaoqing，HE Kaiming，GIRSHICK R，et al. Faster r-cnn：Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39(6)：1137-1149.
[10] REDMON J，DIVVALA S，GIRSHICK R，et al. You only look once：Unified，real-time object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，NV，New York：IEEE，2016：779-788.
[11] REDMON J，FARHADI A. Yolo9000：Better，faster，stronger[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，New York： IEEE，2017：6517-6525.
[12] REDMON J，FARHADI A. Yolov3：An incremental improvement[J]. arXiv preprint，2018，arXiv:1804.02767.
[13] LIU Wei，ANGUELOV D，ERHAN D，et al. Ssd：single shot multibox detector[C]// European Conference on Computer Vision，Amsterdam，Netherlands：Springer International Publishing，2016：21-37.
[14] LIN T Y，GOYAL P，GIRSHICK R，et al. Focal loss for dense object detection[C]// IEEE International Conference on Computer Vision，Venice，Italy：IEEE，2017：2999-3007.
[15] GUHA S，RASTOGI R，SHIM K. Cure：An efficient clustering algorithm for large databases[J]. Information Systems，1998，26(1)：35-58.
[16] MACQUEEN J B. Some methods for classification and analysis of multivariate observations[C]// Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability，Berkeley，CA：University of California Press，1967：281-297.
[17] AGRAWAL R，FALOUTSOS C，SWAMI A N. Efficient similarity search in sequence databases[C]// International Conference on Foundations of Data Organization and Algorithms：4th International Conference，Chicago，Illinois，USA：Springer Berlin Heidelberg，1993：69-84.
[18] ESTER M，KRIEGEL H P，SANDER J，et al. A density-based algorithm for discovering clusters in large spatial databases with noise[C]// Proceedings of 2nd International Conference on knowledge Discovery and Data Mining，Portland，Oregon，USA，1996：226-231.
[19] AVIDAN S. Support vector tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2004，26(8)：1064-1072.
[20] XIANG Yu，ALAHI A，SAVARESE S. Learning to track：Online multi-object tracking by decision making[C]// IEEE International Conference on Computer Vision，Santiago，Chile：IEEE，2015：4705-4713.
[21] MILAN A，REZATOFIGHI H，DICK A，et al. Online multi-target tracking using recurrent neural networks[C]// Proceedings of the AAAI Conference on Artificial Intelligence，San Francisco，California，USA：AAAI，2017：4225-4232.
[22] BEWLEY A，GE Z，OTT L，et al. Simple online and realtime tracking[C]// 2016 IEEE International Conference on Image Processing，Phoenix，Arizona，New York：IEEE，2016：3464-3468.
[23] WOJKE N，BEWLEY A，PAULUS D. Simple online and realtime tracking with a deep association metric[C]// 2017 IEEE International Conference on Image Processing，Beijing，China：IEEE，2017：3645-3649.
[24] SON J，BAEK M，CHO M，et al. Multi-object tracking with quadruplet convolutional neural networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，New York：IEEE，2017：3786-3795.
[25] SCHEIDEGGER S，BENJAMINSSON J，ROSENBERG E，et al. Mono-camera 3d multi-object tracking using deep learning detections and pmbm filtering[C]// 2018 IEEE Intelligent Vehicles Symposium，Suzhou，China：IEEE，2018：433-440.
[26] YU F，KOLTUN V. Multi-scale context aggregation by dilated convolutions[C]// 2016 International Conference on Learning Representations，San Juan，Puerto Rico，2016：1678-1693.
[27] HOWARD A G，ZHU Menglong，CHEN Bo，et al. Mobilenets：Efficient convolutional neural networks for mobile vision applications[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，New York：IEEE，2017：3666-3695.
[28] SOLOVYEV R，WANG Weimin，GABRUSEVA T. Weighted boxes fusion：Ensembling boxes from different object detection models[J]. Image and Vision Computing，2021，107：104-117.
[29] KALMAN R E. A new approach to linear filtering and prediction problems[J]. Journal of Fluids Engineering，1959，82(1)：35-45.