Real-time Target Recognition for Urban Autonomous Vehicles Based on Information Fusion

doi:10.3901/JME.2020.12.165

Abstract

Abstract: Aiming at the problem that the single sensor has insufficient sensing dimensions and poor real-time performance, a real-time target recognition method for urban autonomous vehicles based on the fusion of lidar and camera is proposed. To achieve pixel-level matching of the two sensors, a coordinate transformation model between the two sensors is established; the yolov3-tiny algorithm is improved to increase the accuracy of target detection. Voxel grid filtering was performed on the lidar points, the ground is filtered according to the lidar point slope; the model of clustering radius and distance is established, and non-ground point clouds are clustered; the idea of envelope in images is introduced to obtain the 3D bounding box and pose information of the target; the visual target features are fused with the lidar target features. The experimental results show that the improved yolov3-tiny algorithm has a higher recognition rate for dense urban targets. The lidar algorithm can complete three-dimensional target detection and pose estimation. The fusion recognition system meets the actual driving requirements in terms of accuracy and real-time performance.

Key words: yolo, cluster, pose estimation, lidar, vision, information fusion

CLC Number:

TG156

XUE Peilin, WU Yuan, YIN Guodong, LIU Shuaipeng, LIN Yiheng, HUANG Wenhan, ZHANG Yun. Real-time Target Recognition for Urban Autonomous Vehicles Based on Information Fusion[J]. Journal of Mechanical Engineering, 2020, 56(12): 165-173.

References

[1] WU H,LIU Q,LIU X. A review on deep learning approaches to image classification and object segmentation[J]. TSP,2018,1(1):1-5.
[2] CHERIFI D,KADDARI R,HAMZA Z,et al. Infrared face recognition using neural networks and HOG-SVM[C]//2019 3rd International Conference on Bio-engineering for Smart Technologies (BioSMART). IEEE,2019:1-5.
[3] LUO J H,LIN C H. Pure FPGA implementation of an HOG based real-time pedestrian detection system[J]. Sensors,2018,18(4):1174-1189.
[4] CHENG B,WEI Y,SHI H,et al. Revisiting rcnn:On awakening the classification power of faster rcnn[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018:453-468.
[5] CHAO Y W,VIJAYANARASIMHAN S,SEYBOLD B,et al. Rethinking the faster r-cnn architecture for temporal action localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018:1130-1139.
[6] HE K,GKIOXARI G,DOLLÁR P,et al. Mask r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017:2961-2969.
[7] REDMON J,DIVVALA S,GIRSHICK R,et al. You only look once:Unified,real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016:779-788.
[8] REDMON J,FARHADI A. Yolo 9000:Better,faster,stronger[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017:7263-7271.
[9] REDMON J,FARHADI A. Yolov3:An incremental improvement[EB/OL].[2018-04-08]. Https://arxiv.org/pdf/1804.02767.pdf.
[10] LIU W,ANGUELOV D,ERHAN D,et al. Ssd:Single shot multibox detector[C]//European Conference on Computer Vision. Springer,Cham,2016(83):21-37.
[11] ASVADI A,PREMEBIDA C,PEIXOTO P,et al. 3D lidar-based static and moving obstacle detection in driving environments:An approach based on voxels and multi-region ground planes[J]. Robotics and Autonomous Systems,2016,83(S1):299-311.
[12] AZIM A,AYCARD O. Detection,classification and tracking of moving objects in a 3D environment[C]//2012 IEEE Intelligent Vehicles Symposium. IEEE,2012:802-807.
[13] 周俊静,段建民,杨光祖. 基于雷达测距的车辆识别与跟踪方法[J]. 汽车工程,2014,36(11):1415-1420 ZHOU Junjing,DUAN Jianmin,YANG Guangzu. A vehicle identification and tracking method based on radar ranging[J]. Automotive Engineering,2014,36(11):1415-1420.
[14] ASVADI A,GARROTE L,PREMEBIDA C,et al. Multimodal vehicle detection:fusing 3D-lidar and color camera data[J]. Pattern Recognition Letters,2018,115(Suppl. 1):20-29.
[15] XUE J,WANG D,DU S,et al. A vision-centered multi-sensor fusing approach to self-localization and obstacle perception for robotic cars[J]. Frontiers of Information Technology & Electronic Engineering,2017,18(1):122-138.
[16] WHITNEY D,ROSEN E,ULLMAN D,et al. Ros reality:A virtual reality framework using consumer-grade hardware for ros-enabled robots[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE,2018:1-9.
[17] LEPETIT V,MORENO-NOGUER F,FUA P. Epnp:An accurate o(n) solution to the pnp problem[J]. International Journal of Computer Vision,2009,81(2):155-166.
[18] 刘军,后士浩,张凯,等. 基于增强Tiny YOLOV3算法的车辆实时检测与跟踪[J]. 农业工程学报,2019,35(8):118-125. LIU Jun,HOU Shihao,ZHANG Kai,et al. Real-time vehicle detection and tracking based on enhanced tiny YOLOV3 algorithm[J]. Journal of Agricultural Engineering,2019,35(8):118-125.
[19] LIN T Y,GOYAL P,GIRSHICK R,et al. Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017:2980-2988.
[20] SHRIVASTAVA A,GUPTA A,GIRSHICK R. Training region-based object detectors with online hard example mining[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016:761-769.
[21] ZHANG S,LI S,HU J,et al. An iterative algorithm for optimal variable weighting in K-means clustering[J]. Communications in Statistics-Simulation and Computation,2019,48(5):1346-1365.
[22] WANG D,LU X,RINALDO A. DBSCAN:Optimal rates for density-based cluster estimation[J]. Journal of Machine Learning Research,2019,20(170):1-50.
[23] STARCZEWSKI A,CADER A. Determining the Eps parameter of the DBSCAN algorithm[C]//International Conference on Artificial Intelligence and Soft Computing. Springer,Cham,2019:420-430.
[24] WIBOWO A,SANTOSO H B,RACHMAT C A,et al. Mapping and grouping of farm land with graham scan algorithm on convex hull method[C]//2019 International Conference on Sustainable Engineering and Creative Computing (ICSECC). IEEE,2019:121-126.