关键词: 智能车, 目标检测, 目标跟踪, 传感器融合, 实例分割, 透视投影

Abstract: In the environment perception task of intelligent vehicles, in order to combine the rich semantic information in the camera image with the accurate spatial information in the lidar point clouds, a fusion detection method based on image and point clouds instance matching is proposed. To achieve the fusion detection, the instance masks of the targets in the image are predicted by the instance segmentation network, the point clouds are projected to the image plane through perspective projection transformation, the point clouds belonging to the target are extracted according to the instance mask of each target, and then the clustering algorithm is used to remove the noise, and the convex hull approximating algorithm is used to fit the 3D bounding box of the target. Based on the fusion detection method, a gate is designed to realize multi-target data association and management, and the Kalman filter is used to track the target and estimate the motion state of the target. The experimental results show that the method can effectively fuse the information from image data and point clouds data, accurately and quickly fit the position, size, and direction of the target and estimate the speed of the target, and show robustness in different experimental scenarios.

Key words: intelligent vehicle, object detection, object tracking, sensor fusion, instance segmentation, perspective projection