3D LiDAR Based Real-time Object Recognition System for Unmanned Surface Vehicles

doi:10.3901/JME.2022.04.202

Abstract

Abstract: In order to solve the problem of unmanned surface vehicles (USVs) dynamic environment target dynamic perception, a 3D (three dimension) LiDAR based real-time object recognition system is studied. The structure, hardware components and data communication protocol of the 3D LiDAR based real-time object recognition system are designed. Based on the point cloud library (PCL) library, Qt and Visual Studio platform, the system software is developed, which realizes point cloud data correction, real-time processing, data display, status output, and remote communication and other functions. Considering the characteristics of the 3D point cloud in the surrounding environment of a sailing USV, the 3D laser point cloud is projected to a multi-attribute two-dimensional grid for representation, and the eight-neighbor algorithm is used to realize the clustering of the obstacle grid a. Key technologies such as point cloud data processing, target segmentation, and remote interaction of point cloud images are solved. Finally, the platform of 3D LiDAR based real-time object recognition system for a USV in an outdoor pool environment is constructed, and the results show that the system can reliably and correctly identify the obstacles within a range of 100 m around the USV.

Key words: unmanned surface vehicle, LiDAR, 3D point cloud, target recognition, grid map, eight-neighborhood

CLC Number:

U675

LIU Chenguang, GUO Juehan, WU Yong, CHU Xiumin, WU Wenxiang, LEI Chaofan. 3D LiDAR Based Real-time Object Recognition System for Unmanned Surface Vehicles[J]. Journal of Mechanical Engineering, 2022, 58(4): 202-211.

References

[1] 柳晨光,初秀民,吴青,等. USV发展现状及展望[J].中国造船,2014,55(4):194-205.LIU Chenguang,CHU Xiumin,WU Qing,et al. A review and prospect of USV research[J]. Ship Building of China,2014,55(4):194-205.
[2] 庄佳园,徐玉如,万磊,等.基于雷达图像的水面无人艇目标检测技术[J].哈尔滨工程大学学报,2012,33(2):129-135.ZHUANG Jiayuan,XU Yuru,WAN Lei,et al. Target detection of an unmanned surface vehicle based on a radar image[J]. Journal of Harbin Engineering University,2012,33(2):129-135.
[3] 詹华蕊,姜晓荣.复杂背景下无人艇视频视觉目标图像识别算法[J].舰船科学技术,2020,42(8):79-81.ZHAN Huarui,JIANG Xiaorong. Image recognition algorithm of unmanned aerial vehicle video under complex background[J]. Ship Science and Technology,2020,42(8):79-81.
[4] 胡延平,刘菲,魏振亚,等.基于毫米波雷达与视觉信息融合的车辆跟踪[J/OL].中国机械工程.[2021-06-29]. https://kns.cnki.net/kcms/detail/42.1294.TH.20201112. 1014.004.html. HU Yanping,LIU Fei,WEI Zhenya,et al. Vehicle tracking method based on radar and vision information fusion[J/OL]. Chinese Mechanical Engineering.[2021-06-29]. https://kns.cnki.net/kcms/detail/42.1294. TH.20201112.1014.004.html.
[5] 李小毛,张鑫,王文涛,等.基于3D激光雷达的无人水面艇海上目标检测[J].上海大学学报,2017,23(1):27-36.LI Xiaomao,ZHANG Xin,WANG Wentao,et al. 3D LiDAR-based marine object detection for USV[J]. Journal of Shanghai University,2017,23(1):27-36.
[6] STANISLAS L,DUNBABIN M. Multimodal sensor fusion for robust obstacle detection and classification in the maritime RobotX challenge[J]. IEEE Journal of Oceanic Engineering,2018,44(2):343-351.
[7] MH&L Staff. Maersk first to test lidar on container ship[EB/OL].[2021-06-29]. https://www.mhlnews.com/technology-automation/article/22054981/maersk-first-to-test-lidar-on-container-ship.
[8] 朱飞祥,王少博,邢胜伟,等.基于三维激光雷达的沿海泊位码头识别算法[J].中国航海,2020:75-78,96.ZHU Feixiang,WANG Shaobo,XING Shengwei,et al. Wharf identification algorithm based on 3D LiDAR[J]. Navigation of China,2020:75-78,96.
[9] 陈晓冬,张佳琛,庞伟凇,等.智能驾驶车载激光雷达关键技术与应用算法[J].光电工程,2019,46(7):34-46.CHEN Xiaodong,ZHAGN Jiachen,PANG Weisong,et al. Key technology and application algorithm of intelligent driving vehicle LiDAR[J]. Opto-Electronic Engineering,2019,46(7):34-46.
[10] HRABAR S. An evaluation of stereo and laser-based range sensing for rotorcraft unmanned aerial vehicle obstacle avoidance[J]. Journal of Field Robotics,2012,29(2):215-239.
[11] 李捷,袁夏,赵春霞,等.基于2维激光雷达的小型地面移动机器人自主回收方法[J].机器人,2017,39(5):688-696.LI Jie,YUAN Xia,ZHAO Chunxia,et al. Automatic withdrawal method based on 2D laser radar for small ground mobile robot[J]. Robot,2017,39(5):688-696.
[12] 赵玉梁.基于三维激光雷达的无人船目标检测[D].哈尔滨:哈尔滨工程大学,2018.ZHAO Yuliang. Unmanned surface vehicle target detection based on 3D Lidar[D]. Harbin:Harbin University of Science and Technology,2018.
[13] VILLA J,AALTONEN J,KOSKINEN K T. Path-following with lidar-based obstacle avoidance of an unmanned surface vehicle in harbor conditions[J]. IEEE/ASME Transactions on Mechatronics,2020,25(4):1812-1820.
[14] 郭珏菡,柳晨光,于周讯,等.智能船舶激光雷达感知技术综述与展望[C]//第十五届中国智能交通年会科技论文集.深圳:中国智能交通协会,2020:162-173.GUO Juehan,LIU Chenguang,YU Zhouxun,et al. Application and outlook of ship perception based on LiDAR[C]//Proceedings of the 15 ITS China Congress. Shenzhen:China Intelligent Transportation Systems Association,2020:162-173.
[15] 刘德庆,张杰,金久才.基于三维激光雷达的无人船障碍物自适应栅格表达方法[J].中国激光,2020,47(1):273-278.LIU Deqing,ZHANG Jie,JIN Jiucai. Adaptive grid representation method for unmanned surface vehicle obstacle based on three-dimensional LiDAR[J]. Chinese Journal of Lasers,2020,47(1):273-278.
[16] 柳晨光,初秀民,毛庆洲,等.无人船自适应路径跟踪控制系统[J].机械工程学报,2020,56(8):216-227.LIU Chenguang,CHU Xiumin,MA Qingzhou,et al. Adaptive path following control system for unmanned surface vehicles[J]. Journal of Mechanical Engineering,2020,56(8):216-227.
[17] 王贵槐,谢朔,柳晨光,等.基于激光雷达的内河无人船障碍物识别方法[J].光学技术,2018,44(5):602-608.WANG Guihuai,XIE Shuo,LIU Chenguang,et al. Obstacle identification method based on laser radar for inland unmanned vessel[J]. Optical Technique,2018,44(5):602-608.
[18] MIĄDLICKI K,PAJOR M,SAKÓW M. Ground plane estimation from sparse LIDAR data for loader crane sensor fusion system[C]//Proceedings of 2017 International Conference on Methods and Models in Automation and Robotics. Miedzyzdroje:IEEE,2017:717-722.
[19] 庄加兴,焦侬,殷非.毫米波雷达与激光雷达在无人船上的应用[J].船舶工程,2019,41(11):79-82,119.ZHUANG Jiaxing,JIAO Nong,YIN Fei. Application of MMW Radar and LiDAR in MASS[J]. Ship Engineering,2019,41(11):79-82,119.
[20] QI C R,LIU Wei,WU Chenxia,et al. Frustum PointNets for 3D object detection from RGB-D data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Utah:IEEE,2018:918-927.
[21] 胡晋山,康建荣,张琪,等.一种八邻域图像边界追踪改进算法[J].测绘通报,2018(12):21-25.HU Jinshan,KANG Jianrong,ZHANG Qi,et al. An improving image boundary tracking algorithm based on eight neighborhood[J]. Bulletin of Surveying and Mapping,2018(12):21-25.
[22] 张银,任国全,程子阳,等.三维激光雷达在无人车环境感知中的应用研究[J].激光与光电子学进展,2019,56(13):9-19.ZHANG Yin,REN Guoquan,CHENG Ziyang,et al. Application research of three-dimensional LiDAR in unmanned vehicle environment perception[J]. Laser&Optoelectronics Progress,2019,56(13):9-19.
[23] KAUR P,LAMBA I M S,GOSAIN A. A robust method for image segmentation of noisy digital images[C]//Proceedings of 2011 IEEE International Conference on Fuzzy Systems. Taipei,China:IEEE,2011:1656-1663.
[24] THOMPSON D,COYLE E,BROWN J. Efficient LiDAR-based object segmentation and mapping for maritime environments[J]. IEEE Journal of Oceanic Engineering,2019,44(2):352-362.
[25] 汪汉云.高分辨率三维点云目标识别技术研究[D].长沙:国防科技大学,2015.WANG Hanyun. Research on object detection for high resolution 3D point cloud[D]. Changsha:National University of Defense Technology,2015.
[26] 董明晓,郑康平.一种点云数据噪声点的随机滤波处理方法[J].中国图象图形学报,2004,9(2):245-248.DONG Mingxiao,ZHENG Kangping. A random filter algorithm for reducing noise error of point cloud data[J]. Journal of Image and Graphics,2004,9(2):245-248.