Design and Experiment Research of 3D Environmental Perception System for Split-type Flying Vehicle

doi:10.3901/JME.2024.10.102

Abstract

Abstract: Flying vehicles can switch between ground-driving and air-flying modes according to road and weather conditions, which is of great significance for improving traffic efficiency, adapting to complex terrain, and reducing the cost of transportation facilities. The split-type flying vehicle is composed of three independent modules: an electric vertical take-off and landing aircraft, an intelligent control cockpit, and an autopilot chassis. It has two functions, unmanned driving on the ground and autonomous flight in the air. First of all, based on task requirements, such as automatic driving and multi-module autonomous docking guidance, the sensor configuration and deployment scheme of the 3D environmental perception system is proposed and the sensors are calibrated. Secondly, a multi-sensor fusion environment perception algorithm is designed to realize object detection, drivable area segmentation, and multi-module autonomous docking and positioning. Finally, the real-world verification is carried out on the high-performance edge computing equipment of the split-type flying vehicle. The experimental results show that the perception system can achieve an effective environmental perception effect, which provides a new solution for the automatic driving and autonomous docking guidance of split-type flying vehicles.

Key words: split-type flying vehicle, environmental perception, sensor calibration and algorithm testing

CLC Number:

TG156

LI Ying, WANG Rongxuan, WAN Chenglin, WANG Weida, YANG Chao, XU Bin. Design and Experiment Research of 3D Environmental Perception System for Split-type Flying Vehicle[J]. Journal of Mechanical Engineering, 2024, 60(10): 102-111.

References

[1] 张丹, 吴陈炜, 谢安桓. 城市交通问题的空中解决方案——自主载人飞行器研究综述[J]. 无人系统技术, 2018, 1(2):1-13. ZHANG Dan, WU Chenwei, XIE Anhuan. Aerial solution for urban traffic problems:Overview of autonomous manned aircraft[J]. Unmanned Systems Technology, 2018, 1(2):1-13.
[2] 张扬军, 钱煜平, 诸葛伟林, 等. 飞行汽车的研究发展与关键技术[J]. 汽车安全与节能学报, 2020, 11(1):1-16. ZHANG Yangjun, QIAN Yuping, ZHUGE Weilin, et al. Progress and key technologies of flying cars[J]. Journal of Automotive Safety and Energy, 2020, 11(1):1-16.
[3] 徐国艳, 牛欢, 郭宸阳, 等. 基于三维激光点云的目标识别与跟踪研究[J]. 汽车工程, 2020, 42(1):38-46. XU Guoyan, NIU Huan, GUO Chenyang, et al. Research on target recognition and tracking based on 3D laser point cloud[J]. Automotive Engineering, 2020, 42(1):38-46.
[4] LI P L, CHEN X Z, SHEN S J. Stereo R-CNN based 3D object detection for autonomous driving[C]//IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2019:7644-7652.
[5] XIE L, XIANG C, YU Z X, et al. PI-RCNN:An efficient multi-sensor 3D object detector with point-based attentive cont-conv fusion module[C]//AAAI. Proceedings of the AAAI conference on artificial intelligence. Palo Alto, CA:AAAI Press, 2020, 34(7):12460-12467.
[6] 刘永刚, 于丰宁, 章新杰, 等. 基于激光点云与图像融合的3D目标检测研究[J]. 机械工程学报, 2022, 58(24):289-299. LIU Yonggang, YU Fengning, ZHANG Xinjie, et al. Research on 3D object detection based on laser point cloud and image fusion[J]. Journal of Mechanical Engineering, 2022, 58(24):289-299.
[7] 阳仁贵, 袁运斌, 欧吉坤. 相位实时差分技术应用于飞行器交会对接研究[J]. 中国科学:物理学力学天文学, 2010, 40(5):651-657. YANG Rengui, YUAN Yunbin, OU Jikun. Real-time GNSS carrier phase differential technique for spacecraft rendezvous and docking[J]. Scientia Sinica Phys, Mech & Astron, 2010, 40(5):651-657.
[8] 王辉, 贾自凯, 金忍, 等. 无人机视觉引导对接过程中的协同目标检测[J]. 航空学报, 2022, 43(1):488-502. WANG Hui, JIA Zikai, JIN Ren, et al. Cooperative object detection in UAV-based vision-guided docking[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43(1):488-502.
[9] ZHANG Z Y. Flexible camera calibration by viewing a plane from unknown orientations[C]//IEEE. Proceedings of the Seventh IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 1999, 1:666-673.
[10] REDMON J，FARHADI A. YOLOv3:An incremental improvement[EB/OL]. [2018-04-08]. https://doi.org/10.48550/arXiv.1804.02767.
[11] LIU Z, HU J, WENG L, et al. Rotated region based CNN for ship detection[C]//IEEE. 2017 IEEE International Conference on Image Processing (ICIP). Piscataway, NJ:IEEE, 2017:900-904.
[12] YIN T W, ZHOU X Y, KRAHENBUHL P. Center-based 3D object detection and tracking[C]//IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2021:11784-11793.
[13] ZHOU Y, TUZEL O. Voxelnet:End-to-end learning for point cloud based 3D object detection[C]//IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2018:4490-4499.
[14] YAN Y, MAO Y, LI B. Second:Sparsely embedded convolutional detection[J]. Sensors, 2018, 18(10):3337.
[15] LI B，ZHANG T，XIA T. Vehicle detection from 3D lidar using fully convolutional network[EB/OL]. [2016-08-29]. https://doi.org/10.48550/arXiv.1608.07916.
[16] HOWARD A, SANDLER M, CHU G, et al. Searching for mobilenetv3[C]//IEEE. Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway, NJ:IEEE, 2019:1314-1324.
[17] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2017:4700-4708.
[18] WAN C L, WANG W D, YANG C, et al. Fast parking slot detection in the bird's eye view[C]//International Conference on Autonomous Unmanned Systems. Singapore:Springer, 2022:1183-1193.
[19] GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving? The kitti vision benchmark suite[C]//IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2012:3354-3361.
[20] GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics:The kitti dataset[J]. The International Journal of Robotics Research, 2013, 32(11):1231-1237.
[21] KINGMA D，BA J. Adam:A method for stochastic optimization[EB/OL]. [2015-06-23]. https://doi.org/10.48550/arXiv.1412.6980.
[22] LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft coco:Common objects in context[C]//Springer. Computer Vision-ECCV 2014:13th European Conference. Singapore:Springer, 2014:740-755.
[23] SIMONELLI A, BULO S, PORZI L, et al. Disentangling monocular 3D object detection[C]//IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2019:1991-1999.
[24] ZHOU X，WANG D，KRÄHENBÜHL P. Objects as points[EB/OL]. [2019-04-25]. https://doi.org/10.48550/arXiv. 1904.07850.
[25] JOCHER G，STOKEN A，BOROVEC J，et al. Ultralytics/yolov5:v3.1-Bug fixes and performance improvements[EB/OL]. [2020-08-13]. https://doi.org/10.5281/zenodo.3983579.
[26] CAESAR H, BANKITI V, LANG A H, et al. Nuscenes:A multimodal dataset for autonomous driving[C]//IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2020:11621-11631.
[27] LANG A H, VORA S, CAESAR H, et al. Pointpillars:Fast encoders for object detection from point clouds[C]//IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2019:12697-12705.
[28] BAI X, HU Z, ZHU X, et al. Transfusion:Robust lidar-camera fusion for 3d object detection with transformers[C]//IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2022:1090-1099.
[29] HUANG J, ZHANG L, SHEN Y, et al. DMPR-PS:A novel approach for parking-slot detection using directional marking-point regression[C]///IEEE. 2019 IEEE International Conference on Multimedia and Expo (ICME). Piscataway, NJ:IEEE, 2019:212-217.