Multi-modal Fusion Shield Segment Recognition and Positioning Method Based on Visual Large Model
XIAO Yanqiu1,2, WEN Qing1,2, WANG Pengpeng1,2,3, CUI Guangzhen1,2, JIA Lianhui4, CHEN Zhijun1,2, JIAO Jianqiang1,2, YAO Lei1,2, CHEN Huizi1,2
1. Henan Excellent Engineer College, Zhengzhou University of Light Industry, Zhengzhou 450002; 2. Henan Collaborative Innovation Center of Intelligent Tunnel Boring Machine, Zhengzhou University of light Industry, Zhengzhou 450002; 3. State Key Laboratory of Geomechanics and Geotechnical Engineering Safety, Instituteof Rock and Soil Mechanics, Chinese Academy of Sciences., Wuhan 430064; 4. Electrical and Intelligent Technology Research Institute, China Railway Engineering Equipment Group Co. Ltd, Zhengzhou 450047
XIAO Yanqiu, WEN Qing, WANG Pengpeng, CUI Guangzhen, JIA Lianhui, CHEN Zhijun, JIAO Jianqiang, YAO Lei, CHEN Huizi. Multi-modal Fusion Shield Segment Recognition and Positioning Method Based on Visual Large Model[J]. Journal of Mechanical Engineering, 2026, 62(18): 215-225.
[1] ZHOU Y,WANG Y,DING L,et al. Utilizing IFC for shield segment assembly in underground tunneling[J]. Automation in Construction,2018,93:178-191. [2] 柳献,胡秋斌,朱雁飞,等. 盾构管片自动拼装技术研究现状与展望[J]. 机械工程学报,2025,61(19):183-201. LIU Xian,HU Qiubin,ZHU Yanfei,et al. Research status and prospect of automatic shield segment assembly technology[J]. Journal of Mechanical Engineering,2025,61(19):183-201. [3] MOSTAFA S,SOASA R L,EINSTEIN H H. Toward the automation of mechanized tunneling “exploring the use of big data analytics for ground forecast in TBM tunnels”[J]. Tunnelling and Underground Space Technology,2024,146:105643. [4] WANG X,WANG M,JIANG R,et al. Structural deformation monitoring during tunnel construction:A review[J]. Journal of Civil Structural Health Monitoring,2024,14(3):591-613. [5] JIA D,ZHANG W,LIU Y. Systematic approach for tunnel deformation monitoring with terrestrial laser scanning[J]. Remote Sensing,2021,13(17):3519. [6] BARZEGAR M,BLANKS S,SAINBURY B A,et al. MEMS technology and applications in geotechnical monitoring:A review[J]. Measurement Science and Technology,2022,33(5):052001. [7] ZENG L,SHU W,LIU Z,et al. Vision-based high-precision intelligent monitoring for shield tail clearance[J]. Automation in Construction,2022,134:104088. [8] SINGH S K,BANERJEE B P,RAVAL S. A review of laser scanning for geological and geotechnical applications in underground mining[J]. International Journal of Mining Science and Technology,2023,33(2):133-154. [9] JI C,SUN H,ZHONG R,et al. Precise positioning method of moving laser point cloud in shield tunnel based on bolt hole extraction[J]. Remote Sensing,2022,14(19):4791. [10] CHEN Y,SHEN S,ZHOU A. Extraction of assembly quality information in shield tunnels from LiDAR data[J]. Measurement,2025,245:116541. [11] JIANG F,WANG G,HE P,et al. Application of canny operator threshold adaptive segmentation algorithm combined with digital image processing in tunnel face crevice extraction[J]. The Journal of Supercomputing,2022,78(9):11601-11620. [12] LEI M,LIU L,SHI C,et al. A novel tunnel-lining crack recognition system based on digital image technology[J]. Tunnelling and Underground Space Technology,2021,108:103724. [13] DAI C,JIANG K,WANG Q. Recognition of tunnel lining cracks based on digital image processing[J]. Mathematical Problems in Engineering,2020,2020:5162583. [14] ATTARD L,DEBONO C J,VALENTINO G,et al. Tunnel inspection using photogrammetric techniques and image processing:A review[J]. ISPRS Journal Of Photogrammetry and Remote Sensing,2018,144:180-188. [15] WANG L,TANG C. Effective small crack detection based on tunnel crack characteristics and an anchor-free convolutional neural network[J]. Scientific Reports,2024,14(1):10355. [16] ZHANG C,CHEN X,LIU P,et al. Automated detection and segmentation of tunnel defects and objects using YOLOv8-CM[J]. Tunnelling and Underground Space Technology,2024,150:105857. [17] ZHAO S,SHADABFAR M,ZHANG D,et al. Deep learning-based classification and instance segmentation of leakage-area and scaling images of shield tunnel linings[J]. Structural Control and Health Monitoring,2021,28(6):e2732. [18] XUE Y,CAI X,SHADABFAR M,et al. Deep learning-based automatic recognition of water leakage area in shield tunnel lining[J]. Tunnelling and Underground Space Technology,2020,104:103524. [19] YE Z,LIN W,FARAMARZI A,et al. SAM4Tun:No-training model for tunnel lining point cloud component segmentation[J]. Tunnelling and Underground Space Technology,2025,158:106401. [20] KANG J,CHEN N,LI M,et al. A point cloud segmentation method for dim and cluttered underground tunnel scenes based on the segment anything model[J]. Remote Sensing,2023,16(1):97. [21] YANG H,WANG L,PAN Y,et al. A teacher-student framework leveraging large vision model for data pre-annotation and YOLO for tunnel lining multiple defects instance segmentation[J]. Journal of Industrial Information Integration,2025,44:100790. [22] WU H,ZHOU S,XU Z,et al. FESS-SAM:Full-element semantic segmentation of tunnel linear array images based on the segment anything model[J]. Advanced Engineering Informatics,2025,68:103609. [23] LEE C,OH J. Non-target soil deformation mapping by photogrammetry for underground tunnel model test[J]. KSCE Journal of Civil Engineering,2021,25(1):291-302. [24] CHEN Y,XU B,JIANG Y,et al. Lightweight instance segmentation for rapid leakage detection in shield tunnel linings under extreme low-light conditions[J]. Automation in Construction,2025,177:106368. [25] QIAN H,WANG M,ZHU M,et al. A review of multi-sensor fusion in autonomous driving[J]. Sensors,2025,25(19):6033. [26] 滕洪钊,邓朝晖,吕黎曙,等.多传感器信息融合的加工过程状态监测研究[J]. 机械工程学报,2022,58(6):26-41. TENG Hongzhao,DENG Zhaohui,LÜ Lishu,et al. Research on process state monitoring based on multi-sensor information fusion[J]. Journal of Mechanical Engineering,2022,58(6):26-41. [27] CHEN X,WANG L,CAI J,et al. Autonomous recognition andpositioning of shield segments based on red,green,blue and depth information[J]. Automation in Construction,2023,146:104687. [28] WANG X,YANG M,ZHENG S,et al. Bolt detection and positioning system based on YOLOv5s-T and RGB-D camera[J]. Transactions of Beijing Institute of Technology,2022,42(11):1159-1166. [29] MA Z,LIU S. A review of 3D reconstruction techniques in civil engineering and their applications[J]. Advanced Engineering Informatics,2018,37:163-174.