• CN:11-2187/TH
  • ISSN:0577-6686

机械工程学报 ›› 2026, Vol. 62 ›› Issue (18): 215-225.doi: 10.3901/JME.260740

• 特邀专栏:大模型驱动的高端装备智能运维 • 上一篇    

扫码分享

基于视觉大模型的多模态融合盾构管片识别定位方法

肖艳秋1,2, 文晴1,2, 王鹏鹏1,2,3, 崔光珍1,2, 贾连辉4, 陈志君1,2, 焦建强1,2, 姚雷1,2, 陈汇资1,2   

  1. 1. 郑州轻工业大学河南省卓越工程师学院 郑州 450002;
    2. 郑州轻工业大学智能隧道掘进装备河南省协同创新中心 郑州 450002;
    3. 岩土力学与工程安全全国重点实验室 武汉 430064;
    4. 中铁工程装备集团有限公司电气与智能技术研究院 郑州 450047
  • 收稿日期:2025-12-23 修回日期:2026-06-18 发布日期:2026-11-20
  • 作者简介:肖艳秋,男,1980年出生,博士,教授,博士研究生导师。主要研究方向为复杂装备数字化设计与制造、机电系统动力学分析与控制。E-mail:xiaoyanqiu@zzuli.edu.cn;王鹏鹏(通信作者),男,1989年出生,博士,讲师。主要研究方向为图像测量、机电液控制。E-mail:wpeng121@126.com
  • 基金资助:
    国家自然科学基金(52375034)、河南省高校重点科研(24A460027)、岩土力学与工程安全全国重点实验室开放基金(SKLGME022005)、河南省高等学校重点科研项目计划基础研究专项(23ZX013)、河南省高等学校重点科研和盾构机管片拼装机器人智能化调控系统关键技术研究(25CY041)资助项目。

Multi-modal Fusion Shield Segment Recognition and Positioning Method Based on Visual Large Model

XIAO Yanqiu1,2, WEN Qing1,2, WANG Pengpeng1,2,3, CUI Guangzhen1,2, JIA Lianhui4, CHEN Zhijun1,2, JIAO Jianqiang1,2, YAO Lei1,2, CHEN Huizi1,2   

  1. 1. Henan Excellent Engineer College, Zhengzhou University of Light Industry, Zhengzhou 450002;
    2. Henan Collaborative Innovation Center of Intelligent Tunnel Boring Machine, Zhengzhou University of light Industry, Zhengzhou 450002;
    3. State Key Laboratory of Geomechanics and Geotechnical Engineering Safety, Instituteof Rock and Soil Mechanics, Chinese Academy of Sciences., Wuhan 430064;
    4. Electrical and Intelligent Technology Research Institute, China Railway Engineering Equipment Group Co. Ltd, Zhengzhou 450047
  • Received:2025-12-23 Revised:2026-06-18 Published:2026-11-20

摘要: 针对盾构隧道管片自动化拼装中,因环境复杂、目标特征弱导致的管片抓取位姿难以精确测量的问题,提出一种基于视觉大模型的多模态融合盾构管片空间位姿解算方法。该方法首先构建基于边缘引导与动态权重调度的YOLO-SAM模型,对复杂背景下管片进行检测分割;然后将其映射至三维点云后,通过融合主成分分析(Principal component analysis, PCA)与迭代最近点(Iterative closest point, ICP)的点云配准融合的跨模态特征提取方法,实现管片螺栓几何特征的高精度提取;最后,结合管片螺栓与定位孔的多特征融合空间向量约束关系,解算管片空间位姿,从而解决单一螺栓旋转对称导致的位姿解算难题。将本文提出的算法应用于全尺寸6米级管片拼装实验平台上,其验证结果表明,所提分割方法的交并比(Intersection over union, IoU)达96.20%,测量位置误差≤2 mm,姿态误差≤2°,该测量精度能够满足盾构管片拼装的工程精度要求,为后续的管片拼装自动识别搭建了完整的技术框架。

关键词: 盾构隧道, YOLO-SAM模型, 多模态融合, 位姿测量

Abstract: In the automatic assembly of shield tunnel segments, it is difficult to accurately measure the grasping pose of segments due to the complex environment and weak target characteristics. To solve this problem, a multi-modal fusion shield segment spatial pose calculation method based on visual large model is proposed. Firstly, the YOLO-SAM model based on edge guidance and dynamic weight scheduling is constructed to detect and segment the segment under complex background, and then it is mapped to the three-dimensional point cloud. Through the cross-modal feature extraction method of point cloud registration fusion of principal component analysis (PCA) and iterative closest point (ICP), the high-precision extraction of geometric features of segment bolts is realized. Finally, combined with the multi-feature fusion space vector constraint relationship between the segment bolt and the positioning hole, the spatial pose of the segment is solved, so as to solve the pose calculation problem caused by the rotational symmetry of a single bolt. The algorithm proposed in this paper is applied to the full-scale 6-meter segment assembly experimental platform. The verification results show that the intersection over union (IoU) of the proposed segmentation method is 96.20%, the measurement position error is less than or equal to 2 mm, and the attitude error is less than or equal to 2 °. The measurement accuracy can meet the engineering accuracy requirements of shield segment assembly, and a complete technical framework for subsequent automatic segment assembly recognition is built.

Key words: shield tunnel, YOLO-SAM model, multimodal fusion, pose measurement

中图分类号: