基于三维点云深度学习的飞机表面多圆孔基元提取方法

doi:10.3901/JME.2022.14.190

摘要/Abstract

摘要： 在飞机部件自动制孔系统中，快速、精确检测飞机表面圆孔对飞机装配质量具有重要作用，但从大规模三维测量点云数据中自动化、精确、快速检测所有圆孔特征依旧是一个难点。鉴于此，提出一种基于三维点云深度学习的飞机表面多圆孔基元提取方法。使用三维点云深度学习网络预测三维测量点云中初始圆孔边界点，并基于初始圆孔边界点，学习圆孔法向。同时，设计基于学习的加权最小二乘(Weighted least square，WLS)方法拟合圆孔参数，并将圆孔边界点检测误差、圆孔参数估计误差、圆孔法向学习误差作为多任务联合损失进行网络训练。通过在不同噪声、不同分辨率的模拟点云数据和实测点云数据上进行测试，并与现有先进边界提取、圆孔拟合方法进行对比。试验结果表明，所提出的方法在边界点识别准确度、圆孔参数计算准确度等方面获得了优越的综合性能。

关键词: 飞机表面圆孔检测, 三维测量, 深度学习, 加权最小二乘

Abstract: In the aircraft automatic hole manufacturing system, rapidly and accurately detecting the circle holes on the surface of the aircraft is important for the assembly quality of the aircraft. However, it is still difficult to detect them accurately and quickly from the measured point clouds. A method for extracting multiple circle primitives based on 3D point cloud deep learning is proposed. Specifically, a 3D point cloud network is presented to predict the initial circle boundary points, based on which, the circles' normals are computed. Learning-based weighted least squares (WLS) is then designed to estimate the circle parameters. Finally, the circle boundary point classification, circle parameter estimation and circle normal computation are co-trained with a multi-task loss to enhance the quality of circle extraction. Comparisons on the simulated point clouds and real-scanned point clouds of different noise intensities and resolutions exhibit clear improvements in terms of noise-robustness and extraction accuracy.

Key words: aircraft surface hole detection, 3D measurement, deep learning, weighted least squares

中图分类号:

TH741

陈红华, 魏泽勇, 谢乾, 魏明强, 汪俊. 基于三维点云深度学习的飞机表面多圆孔基元提取方法[J]. 机械工程学报, 2022, 58(14): 190-202.

CHEN Honghua, WEI Zeyong, XIE Qian, WEI Mingqiang, WANG Jun. Method for Extracting Multiple Circle Primitives Extraction of Aircraft Surface Based on 3D Point Cloud Deep Learning[J]. Journal of Mechanical Engineering, 2022, 58(14): 190-202.

参考文献

[1] XIE Q, LU D, DU K, et al. Aircraft skin rivet detection based on 3D point cloud via multiple structures fitting[J]. Computer-Aided Design, 2019, 120:102805.
[2] LI B, WANG X, YANG H, et al. Aircraft rivets defect recognition method based on magneto-optical images[C]//International Conference on Machine Vision and Human-machine Interface, April 24-25, 2010, Kaifeng, China:IEEE, 2010:788-791.
[3] 陆岩.基于微分边缘检测的自动化制孔基准孔快速识别定位技术[D].杭州:浙江大学, 2017.CHEN Yan. Rapid identification and location technology of reference hole in automatic drilling based on differential edge detection[D]. Hangzhou:Zhejiang University, 2017.
[4] 陈璐,关立文.基于改进Zernike矩的亚像素钻铆圆孔检测方法[J].清华大学学报, 2019, 59(6):438-444. CHEN Lu, GUAN Liwen. Subpixel drilling and riveting circular hole detection method based on an improved Zernike moment[J]. Journal of Tsinghua University, 2019, 59(6):438-444.
[5] 杨爽.基于深度学习的复合材料圆孔检测方法研究[D].杭州:浙江大学, 2019. YANG Shuang. Deep learning based approach for circular hole detection on composite parts[D]. Hangzhou:Zhejiang University, 2019.
[6] 谭小群,唐婧仪,于薇薇,等.基于线激光扫描和图像处理的基准孔检测技术研究[J].现代制造工程, 2019(4):115-121. TAN Xiaoqun, TANG Jingyi, YU Weiwei, et al. Research on reference hole detection technology based on line laser scanning and image processing[J]. Modern Manufacturing Engineering, 2019(4):115-121.
[7] 庄志炜,田威,李波,等.基于模板匹配的孔位与法矢检测算法[J].计算机集成制造系统, 2021, 27(12):3484-3493. ZHUANG Zhiwei, TIAN Wei, LI Bo, et al. Detection algorithm of hole position and normal based on template matching[J]. Computer Integrated Manufacturing Systems, 2021, 27(12):3484-3493.
[8] 石循磊,杜坤鹏,张继文,等.基于线激光扫描的飞机表面锪窝孔参数提取方法[J].机械工程学报, 2020, 56(8):166-172. SHI Xunlei, DU Kunpeng, ZHANG Jiwen, et al. Method for extracting hole parameters of aircraft surface based on linear laser scanning[J]. Journal of Mechanical Engineering, 2020, 56(8):166-172.
[9] BARBER C B, DOBKIN D P, HUHDANPAA H. The quickhull algorithm for convex hulls[J]. ACM Transactions on Mathematical Software (TOMS), 1996, 22(4):469-483.
[10] AKKIRAJU N, EDELSBRUNNER H, FACELLO M, et al. Alpha shapes:Definition and software[C]//Proceedings of the 1st International Computational Geometry Software Workshop. 1995, 63:66.
[11] KETTNER L, NAHER S, GOODMAN J E, et al. Two computational geometry libraries:LEDA and CGAL[C]//Handbook of Discrete and Computational Geometry. Florida:Chapman&Hall/CRC, 2004:1435-1463.
[12] OZTIRELI A C, GUENNEBAUD G, GROSS M. Feature preserving point set surfaces based on non-linear kernel regression[J]. Computer Graphics Forum, 2009, 28(2):493-501.
[13] BAZAZIAN D, CASAS J R, RUIZ-HIDALGO J. Fast and robust edge extraction in unorganized point clouds[C]//International Conference on Digital Image Computing:Techniques and Applications (DICTA), January 07, 2015, Adelaide, Australia:IEEE, 2015:1-8.
[14] MINEO C, PIERCE S G, SUMMAN R. Novel algorithms for 3D surface point cloud boundary detection and edge reconstruction[J]. Journal of Computational Design and Engineering, 2019, 6(1):81-91.
[15] RUSU R B, COUSINS S. 3d is here:Point cloud library (pcl)[C]//2011 IEEE International Conference on Robotics and Automation, May 09, 2011, Shanghai, China:IEEE, 2011:1-4.
[16] GERSHO A, GRAY R M. Vector quantization and signal compression[M]. Berlin/Heidelberg:Springer Science&Business Media, 2012.
[17] BELTON D, LICHTI D D. Classification and segmentation of terrestrial laser scanner point clouds using local variance information[J]. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci, 2006, 36(5):44-49.
[18] YU L, LI X, FU C W, et al. EC-Net:An edge-aware point set consolidation network[C]//Proceedings of the European Conference on Computer Vision (ECCV), September 8-14, 2018, Munich, Germany:Springer, 2018:386-402.
[19] WANG X, XU Y, XU K, et al. Pie-net:Parametric inference of point cloud edges[J]. Advances in Neural Information Processing Systems, 2020, 33:20167-20178.
[20] LOIZOU M, AVERKIOU M, KALOGERAKIS E. Learning part boundaries from 3D point clouds[J]. Computer Graphics Forum., 2020, 39(5):183-195.
[21] CHERNOV N. Circular and linear regression:Fitting circles and lines by least squares[M]. Florida:CRC Press, 2010.
[22] CALAFIORE G. Approximation of n-dimensional data using spherical and ellipsoidal primitives[J]. IEEE Transactions on Systems, Man, and Cybernetics-Part A:Systems and Humans, 2002, 32(2):269-278.
[23] GUEVARA I L D, MUOZ J, COZAR O D D, et al. Robust fitting of circle arcs[J]. Journal of Mathematical Imaging and Vision, 2011, 40(2):147-161.
[24] ABDUL-RAHMAN H, CHERNOV N. Fast and numerically stable circle fit[J]. Journal of Mathematical Imaging and Vision, 2014, 49(2):289-295.
[25] CHERNOV N, LESORT C. Least squares fitting of circles[J]. Journal of Mathematical Imaging and Vision, 2005, 23(3):239-252.
[26] AL-SHARADQAH A. Further statistical analysis of circle fitting[J]. Electronic Journal of Statistics, 2014, 8(2):2741-2778.
[27] KASA I. A circle fitting procedure and its error analysis[J]. IEEE Transactions on Instrumentation and Measurement, 1976(1):8-14.
[28] AL-SHARADQAH A, CHERNOV N. Error analysis for circle fitting algorithms[J]. Electronic Journal of Statistics, 2009, 3:886-911.
[29] PRATT V. Direct least-squares fitting of algebraic surfaces[J]. ACM SIGGRAPH Computer Graphics, 1987, 21(4):145-152.
[30] TAUBIN G. Estimation of planar curves, surfaces, and nonplanar space curves defined by implicit equations with applications to edge and range image segmentation[J]. IEEE Computer Architecture Letters, 1991, 13(11):1115-1138.
[31] WANG H, SUTER D. Using symmetry in robust model fitting[J]. Pattern Recognition Letters, 2003, 24(16):2953-2966.
[32] NURUNNABI A, WEST G, BELTON D. Robust locally weighted regression techniques for ground surface points filtering in mobile laser scanning three dimensional point cloud data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 54(4):2181-2193.
[33] DUDA R O, HART P E. Use of the Hough transformation to detect lines and curves in pictures[J]. Communications of the ACM, 1972, 15(1):11-15.
[34] LUND U J. Monte Carlo maximum likelihood circle fitting using circular density functions[J]. Computational Statistics, 2013, 28(2):393-411.
[35] FROSIO I, BORGHESE N A. Real-time accurate circle fitting with occlusions[J]. Pattern Recognition, 2008, 41(3):1041-1055.
[36] DE MARCO T, CAZZATO D, LEO M, et al. Randomized circle detection with isophotes curvature analysis[J]. Pattern Recognition, 2015, 48(2):411-421.
[37] FISCHLER M A, BOLLES R C. Random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6):381-395.
[38] DORST L. Total least squares fitting of k-spheres in n-D Euclidean space using an (n+2)-D isometric representation[J]. Journal of Mathematical Imaging and Vision, 2014, 50(3):214-234.
[39] NURUNNABI A, SADAHIRO Y, LAEFER D F. Robust statistical approaches for circle fitting in laser scanning three-dimensional point cloud data[J]. Pattern Recognition, 2018, 81:417-431.
[40] ROUSSEEUW P J, LEROY A M. Robust regression and outlier detection[M]. New Jersey:John Wiley&Sons, 2005.
[41] CLEVELAND W S. Robust locally weighted regression and smoothing scatterplots[J]. Journal of the American Statistical Association, 1979, 74(368):829-836.
[42] TORR P H S, ZISSERMAN A. MLESAC:A new robust estimator with application to estimating image geometry[J]. Computer Vision and Image Understanding, 2000, 78(1):138-156.
[43] KANATANI K, SUGAYA Y, KANAZAWA Y. Guide to 3D Vision Computation[M]. Berlin/Heidelberg:Springer, 2016.
[44] AHN S J, RUH W, WARNECKE H J. Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola[J]. Pattern Recognition, 2001, 34(12):2283-2303.
[45] KANATANI K, RANGARAJAN P. Hyper least squares fitting of circles and ellipses[J]. Computational Statistics&Data Analysis, 2011, 55(6):2197-2208.
[46] COOPE I D. Circle fitting by linear and nonlinear least squares[J]. Journal of Optimization Theory and Applications, 1993, 76(2):381-388.
[47] LI L, SUNG M, DUBROVINA A, et al. Supervised fitting of geometric primitives to 3d point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 15-20, 2019, Long Beach, CA, USA:IEEE, 2019:2652-2660.
[48] SHARMA G, LIU D, MAJI S, et al. Parsenet:A parametric surface fitting network for 3d point clouds[C]//Proceedings of the European Conference on Computer Vision (ECCV), August 23-28, 2020, Glasgow, UK:Springer, 2020:261-276.
[49] NURUNNABI A, WEST G, BELTON D. Outlier detection and robust normal-curvature estimation in mobile laser scanning 3D point cloud data[J]. Pattern Recognition, 2015, 48(4):1404-1419.
[50] WANG Y, SUN Y, LIU Z, et al. Dynamic graph cnn for learning on point clouds[J]. ACM Transactions on Graphics (TOG), 2019, 38(5):1-12.
[51] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words:Transformers for image recognition at scale[J]. ArXiv Preprint ArXiv:2010.11929, 2020.
[52] QI C R, SU H, MO K, et al. Pointnet:Deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, November 09, 2017, Honolulu, HI, USA:IEEE, 2017:652-660.