Trajectory Prediction of Bevel-Tip Flexible Needle Based on Force and Vision Perception

doi:10.3901/JME.2021.11.052

Abstract

Abstract: It is a challenge to precisely control the flexible needle to reach the target in clinical applications. The force acting on the flexible needle can cause tissue deformation and needle bending, resulting in misalignment of the needle tip. The interaction between the needle and soft tissue involves a large number of biophysical characteristics, and these parameters cannot be estimated through physical modeling or directly. In order to solve this problem, a method to predict the needle trajectory is proposed. The force analysis for the flexible needle is carried out and the corresponding mechanical model is established; on the basis of the mechanical model, a force-vision perception prediction model based on BP neural network is established and needle tip trajectory is predicted. Three different types of flexible needles are tested, data are collected to train the model. Finally, the needle tip trajectory was obtained through experiments, which are compared with the model prediction. The results show that the displacements in x and y directions predicted by the model can accord with the experiments and model error is within 2mm, which can predict the insertion trajectory more accurately.

Key words: needle insertion, BP neural network, soft tissue, image processing

CLC Number:

TH789

WANG Linze, GAO Dedong, BAI Huiquan, ZHAO Yan, CUI Jiali, LI Murong, LEI Yong. Trajectory Prediction of Bevel-Tip Flexible Needle Based on Force and Vision Perception[J]. Journal of Mechanical Engineering, 2021, 57(11): 52-60.

References

[1] HEBARD S,HOCKING G. Echogenic technology can improve needle visibility during ultrasound-guided regional anesthesia[J]. Reg. Anesth. Pain. Med.,2011,36(2):185-189.
[2] NATHAN O,JONATAN S,KEVIN B,et al. Power doppler ultrasound imaging with mechanical perturbation for improved intraoperative needle tip identification during prostate brachytherapy:A phantom study[C]//Image-Guided Procedures,Robotic Interventions,and Modeling 2019. 2019:31-37.
[3] RAMI N,BENNO H W,ROSS S,et al. Real-time in vivo characterization of primary liver tumors with diffuse optical spectroscopy during percutaneous needle interventions:Feasibility study in woodchucks[J]. Investigative Radiology,2015,50(7):443-448.
[4] LIU W,YANG Z,LI P,et al. Mechanics of tissue rupture during needle insertion in transverse isotropic soft tissue[J].Medical and Biological Engineering and Computing,2019,57(6):1353-1366.
[5] RENFREW M,GRISWOLD M,AVUOLU M C. Active localization and tracking of needle and target in robotic image-guided intervention systems[J]. Autonomous Robots,2017,42(4):1-15.
[6] ROSSA C,TAVAKOLI M. Issues in closed-loop needle steering[J]. Control Engineering Practice,2017,62:55-69.
[7] KHADEM M,ROSSA C,SLOBODA R S,et al. Mechanics of tissue cutting during needle insertion in biological tissue[J]. IEEE Robotics and Automation Letters,2016,1(2):800-807.
[8] 张永德,石开铭,赵燕江,等. 斜尖柔性针穿刺软组织受力建模及测量方法研究[J]. 仪器仪表学报,2018,39(6):164-172. ZHANG Yongde,SHI Kaiming,ZHAO Yanjiang,et al.Study on force modeling and measurement for bevel tip flexible needle insertion into soft tissue[J]. Chinese Journal of Scientific Instrument,2018,39(6):164-172.
[9] GESSERT N,PRIEGNITZ T,SAATHOFF T,et al. Spatio-temporal deep learning models for tip force estimation during needle insertion[J]. International Journal of Computer Assisted Radiology and Surgery,2019,14(9):1485-1493.
[10] JIANG S,WANG X. Mechanics-based interactive modeling for medical flexible needle insertion in consideration of nonlinear factors[J]. Journal of Computational and Nonlinear Dynamics,2016,11(1):1-11.
[11] KATAOKA H,WASHIO T,AUDETTE M,et al. A model for relations between needle deflection,force,and thickness on needle penetration[C]//Medical Image Computing & Computer-assisted Intervention-miccai,International Conference,Utrecht,the Netherlands,October. Springer,Berlin,Heidelberg,2001:966-974.
[12] ABOLHASSANI N,PATEL R. Deflection of a flexible needle during insertion into soft tissue[C]//2006 International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE,2006:3858-3861.
[13] SADJADI H,HASHTRUDI-ZAAD K,FICHTINGER G. Needle deflection estimation:Prostate brachytherapy phantom experiments[J]. Int. J. Comput. Assist. Radiol. Surg.,2014,9(6):921-929.
[14] 杜海艳,张永德,赵燕江,等. 斜尖针穿刺软组织建模及针尖轨迹预测[J]. 仪器仪表学报,2015,36(8):1744-1751. DU Haiyan,ZHANG Yongde,ZHAO Yanjiang,et al.Modeling of bevel-tipped needle inserting into soft tissue and estimation of needle tip trajectory[J]. Chinese Journal of Scientific Instrument,2015,36(8):1744-1751.
[15] DIMAIO S P,SALCUDEAN S E. Needle insertion modeling and simulation[J]. IEEE Transactions on Robotics and Automation,2003,19(5):864-875.
[16] KOBAYASHI Y,ONISHI A,WATANABE H,et al. In vitro validation of viscoelastic and nonlinear physical model of liver for needle insertion simulation[C]//IEEE Ras & Embs International Conference on Biomedical Robotics & Biomechatronics. IEEE,2008:469-476.
[17] 郑浩峻,姚望,高德东,等. 机器人辅助柔性针穿刺路径的悬臂梁预测模型[J]. 清华大学学报,2011,51(8):1078-1083. ZHENG Haojun,YAO Wang,GAO Dedong,et al. Projecting beam model for robot-assisted flexible needle insertion[J]. Journal of Tsinghua University,2011,51(8):1078-1083.
[18] 高德东,郑浩峻,姚望,等. 基于虚拟弹簧模型的柔性针穿刺仿真研究[J]. 计算机工程,2010,36(18):273-276.GAO Dedong,ZHENG Haojun,YAO Wang,et al. Study on flexible needle insertion simulation based on virtual spring model[J]. Computer Engineering,2010,36(18):273-276.
[19] MEDAN G,JOSKOWICZ L. Flexible needle and patient tracking using fractional scanning in interventional CT procedures[J]. International Journal of Computer Assisted Radiology and Surgery,2019,14(6):1039-1047.
[20] LI P,YANG Z,JIANG S. Needle-tissue interactive mechanism and steering control in image-guided robot-assisted minimally invasive surgery:A review[J]. Medical & Biological Engineering & Computing,2018,56(6):931-949.
[21] MIGNON P,POIGNET P,TROCCAZ J. Automatic robotic steering of flexible needles from 3D ultrasound images in phantoms and ex vivo biological tissue[J]. Annals of Biomedical Engineering,2018,46(9):1385-1396.
[22] WANG L,GAO D,FU J,et al. Simulation of coupling process of flexible needle insertion into soft tissue based on ABAQUS[J]. Computers,Materials and Continua,2020,64(2):1153-1169.
[23] 高德东. 柔性针穿刺软组织变形机理及动态轨迹规划方法研究[D]. 杭州:浙江大学,2016.GAO Dedong. Tissue deformation mechanism and dynamic path planning for flexible needle into soft tissue[D]. Hangzhou:Zhejiang University,2016.
[24] ASADIAN A,KERMANI M R,PATEL R V. A novel force modeling scheme for needle insertion using multiple Kalman filters[J]. IEEE Transactions on Instrumentation and Measurement,2011,61(2):429-438.
[25] SADJADI H,HASHTRUDI-ZAAD K,FICHTINGER G. Needle deflection estimation:Prostate brachytherapy phantom experiments[J]. Int. J. Comput. Assist. Radiol. Surg.,2014,9(6):921-929.
[26] PAN L,SHAN J,YAN Y,et al. Biomaterial characteristics and application of silicone rubber and PVA hydrogels mimicked in organ groups for prostate brachytherapy[J]. Journal of the Mechanical Behavior Biomedical Materials, 2015,49:220-234.
[27] 朱侗,高德东,赵广伟. 针穿刺软组织试验假体的制备与选择[J]. 科技视界,2015(17):56,68.ZHU Tong,GAO Dedong,ZHAO Guangwei. Preparation and selection of needle insert soft tissue experimental prosthesis[J]. Science & Technology Vision,2015(17):56,68.
[28] 白辉全,高德东,王珊,等. 基于图像形态学的软组织变形测量方法研究[J]. 图学学报,2015,36(5):771-775. BAI Huiquan,GAO Dedong,WANG Shan,et al. A measurement method for soft tissue deformation based on image morphology[J]. Journal of Graphics,2015,36(5):771-775.
[29] BARBE L,BAYLE B,et al. In vivo model estimation and haptic characterization of needle insertions[J]. International Journal of Robotics Research,2007,26(11-12):1283-1301.
[30] PODDER T K,SHERMAN J,FULLER D,et al. In-vivo measurement of surgical needle intervention parameters:a pilot study[C]//in Proc. 28th IEEE EMBS Annu. Int. Conf,2006:3652-3655.
[31] 赵诗剑,高德东,赵梦潇,等. 基于正向运动学的穿刺针半径拟合模型[J]. 青海大学学报,2020,38(2):42-48. ZHAO Shijian,GAO Dedong,ZHAO Mengxiao,et al. Radius fitting model of flexible needle based on forward kinematics[J]. Journal of Qinghai University,2020,38(2):42-48.