家庭服务机器人手眼协调系统设计

doi:10.3901/JME.2022.05.69

机械工程学报 ›› 2022, Vol. 58 ›› Issue (5): 69-77.doi: 10.3901/JME.2022.05.69

扫码分享

家庭服务机器人手眼协调系统设计

田蔚瀚¹, 罗亚哲¹, 李逸飞¹, 陈殿生^1,2

1. 北京航空航天大学机械工程及自动化学院北京 100191;
2. 北京航空航天大学北京生物医学工程高精尖中心北京 100191

收稿日期:2021-03-05 修回日期:2021-07-27 出版日期:2022-03-05 发布日期:2022-04-28
通讯作者: 陈殿生(通信作者),男,1969年出生,博士,教授,博士研究生导师。主要研究方向为服务机器人、软体机器人。E-mail:chends@163.com E-mail:chends@163.com
作者简介:田蔚瀚,男,1994年出生。主要研究方向为服务机器人。E-mail:tianweihan@buaa.edu.cn
基金资助:
国家重点研发计划资助项目(2018YFB1307103)。

Design of Hand-eye Coordination System for Home Service Robot

TIAN Yu-han¹, LUO Ya-zhe¹, LI Yi-fei¹, CHEN Dian-sheng^1,2

1. School of Mechanical Engineering and Automation, Beihang University, Beijing 100191;
2. Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191

Received:2021-03-05 Revised:2021-07-27 Online:2022-03-05 Published:2022-04-28

摘要/Abstract

摘要： 针对目前智能家庭场景下,家居服务机器人对不同物品位姿的识别准确度与操作性差等问题,进行机器人手眼协调识别技术与物品灵巧操作规划方法的研究,提出了视觉前馈目标定位与视觉反馈位姿匹配相结合的识别与多位姿抓取方法。首先,基于SSD深度神经网络模型识别待抓取物品,并进行三维测定,以得到物品粗略坐标位置;其次,根据所得三维坐标预设深度相机与物品的相对位姿,采集物品与障碍物的深度信息,并基于Linemod算法与内置三维模型进行位姿匹配,完成对物品精确位姿的测定;最后,依据所得物品与障碍物的位置与姿态,规范臂手系统的抓取位姿,实现灵巧抓取物品。由以上原理,设计手眼协调测试实验进行不同桌面高度的物品抓取成功率与抓取误差测试,实验抓取总成功率高,且误差满足实际精度要求;该研究有利于提高家居服务机器人物品操作的灵巧性与适应性,对于家居服务机器人产业的发展具有重要意义。

关键词: 服务机器人, 手眼协调, 物品抓取, 位姿识别

Abstract: Based on the needs of item identification and operation in current smart home, and in view of the poor dexterity and accuracy of different poses of objects in existing household service robot, the research on robot hand-eye coordination recognition technology and item dexterous operation planning method are studied. In this paper, a recognition and multi-pose grasping method based on visual feedforward to locate three-dimensional position of target and visual feedback to match target pose is proposed.Firstly, SSD depth neural network model is used to identified items, and the three-dimensional position is carried out as the rough coordinate of the target. Then, the relative coordinate of the depth camera and the object is preset according to the obtained three-dimensional coordinates, and the depth information of the object and the obstacle is collected. At the same time, the pose matching is executed by comparing the built-in three-dimensional model with the accurate object by means of Linemod algorithm.Finally, according to the position and posture of the obtained objects and obstacles, the grasping posture of the arm control system is standardized to achieve the dexterous grasping of objects. From the above principles, Design hand-eye control experiments of tables at different heights to test success rate and grasping error by standardizing the position and process of item grabbing. As a result, there is a high successful rate, and the root mean square errors of grasping in three orientations are less than 0.006m. Therefore, it is applicable to the grasping and other operations of cube, column, sphere and other items; It is of great significance to the development of home service robot industry.

Key words: home service robot, hand-eye coordination, pose recognition, object grasping

中图分类号:

TP242

田蔚瀚, 罗亚哲, 李逸飞, 陈殿生. 家庭服务机器人手眼协调系统设计[J]. 机械工程学报, 2022, 58(5): 69-77.

TIAN Yu-han, LUO Ya-zhe, LI Yi-fei, CHEN Dian-sheng. Design of Hand-eye Coordination System for Home Service Robot[J]. Journal of Mechanical Engineering, 2022, 58(5): 69-77.

参考文献

[1] Executive Summary Services Robots 2020[OL].[2020-10-08].https://ifr.org/free-downloads/.
[2] EA AFFUM,AGYEMAN-PREMPEH K,ADUMATTAC,et al.Smart home energy management system based on the Internet of things (Io T)[J].International Journal of Advanced Computer Science and Applications (IJACSA),2021,12(2):1.
[3] CHOI W,KIM J,LEE S,et al.Smart home and internet of things:A bibliometic study[J].Journal of Cleaner Production,2021(1):301.
[4] LI X,ZHONG J,KAMRUZZAMAN M M.Complicated robot activity recognition by quality-aware deep reinforcement learning[J].Future Generation Computer Systems,2021,117:1.
[5] KIM U,KIM J.A stabilized feedback episodic memory (SF-EM) and home service provision framework for robot and Io T collaboration[J].IEEE Transactions on Cybernetics,2020,50(5):1.
[6] 赵雅婷,赵韩,梁昌勇,等.养老服务机器人现状及其发展建议[J].机械工程学报,2019,55(23):13-24.ZHAO Yating,ZHAO Han,LIANG Changyong,et al.Current situation and development suggestions of old-age service robot[J].Journal of Mechanical Engineering,2019,55(23):13-24.
[7] 祁若龙,张珂,周维佳,等.机械臂高斯运动轨迹规划操作成功概率预估计方法[J].机械工程学报,2019,55(1):54-63.QI Ruolong,ZHANG Ke,ZHOU Weijia,et al.Trajectory planning and success probability estimation of operation for Gaussian motion manipulators[J].Journal of Mechanical Engineering,2019,55(1):54-63.
[8] CAO Z,YIN L,FU Y.Vision-based stabilization of nonholonomic mobile robots by integrating sliding-mode control and adaptive approach[J].Chinese Journal of Mechanical Engineering,2013,26(1):21-28.
[9] 杨扬.基于机器视觉的服务机器人智能抓取研究[D].上海:上海交通大学,2014.YANG Yang.Study on the machine vision based intelligent grasping for service robot[D].Shanghai:Shanghai Jiao Tong University,2014.
[10] BESL P,MCKAY N.Method for registration of 3-Dshapes[C]//Robotics-DL tentative.International Society for Optics and Photonics,1992:586-606.
[11] HINTERSTOISSER S,HOLZER S,CAGNIART C,et al.Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes[C]//IEEEInternational Conference on Computer Vision.IEEE,2012:858-865.
[12] SU H,MAJI S,KALOGERAKIS E,et al.Multi-view convolutional neural networks for 3D shape recognition[J].ICCV,2015(1):945-953.
[13] QI C,SU H,MO K,et al.Point Net:Deep learning on point sets for 3D classification and segmentation[J].2017,16:1.
[14] QI C,YI L,SU H,et al.Point Net++:Deep hierarchical feature learning on point sets in a metric space[J].2017,6:1.
[15] LIU Y,FAN B,MENG G.,et al.Dense Point:Learning densely contextual representation for efficient point cloud processing[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).IEEE,2020.
[16] WU W,QI Z,FUXIN L.Point Conv:Deep convolutional networks on 3D point clouds[C]//2019 IEEE/CVFConference on Computer Vision and Pattern Recognition (CVPR).IEEE,2020.
[17] 卢冠男.基于机器视觉的工业机器人抓取系统的研究[D].合肥:合肥工业大学,2017.LU Guannan.Research on industrial robot crawling system based on machine vision[D].Hefei:Hefei University of Technology,2017.
[18] ZHANG H,WANG F,WANG J,et al.Robot grasping method optimization using improved deep deterministic policy gradient algorithm of deep reinforcement learning[J].The Review of Scientific Instruments,2021,92(2):1.
[19] XU L,ZHOU Z,WANG C.An robot vision grasping network based on inception-lite[J].Journal of Physics:Conference Series,2021,1748(2):1.
[20] LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single shot Multi Box detector[C]//European Conference on Computer Vision.Springer,Cham,2016:21-37.
[21] HINTERSTOISSER S,LEPETIT V,ILIC S,et al.Model based training,detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[C]//Asian Conference on Computer Vision.Springer,Berlin,Heidelberg,2012:548-562.
[22] HINTERSTOISSER S,CAGNIART C,ILIC S,et al.Gradient response maps for real-time detection of textureless objects[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2012,34(5):876-888.

家庭服务机器人手眼协调系统设计

Design of Hand-eye Coordination System for Home Service Robot

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价