Design of Hand-eye Coordination System for Home Service Robot

doi:10.3901/JME.2022.05.69

Abstract

Abstract: Based on the needs of item identification and operation in current smart home, and in view of the poor dexterity and accuracy of different poses of objects in existing household service robot, the research on robot hand-eye coordination recognition technology and item dexterous operation planning method are studied. In this paper, a recognition and multi-pose grasping method based on visual feedforward to locate three-dimensional position of target and visual feedback to match target pose is proposed.Firstly, SSD depth neural network model is used to identified items, and the three-dimensional position is carried out as the rough coordinate of the target. Then, the relative coordinate of the depth camera and the object is preset according to the obtained three-dimensional coordinates, and the depth information of the object and the obstacle is collected. At the same time, the pose matching is executed by comparing the built-in three-dimensional model with the accurate object by means of Linemod algorithm.Finally, according to the position and posture of the obtained objects and obstacles, the grasping posture of the arm control system is standardized to achieve the dexterous grasping of objects. From the above principles, Design hand-eye control experiments of tables at different heights to test success rate and grasping error by standardizing the position and process of item grabbing. As a result, there is a high successful rate, and the root mean square errors of grasping in three orientations are less than 0.006m. Therefore, it is applicable to the grasping and other operations of cube, column, sphere and other items; It is of great significance to the development of home service robot industry.

Key words: home service robot, hand-eye coordination, pose recognition, object grasping

CLC Number:

TP242

TIAN Yu-han, LUO Ya-zhe, LI Yi-fei, CHEN Dian-sheng. Design of Hand-eye Coordination System for Home Service Robot[J]. Journal of Mechanical Engineering, 2022, 58(5): 69-77.

References

[1] Executive Summary Services Robots 2020[OL].[2020-10-08].https://ifr.org/free-downloads/.
[2] EA AFFUM,AGYEMAN-PREMPEH K,ADUMATTAC,et al.Smart home energy management system based on the Internet of things (Io T)[J].International Journal of Advanced Computer Science and Applications (IJACSA),2021,12(2):1.
[3] CHOI W,KIM J,LEE S,et al.Smart home and internet of things:A bibliometic study[J].Journal of Cleaner Production,2021(1):301.
[4] LI X,ZHONG J,KAMRUZZAMAN M M.Complicated robot activity recognition by quality-aware deep reinforcement learning[J].Future Generation Computer Systems,2021,117:1.
[5] KIM U,KIM J.A stabilized feedback episodic memory (SF-EM) and home service provision framework for robot and Io T collaboration[J].IEEE Transactions on Cybernetics,2020,50(5):1.
[6] 赵雅婷,赵韩,梁昌勇,等.养老服务机器人现状及其发展建议[J].机械工程学报,2019,55(23):13-24.ZHAO Yating,ZHAO Han,LIANG Changyong,et al.Current situation and development suggestions of old-age service robot[J].Journal of Mechanical Engineering,2019,55(23):13-24.
[7] 祁若龙,张珂,周维佳,等.机械臂高斯运动轨迹规划操作成功概率预估计方法[J].机械工程学报,2019,55(1):54-63.QI Ruolong,ZHANG Ke,ZHOU Weijia,et al.Trajectory planning and success probability estimation of operation for Gaussian motion manipulators[J].Journal of Mechanical Engineering,2019,55(1):54-63.
[8] CAO Z,YIN L,FU Y.Vision-based stabilization of nonholonomic mobile robots by integrating sliding-mode control and adaptive approach[J].Chinese Journal of Mechanical Engineering,2013,26(1):21-28.
[9] 杨扬.基于机器视觉的服务机器人智能抓取研究[D].上海:上海交通大学,2014.YANG Yang.Study on the machine vision based intelligent grasping for service robot[D].Shanghai:Shanghai Jiao Tong University,2014.
[10] BESL P,MCKAY N.Method for registration of 3-Dshapes[C]//Robotics-DL tentative.International Society for Optics and Photonics,1992:586-606.
[11] HINTERSTOISSER S,HOLZER S,CAGNIART C,et al.Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes[C]//IEEEInternational Conference on Computer Vision.IEEE,2012:858-865.
[12] SU H,MAJI S,KALOGERAKIS E,et al.Multi-view convolutional neural networks for 3D shape recognition[J].ICCV,2015(1):945-953.
[13] QI C,SU H,MO K,et al.Point Net:Deep learning on point sets for 3D classification and segmentation[J].2017,16:1.
[14] QI C,YI L,SU H,et al.Point Net++:Deep hierarchical feature learning on point sets in a metric space[J].2017,6:1.
[15] LIU Y,FAN B,MENG G.,et al.Dense Point:Learning densely contextual representation for efficient point cloud processing[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).IEEE,2020.
[16] WU W,QI Z,FUXIN L.Point Conv:Deep convolutional networks on 3D point clouds[C]//2019 IEEE/CVFConference on Computer Vision and Pattern Recognition (CVPR).IEEE,2020.
[17] 卢冠男.基于机器视觉的工业机器人抓取系统的研究[D].合肥:合肥工业大学,2017.LU Guannan.Research on industrial robot crawling system based on machine vision[D].Hefei:Hefei University of Technology,2017.
[18] ZHANG H,WANG F,WANG J,et al.Robot grasping method optimization using improved deep deterministic policy gradient algorithm of deep reinforcement learning[J].The Review of Scientific Instruments,2021,92(2):1.
[19] XU L,ZHOU Z,WANG C.An robot vision grasping network based on inception-lite[J].Journal of Physics:Conference Series,2021,1748(2):1.
[20] LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single shot Multi Box detector[C]//European Conference on Computer Vision.Springer,Cham,2016:21-37.
[21] HINTERSTOISSER S,LEPETIT V,ILIC S,et al.Model based training,detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[C]//Asian Conference on Computer Vision.Springer,Berlin,Heidelberg,2012:548-562.
[22] HINTERSTOISSER S,CAGNIART C,ILIC S,et al.Gradient response maps for real-time detection of textureless objects[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2012,34(5):876-888.