面向机器人环境共融的图像去雪算法

doi:10.3901/JME.2019.11.098

机械工程学报 ›› 2019, Vol. 55 ›› Issue (11): 98-104.doi: 10.3901/JME.2019.11.098

• 特邀专栏：共融机器人 • 上一篇下一篇

面向机器人环境共融的图像去雪算法

李鹏越^1,2,3, 田建东^2,3, 王国霖^2,3,4, 李小毛⁵, 唐延东^2,3, 吴成东¹

1. 东北大学机器人科学与工程学院沈阳 110819;
2. 中国科学院沈阳自动化研究所机器人学国家重点实验室沈阳 110016;
3. 中国科学院机器人与智能制造创新研究院沈阳 110016;
4. 中国科学院大学北京 100049;
5. 上海大学机电工程与自动化学院上海 200444

收稿日期:2018-09-14 修回日期:2018-11-21 出版日期:2019-06-05 发布日期:2019-06-05
通讯作者: 田建东(通信作者),男,1980年出生,研究员,博士研究生导师。主要研究方向为机器人视觉、成像计算与图像恢复。E-mail:tianjd@sia.cn
作者简介:李鹏越,男,1990年出生,博士研究生。主要研究方向为坏天气下的图像处理和机器人视觉。E-mail:lipengyue@sia.cn;李小毛,男,1981年出生,副研究员,硕士研究生导师。主要研究方向为图像处理和无人艇。E-mail:lixiaomao@shu.edu.cn
基金资助:
国家自然科学基金资助项目（91648118，61473280，61821005）。

Image Snow Removal Methods for Robotic Environment Fusion

LI Pengyue^1,2,3, TIAN Jiandong^2,3, WANG Guolin^2,3,4, LI Xiaomao⁵, TANG Yandong^2,3, WU Chengdong¹

1. Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819;
2. State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016;
3. Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110016;
4. University of Chinese Academy of Sciences, Beijing 100049;
5. School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444

Received:2018-09-14 Revised:2018-11-21 Online:2019-06-05 Published:2019-06-05

摘要/Abstract

摘要： 针对雪天气影响共融机器人视觉系统鲁棒性的问题，提出了一种基于雪模型和深度学习融合的去雪算法。根据雪的成像过程推导了一个简化的雪模型，设计了一个基于该模型的深度去雪网络，该网络由雪花检测子网络和去除子网络串联组成。雪花检测子网络采用了残差学习网络，该网络可以准确地学习雪图像和无雪图像之间的差异。去雪子网络采用了密集连接的U型网络。它一方面利用U型网络保留背景的细节信息，另一方面利用DenseNet将低层特征复用到高层的特点来提高去雪的准确度，将它们结合后缓解了去雪过度导致背景细节丢失和去雪不彻底之间的矛盾。试验证明这种基于雪模型的深度去雪网络能够较好地检测和去除图像中的雪花。

关键词: 环境共融, 机器人, 去雪, 深度学习

Abstract: Aiming at the problem that snow weather affects the robustness of the fusion robotic vision system, a snow removal method based on snow model and deep learning is proposed. A simplified snow model is derived based on the snow imaging process, and a deep snow removal network is designed based on this model. The network consists of a snowflakes detection sub-network and a snowflakes removal sub-network. The snowflakes detection sub-network uses a residual learning network to accurately learn the difference between snow images and snow-free images. The desnowing sub-network adopts a densely connected U network. It usually can relieve the contradiction of over-desnowing and under-desnowing by using U-net to preserve image details and feature reuse of DenseNet to accuratelly remove snowflakes. Experiments show that the snow model-based deep networks can effectively detect and remove snowflakes from images.

Key words: deep learning, desnowing, environment fusion, robot

中图分类号:

TP242

李鹏越, 田建东, 王国霖, 李小毛, 唐延东, 吴成东. 面向机器人环境共融的图像去雪算法[J]. 机械工程学报, 2019, 55(11): 98-104.

LI Pengyue, TIAN Jiandong, WANG Guolin, LI Xiaomao, TANG Yandong, WU Chengdong. Image Snow Removal Methods for Robotic Environment Fusion[J]. Journal of Mechanical Engineering, 2019, 55(11): 98-104.

参考文献

[1] NARASIMHAN S G,NAYAR S K. Contrast restoration of weather degraded images[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence,2003,25(6):713-724.
[2] FU Xueyang,HUANG Jiabin,ZENG Delu,et al. Removing rain from single images via a deep detail network[C]//IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society,2017:1715-1723.
[3] ZHANG He,PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]//IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society,2018:695-704.
[4] LUO Yu,XU Yong,JI Hui. Removing rain from a single image via discriminative sparse coding[C]//IEEE International Conference on Computer Vision. IEEE,2015:3397-3405.
[5] WANG Yinglong,LIU Shuaicheng,CHEN Chen,et al. A hierarchical approach for rain or snow removing in a single color image[J]. IEEE Transactions on Image Processing,2017,26(8):3936-3950.
[6] LI Yu,TAN R T,GUO Xiaojie,et al. Rain streak removal using layer priors[C]//Computer Vision and Pattern Recognition. IEEE,2016:2736-2744.
[7] DING Xinghao,CHEN Liqin,ZHENG Xianhui,et al. Single image rain and snow removal via guided L0 smoothing filter[J]. Multimedia Tools & Applications,2016,75(5):2697-2712.
[8] REN Weihong,TIAN Jiandong,HAN Zhi,et al. Video desnowing and deraining based on matrix decomposition[C]//IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society,2017:2838-2847.
[9] CAI Bolun,XU Xiangmin,JIA Kui,et al. DehazeNet:An end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing,2016,25(11):5187-5198.
[10] HE Kaiming,SUN Jian,TANG Xiaoou. Single image haze removal using dark channel prior[J]. IEEE Trans. Pattern Anal. Mach. Intell.,2011,33(12):2341-2353.
[11] LI Boyi,PENG Xiulian,WANG Zhangyuang,et al. AOD-net:All-in-one dehazing network[C]//IEEE International Conference on Computer Vision. IEEE Computer Society,2017:4780-4788.
[12] XU Jiang,ZHAO Wei,LIU Peng,et al. An improved guidance image based method to remove rain and snow in a single image[J]. Computer & Information Science,2012,5(3):49-55.
[13] PEI S C,TSAI Y T,LEE C Y. Removing rain and snow in a single image using saturation and visibility features[C]//IEEE International Conference on Multimedia and Expo Workshops. IEEE,2014:1-6.
[14] ZHENG Xianhui,LIAO Yinghao,GUO Wei,et al. Single-image-based rain and snow removal using multi-guided filter[C]//International Conference on Neural Information Processing. Springer,Berlin,Heidelberg,2013:258-265.
[15] GARG K,NAYAR S K. Detection and removal of rain from videos[C]//Computer Vision and Pattern Recognition,2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. IEEE,2004:I-528-I-535.
[16] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al. Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society,2016:770-778.
[17] RONNEBERGER O,FISCHER P,BROX T. U-net:Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer,Cham,2015:234-241.
[18] HUANG Gao,LIU Zhuang,MAATEN L V D,et al. Densely connected convolutional networks[C]//IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society,2017:2261-2269.

面向机器人环境共融的图像去雪算法

Image Snow Removal Methods for Robotic Environment Fusion

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	战强, 李伟. 球形移动机器人的研究进展与发展趋势[J]. 机械工程学报, 2019, 55(9): 1-17.
[2]	李剑锋, 张凯, 张雷雨, 张子康, 左世平. 并联踝康复机器人的设计与运动性能评价[J]. 机械工程学报, 2019, 55(9): 29-39.
[3]	裴洪, 胡昌华, 司小胜, 张建勋, 庞哲楠, 张鹏. 基于机器学习的设备剩余寿命预测方法综述[J]. 机械工程学报, 2019, 55(8): 1-13.
[4]	王奉涛, 刘晓飞, 敦泊森, 邓刚, 韩清凯, 李宏坤. 基于萤火虫优化的核自动编码器在中介轴承故障诊断中的应用[J]. 机械工程学报, 2019, 55(7): 58-64.
[5]	李川, 张绍辉, José Valente de Oliveira. 基于次优网络深度学习的3D打印机故障诊断[J]. 机械工程学报, 2019, 55(7): 73-80.
[6]	雷亚国, 杨彬, 杜兆钧, 吕娜. 大数据下机械装备故障的深度迁移诊断方法[J]. 机械工程学报, 2019, 55(7): 1-8.
[7]	姜洪开, 邵海东, 李兴球. 基于深度学习的飞行器智能故障诊断方法[J]. 机械工程学报, 2019, 55(7): 27-34.
[8]	李满宏, 张明路, 张建华, 田颖, 马艳悦. 基于增强学习的六足机器人自由步态规划[J]. 机械工程学报, 2019, 55(5): 36-44.
[9]	游虹, 尚伟伟, 张彬, 张飞, 丛爽. 基于高速视觉的绳索牵引并联机器人轨迹跟踪控制[J]. 机械工程学报, 2019, 55(5): 19-26.
[10]	姚建涛, 陈新博, 陈俊涛, 张弘, 李海利, 赵永生. 轮足式仿生软体机器人设计与运动分析[J]. 机械工程学报, 2019, 55(5): 27-35.
[11]	董悫, 张立建, 易旺民, 万毕乐, 孟少华, 胡瑞钦. 基于动力学前馈的空间机器人多销孔装配力柔顺控制[J]. 机械工程学报, 2019, 55(4): 207-217.
[12]	郝大贤, 王伟, 王琦珑, 贠超. 复合材料加工领域机器人的应用与发展趋势[J]. 机械工程学报, 2019, 55(3): 1-17.
[13]	汪久根, 柯梁亮. 基于残差网络的RV减速器故障诊断[J]. 机械工程学报, 2019, 55(3): 73-80.
[14]	张建华, 许晓林, 刘璇, 张明路. 双臂协调机器人相对动力学建模[J]. 机械工程学报, 2019, 55(3): 34-42.
[15]	高培源,牛成玉,云亚文,雍明超,方海潇. 电力故障应急机器人设计[J]. 电气工程学报, 2019, 14(3): 103-107.