Driverless Walking Method of Electric Construction Machinery Based on Environment Recognition

doi:10.3901/JME.2021.10.042

Abstract

Abstract: Driverless technology is the technology that the vehicle is completely controlled by the driverless system to complete the driving task under any environment and road conditions. The operating environment of construction machinery is harsh, and some tasks are highly repetitive; driverless technology of construction machinery can effectively reduce the operation risk of drivers, save labor costs and improve operation efficiency. To realize the driverless electric construction machinery, according to the characteristics of construction machinery, an end-to-end decision system based on monocular camera is adopted. Considering the operation scenarios particularity, a semantic segmentation network is proposed that can fuse the spatial features acquired by the camera in real time, shorten the image processing time, and reduce the system memory occupation. Through testing in the Udacity, it is found that the system can smoothly complete the driving simulation task. To validate the system, an electric crawler excavator is chosen as the system’s experimental platform. The driverless experiments is carried out on an unstructured road. It can safely complete straight-line driving and turning tasks, and it has good control characteristics when returning to the center at a small angle, lays the foundation for the multi-camera construction machinery complex working condition recognition and decision system in addition.

Key words: construction machinery, driverless technology, deep learning, end-to-end, semantic segmentation

CLC Number:

TP242

LIN Tianliang, YAO Yu, XU Wenjie, FU Shengjie, REN Haoling, CHEN Qihuai. Driverless Walking Method of Electric Construction Machinery Based on Environment Recognition[J]. Journal of Mechanical Engineering, 2021, 57(10): 42-49.

References

[1] GE Lei,QUAN Long,ZHANG Xiaogang,et al. Power matching and energy efficiency improvement of hydraulic excavator driven with speed and displacement variable power source[J]. Chinese Journal of Mechanical Engineering,2019,32:100.
[2] XU Bing,SHEN Jun,LIU Shihao,et al. Research and development of electro-hydraulic control valves oriented to industry 4.0:A review[J]. Chinese Journal of Mechanical Engineering,2020,33:29.
[3] Baidu. Apollo pilot safety report[EB/OL].[2020-04-25]. http://www.Apollo.auto/platform/sec-urity_cn.html.
[4] CHO H,SEO Y W,KUMAR B V K V,et al. A multi-sensor fusion system for moving object detection and tracking in urban driving environments[C]//IEEE International Conference on Robotics and Automation (ICRA). IEEE,2014:1836-1843.
[5] Chen Zhilu,Huang Xinming. End-to-end learning for lane keeping of self-driving cars[C]//2017 IEEE Intelligent Vehicles Symposium (IV). IEEE,2017:1856-1860.
[6] 郭景华,罗禹贡,李克强. 智能电动车辆横纵向协调与重构控制[J]. 控制理论与应用,2014,31(9):1238-1244. GUO Jinghua,LUO Yugong,LI Keqiang. Cooperative and reconfigurable lateral and longitudinal control of intelligent electric vehicles[J]. Control Theory & Applications,2014,31(9):1238-1244.
[7] TALEBPOUR A,MAHMASSANI H S,Hamdar S H. Modeling lane-changing behavior in a connected environment:A game theory approach[J]. Transportation Research Part C,2015,59:216-232.
[8] Ion A,Carreira J,Sminchisescu C. Probabilistic joint image segmentation and labeling by figure-ground composition[J]. International Journal of Computer Vision,2014,107(1):40-57.
[9] Long J,Shelhamer E,Darrell T. Fully convolutional networks for semantic segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR),Boston,MA,2015:3431-3440.
[10] Badrinarayanan V,Kendall A,Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Trans. Pattern Anal. Mach. Intell.,2017,39(12):2481-2495.
[11] Romera E,Alvarez J M,Bergasa L M,et al. ERFNet:Efficient residual factorized ConvNet for real-time semantic segmentation[J]. IEEE Transactions on Intelligent Transportation Systems,2018,19(1):263-272.
[12] Brostow G J,Fauqueur J,Cipolla R. Semantic object classes in video:A high-definition ground truth database[J]. Pattern Recognition Letters,2009,30(2):88-97.
[13] HEYLEN J,IVEN S,BRABANDERE B D,et al. From pixels to actions:Learning to drive a car with deep neural networks[C]//2018 IEEE Winter Conference on Applications of Computer Vision(WACV). IEEE,2018:606-615.

[1]	LI Chunkai, WANG Jiaxin, SHI Yu, DAI Yue. Study on the Dynamic Behavior of GTAW Melt Pool Laser Streak and Penetration Prediction Method [J]. Journal of Mechanical Engineering, 2024, 60(6): 236-244.
[2]	ZHAO Dewang, JIANG Chao, ZHAO Yanguang, YANG Wenping, FAN Junling. Fatigue Performance Prediction Study of Al-Li Alloy-based on Experimental and “Shallow” + “Deep” Hybrid Neural Network Methods [J]. Journal of Mechanical Engineering, 2024, 60(16): 190-199.
[3]	FU Ling, SHE Lingjuan, YAN Dulei, ZHANG Peng, LONG Xiangyun. Fatigue Damage Prediction Framework of The Boom System Based on Embedded Physical Information and Attention Mechanism BiLSTM Neural Network [J]. Journal of Mechanical Engineering, 2024, 60(13): 205-215.
[4]	CHEN Qian, CHEN Kangkang, DONG Xingjian, HUANGFU Yifan, PENG Zhike, MENG Guang. Interpretable Convolutional Neural Network for Mechanical Equipment Fault Diagnosis [J]. Journal of Mechanical Engineering, 2024, 60(12): 65-76.
[5]	LIU Yuekai, WANG Tianyang, CHU Fulei. Study on Fault Diagnostics and Knowledge Embedding of Complex Rotating Machinery Components Based on Low Delay Interpretable Deep Learning [J]. Journal of Mechanical Engineering, 2024, 60(12): 107-115.
[6]	LAI Xuwei, DING Kun, ZHANG Kai, HUANG Fengfei, ZHENG Qing, LI Zhixuan, DING Guofu. Improved Interpretable-based Physically Guided Spatial Attention for Cross-process Parameters End Milling Cutter Wear Identification [J]. Journal of Mechanical Engineering, 2024, 60(12): 147-157.
[7]	SHAO Haidong, XIAO Yiming, DENG Qianwang, REN Yingying, HAN Te. Trustworthy Mechanical Fault Diagnosis Using Uncertainty-aware Network [J]. Journal of Mechanical Engineering, 2024, 60(12): 194-206.
[8]	LI Jiufa, ZOU Bowen, REN Yue. Research on Vehicle Trajectory Prediction Considering Vehicle-lane Interaction [J]. Journal of Mechanical Engineering, 2024, 60(10): 76-85.
[9]	XU Siyuan, ZHANG Weidong, MAO Yuming, NIU Bin. Data-driven Design Optimization of Heterogeneous Cellular Structures for Realizing the Specified Deformation [J]. Journal of Mechanical Engineering, 2024, 60(1): 85-95.
[10]	HE Yunze, LI Xiang, WANG Hongjin, HOU Yuejun, ZHANG Fan, MU Xinying, LIU Hao, CHENG Hao, LI Shihua, LI Jie. A Review: Full-cycle Nondestructive Testing Based on Visible Light and Thermography of Wind Turbine Blade [J]. Journal of Mechanical Engineering, 2023, 59(6): 32-45.
[11]	REN Haoling, WU Jiangdong, LIN Tianliang, ZHANG Chunhui, LI Yukun. Construction Machinery Localization and Mapping System Based on Multi-sensor Tight Coupling [J]. Journal of Mechanical Engineering, 2023, 59(24): 323-333.
[12]	PENG Pai, GENG Keke, WANG Ziwei, LIU Zhichao, YIN Guodong. Review on Environmental Perception Methods of Autonomous Vehicles [J]. Journal of Mechanical Engineering, 2023, 59(20): 281-303.
[13]	KANG Jie, WANG Yin, LUO Jie, SUN Jiabao, ZENG Shuhong. Automated Operational Modal Analysis Method:A Survey [J]. Journal of Mechanical Engineering, 2023, 59(13): 89-109.
[14]	WU Qiangewei, WEN Long. Union-adversarial Training for Deep Learning based on Surface Defect Detection [J]. Journal of Mechanical Engineering, 2023, 59(12): 173-182.
[15]	LI Xiong, SU Jianning, ZHANG Zhipeng. Product Conceptual Sketch Generation Design Using Deep Learning [J]. Journal of Mechanical Engineering, 2023, 59(11): 16-30.