• CN:11-2187/TH
  • ISSN:0577-6686

机械工程学报 ›› 2020, Vol. 56 ›› Issue (20): 47-58.doi: 10.3901/JME.2020.20.047

• 材料科学与工程 • 上一篇    下一篇

扫码分享

基于深度强化学习与有限元仿真集成的拉深成形控制

郭鹏, 张新艳, 余建波   

  1. 同济大学机械与能源工程学院 上海 201804
  • 收稿日期:2019-08-18 修回日期:2020-04-18 出版日期:2020-10-20 发布日期:2020-12-18
  • 作者简介:郭鹏,男,1994年出生。主要研究方向为生产系统控制。E-mail:guopeng19940821@163.com 张新艳,女,1974年出生,讲师,硕士研究生导师。主要研究方向为物流系统规划与分析。E-mail:alicezxy@tongji.edu.cn;余建波(通信作者),男,1978年出生,博士,教授,博士研究生导师。主要研究方向为设备智能预诊维护与可靠性、复杂制造过程质量控制、机器学习、生产系统设计优化。E-mail:jbyu@tongji.edu.cn
  • 基金资助:
    国家自然科学基金资助项目(71777173)。

Control of Deep Drawing Process Based on Integration of Deep Reinforcement Learning and Finite Element Method

GUO Peng, ZHANG Xinyan, YU Jianbo   

  1. School of Mechanical Engineering, Tongji University, Shanghai 201804
  • Received:2019-08-18 Revised:2020-04-18 Online:2020-10-20 Published:2020-12-18

摘要: 金属板材拉深过程中的压边力是决定成品质量的关键参数,传统压边力控制方法往往需要对高度非线性的拉深过程进行建模,导致其控制结果与实际存在较大偏差。提出一种基于深度强化学习与有限元仿真集成的金属板材拉深过程控制模型,利用深度神经网络强大的预测能力来提取拉深加工过程中的状态信息并进行可靠预测,结合强化学习的决策能力来进行压边力控制策略的学习优化,避免了精确系统动力学模型的拟合以及先验知识的获取。同时,针对板材拉深加工中常见的拉裂质量缺陷与起皱质量缺陷,建立拉深成形性能评价函数,为深度强化学习提供回报信号来指导学习过程,并利用有限元仿真构成深度强化学习的环境模型。试验表明,深度强化学习模型能够有效地进行压边力控制策略优化,有效提高产品质量。所提出的压边力控制模型利用无模型的深度强化学习,能避免拉深过程的系统模型拟合,可提高压边力控制策略的控制效果,同时结合循环神经网络能解决板材拉深加工过程中的部分可观察性问题。

关键词: 板材拉深成形, 质量控制, 深度强化学习, 有限元仿真, 优化控制

Abstract: The blank holder force in the deep drawing process is the key to determine the quality of finished products. The traditional blank holder force control methods often need to model the highly nonlinear deep drawing process, resulting in a large deviation of the control result from the real situation. The proposed control model based on integration of deep reinforcement learning and finite element method, uses the prediction ability of deep neural network to extract the state information of deep drawing process and predict system state, uses the decision ability of reinforcement learning to optimize control policy, and avoids the fitting of highly nonlinear process dynamic and the acquirement of prior knowledge. Besides, according to the common quality defects in deep drawing process, which are crack and wrinkle, the evaluation function of deep drawing process is established to provide the reward signal to guide the deep reinforcement learning, and the environment model of deep reinforcement learning is constructed by using finite element simulation. Experiments show that the deep reinforcement learning model can effectively optimize the blank holder force control policy and improve the product quality. The proposed blank holder force control model uses model-free deep reinforcement learning to avoid the fitting of deep drawing process, and improves the control effect of blank holder force control policy. The usage of recurrent neural network solves the problem of partial observability in the deep drawing process.

Key words: deep drawing, quality control, deep reinforcement learning, finite element analysis, optimal control

中图分类号: