Abstract: The blank holder force in the deep drawing process is the key to determine the quality of finished products. The traditional blank holder force control methods often need to model the highly nonlinear deep drawing process, resulting in a large deviation of the control result from the real situation. The proposed control model based on integration of deep reinforcement learning and finite element method, uses the prediction ability of deep neural network to extract the state information of deep drawing process and predict system state, uses the decision ability of reinforcement learning to optimize control policy, and avoids the fitting of highly nonlinear process dynamic and the acquirement of prior knowledge. Besides, according to the common quality defects in deep drawing process, which are crack and wrinkle, the evaluation function of deep drawing process is established to provide the reward signal to guide the deep reinforcement learning, and the environment model of deep reinforcement learning is constructed by using finite element simulation. Experiments show that the deep reinforcement learning model can effectively optimize the blank holder force control policy and improve the product quality. The proposed blank holder force control model uses model-free deep reinforcement learning to avoid the fitting of deep drawing process, and improves the control effect of blank holder force control policy. The usage of recurrent neural network solves the problem of partial observability in the deep drawing process.

Key words: deep drawing, quality control, deep reinforcement learning, finite element analysis, optimal control