Abstract: The intelligent manufacturing system adopts a large number of advanced information technologies such as the Internet of things, so that the workshop has accumulated a large amount of real-time production data. At the same time, the complex manufacturing system is prone to a series of disturbance events during the operation process, which puts forward higher requirements for the workshop’s real-time response capability. Therefore, in the manufacturing environment supported by industrial big data, a deep-reinforcement-learning-based real-time scheduling method is proposed for the hybrid flow shop scheduling problem with sequence-dependent setup times and blocking (HFSP-SDST-B), so as to realize the reasonable allocation of manufacturing resources and minimization of makespan. As a sequential decision-making problem, HFSP-SDST-B can be modeled as a markov decision process. At each scheduling point, the agent selects the corresponding scheduling rule according to the current production state, so as to perform the reasonable job sorting and machine allocation. In order to realize the real-time scheduling method driven by production data, the scheduling point considering the blocking, general production state characteristics, heuristic rules based on genetic programming and reward function are designed. Then a training method based on proximal policy optimization algorithm is proposed, so that the agent can build an effective mapping between state and rule. Finally, the experimental results show that compared with the existing dynamic scheduling methods, this method has superiority and generality, and can effectively deal with the unknown situation of stochastic disturbance time and new order insertion through learning.

Key words: hybrid flow shop scheduling problem, real-time scheduling, reinforcement learning, sequence-dependent setup times, blocking