Abstract: The job-shop scheduling problem is a classic NP-hard combinatorial optimization problem,and the quality of scheduling directly impacts the operational efficiency of manufacturing systems.In order to obtain a better scheduling strategy with the goal of minimizing the maximum completion time,a Deep Reinforcement Learning (DRL) scheduling method based on Proximal Policy Optimization (PPO) and Convolutional Neural Network (CNN) is proposed. A three-channel state representation method is designed,with 16 heuristic scheduling rules selected as the action space,and the reward function is equivalent to minimizing the total idle time of machines. In order to enable the trained scheduling strategy to handle scheduling instances of different scales,Spatial Pyramid Pooling (SPP) is applied in the convolutional neural network to convert feature matrices of different dimensions into fixed-length feature vectors.Computational experiments are conducted on 42 Job-Shop Scheduling Problem (JSSP) instances from the public OR-Library. The results of the simulation experiments show that the proposed algorithm outperforms single heuristic scheduling rules and genetic algorithms,achieving better results than existing deep reinforcement learning algorithms in most instances,and with the smallest average completion time.

Key words: Deep Reinforcement Learning(DRL), job-shop scheduling problem, Convolutional Neural Network(CNN), Proximal Policy Optimization(PPO), Spatial Pyramid Pooling(SPP)