A dynamic flexible job shop scheduling method based on deepreinforcement learning

doi:10.16731/j.cnki.1671-3133.2025.02.002

Abstract

Abstract: The study of the artificial intelligence algorithms for job shop scheduling has gained attention due to the advancements in intelligent manufacturing technologies like smart factories. Dynamic events in the job shop are crucial factors affecting scheduling effectiveness. To this end,it proposes a novel approach employing the deep reinforcement learning to solve the dynamic flexible job shop scheduling problem with random job arrival. Initially,a mathematical model is formulated for the dynamic job shop scheduling problem with the objective of minimizing the total tardiness. Subsequently,eight job shop state features are extracted,and six composite scheduling rules are designed. An ε-greedy action selection strategy is adopted,and the reward function is designed. Finally,the advanced D3QN algorithm is introduced to solve the problem and the effectiveness of this method is verified on different scale of instances. The results show that the D3QN algorithm effectively solves the dynamic flexible job shop scheduling problem with random job arrival,and the winning rate in all instances is 58.3 %.Compared with traditional DQN and DDQN algorithm,the total tardiness is reduced by 11.0 % and 15.4 % respectively,which proves that this method further enhances the production efficiency of the job shop.

Key words: deep reinforcement learning, D3QN algorithm, random job arrival, flexible job shop scheduling problem, dynamicscheduling

CLC Number:

TP181

YANG Dan, SHU Xiantao, YU Zhen, LU Guangtao, JI Songlin, WANG Jiabing. A dynamic flexible job shop scheduling method based on deepreinforcement learning[J]. Modern Manufacturing Engineering, 2025, 533(2): 10-16.

References

[1] LI Xixing,GUO Xing,HONG Tao,et al.Survey of integrated flexible job shop scheduling problems[J].Computers & Industrial Engineering,2022,174:108786.
[2] 何斌,张接信,张富强.柔性作业车间调度问题简明分析[J].装备制造技术,2018(5):68-71,85.
[3] GAREY M R,JOHNSON D S,SETHI R.The complexity of flow shop and job shop scheduling[J].Mathematics of Operations Research,1976,1(2):117-129.
[4] 范华丽,熊禾根,蒋国璋,等.动态车间作业调度问题中调度规则算法研究综述[J].计算机应用研究,2016,33(3):648-653.
[5] DEMIR Y,YILMAZ H.An efficient priority rule for flexible job shop scheduling problem[J].Journal of Engineering Research and Applied Science,2021,10(2):1906-1918.
[6] 周鹏鹏,翟志波,戴玉森.基于改进遗传算法的柔性作业车间调度问题研究[J].组合机床与自动化加工技术,2023(3):183-186,192.
[7] 张守京,王彦亭.基于改进NSGA2的柔性车间多目标智能调度问题研究[J].现代制造工程,2020(9):23-31.
[8] 蔡敏,王艳,纪志成.混合粒子群优化算法求解模糊柔性作业车间调度问题[J].南京理工大学学报(自然科学版),2021,45(3):352-360.
[9] 吴秀丽,刘夏晶.差分进化算法求解分布式柔性作业车间调度问题[J].计算机集成制造系统,2019,25(10):2539-2558.
[10] XIA W,WU Z.An effective hybrid optimization approach for multi-objective flexible job-shop scheduling problems[J].Computers & Industrial Engineering,2005,48(2):409-425.
[11] 崔雪艳,万烂军,赵昊鑫,等.基于深度强化学习的柔性作业车间调度方法[J].制造技术与机床,2023(12):165-170.
[12] 张琦,张彬.动态环境下的柔性作业车间调度问题研究[J].机电产品开发与创新,2023,36(6):92-94.
[13] 胡一凡,张利平,白雪,等.深度强化学习求解柔性装配作业车间调度问题[J].华中科技大学学报(自然科学版),2023,51(2):153-160.
[14] ZHAO Yejian,WANG Yanhong,TAN Yuanyuan.Dynamic jobshop scheduling algorithm based on deep Q network[J].IEEE Access,2021(9):122995-123011.
[15] LUO Shu,ZHANG Linxuan,FAN Yushun.Dynamic multi-objective scheduling for flexible job shop by deep reinforcement learning[J].Computers & Industrial Engineering,2021,159:107489.
[16] CHANG Jingru,YU Dong,HU Yi.Deep reinforcement learning for dynamic flexible job shop scheduling with random job arrival[J].Processes,2022,10(4):760.
[17] 程俊夫.基于双重深度Q学习与邻域搜索算法的动态多目标柔性作业车间调度问题研究[D].成都:四川大学,2022.
[18] 罗梓珲,江呈羚,刘亮,等.基于深度强化学习的智能车间调度方法研究[J].物联网学报,2022,6(1):53-64.
[19] MOHAN J,LANKA K,RAO A N.A review of dynamic job shop scheduling techniques[J].Procedia Manuf.,2019,30:34-39.
[20] 李宝帅,叶春明.深度强化学习算法求解作业车间调度问题[J].计算机工程与应用,2021,57(23):248-254.