基于DQN的改进NSGA-Ⅱ求解多目标柔性作业车间调度问题

doi:10.16731/j.cnki.1671-3133.2025.09.001

现代制造工程 ›› 2025, Vol. 540 ›› Issue (9): 1-11.doi: 10.16731/j.cnki.1671-3133.2025.09.001

• 先进制造系统管理运作 • 下一篇

基于DQN的改进NSGA-Ⅱ求解多目标柔性作业车间调度问题

郑国梁¹, 张朝阳^1,2, 吉卫喜^1,2, 于俊杰¹

1 江南大学机械工程学院,无锡 214122;
2 江苏省食品制造装备重点实验室,无锡 214122

收稿日期:2024-09-09 出版日期:2025-09-18 发布日期:2025-09-23
通讯作者: 张朝阳,副教授,博士,主要研究方向为智能制造系统与低碳制造。E-mail:cyzhang@jiangnan.edu.cn。
作者简介:郑国梁,硕士研究生,主要研究方向为智能制造系统。E-mail:244435762@qq.com。
基金资助:
国家自然科学基金青年科学基金项目(51805213)

Solving multi-objective flexible job shop scheduling problems with an improved NSGA-Ⅱ based on DQN

ZHENG Guoliang¹, ZHANG Chaoyang^1,2, JI Weixi^1,2, YU Junjie¹

1 School of Mechanical Engineering,Jiangnan University,Wuxi 214122,China;
2 Jiangsu Key Laboratory of Advanced Food Manufacturing Equipment and Technology, Wuxi 214122,China

Received:2024-09-09 Online:2025-09-18 Published:2025-09-23

摘要/Abstract

摘要： 提出了一种基于深度Q网络(Deep Q-Network,DQN)改进的非支配排序遗传算法(Non-dominated Sorting Genetic Algorithm Ⅱ,NSGA-Ⅱ),以解决以最小化最大完工时间和最小化能源消耗为目标的多目标柔性作业车间调度问题(Multi-Objective Flexible Job shop Scheduling Problem,MO-FJSP)。通过在DQN算法中定义马尔可夫决策过程和奖励函数,考虑选定设备对完工时间和能源消耗的局部及全局影响,提高了NSGA-Ⅱ初始种群的质量。改进的NSGA-Ⅱ通过精英保留策略确保运行过程中的种群多样性,并保留了进化过程中优质的个体。将DQN算法生成的初始解与贪婪算法生成的初始解进行对比,验证了DQN算法在生成初始解方面的有效性。此外,将基于DQN算法的改进NSGA-Ⅱ与其他启发式算法在标准案例和仿真案例上进行对比,证明了其在解决MO-FJSP方面的有效性。

关键词: 深度Q网络算法, 多目标柔性作业车间调度问题, 奖励函数, 非支配排序遗传算法

Abstract: An improved Non-dominated Sorting Genetic Algorithm Ⅱ (NSGA-Ⅱ) based on the Deep Q-Network (DQN) is proposed to solve the Multi-Objective Flexible Job shop Scheduling Problem (MO-FJSP) with the goals of minimizing makespan and energy consumption. The Markov decision process and a reward function are defined in the DQN algorithm, considering the influence of selected machines on makespan and energy consumption both locally and globally. This approach enhances the quality of the initial population of the NSGA-Ⅱ. The elite retention strategy of the NSGA-Ⅱ is improved to ensure population diversity during execution and preserve high-quality individuals throughout the evolutionary process. The effectiveness of the DQN algorithm in generating initial solutions is validated by comparing its initial solutions with those generated by a greedy algorithm. Furthermore, the improved NSGA-Ⅱ based on the DQN algorithm is compared with other heuristic algorithms on standard and simulation cases, demonstrating its effectiveness in solving MO-FJSP.

Key words: Deep Q-Network (DQN) algorithm, multi-objective flexible job shop scheduling problem, reward function, Non-dominated Sorting Genetic Algorithm Ⅱ (NSGA-Ⅱ)

中图分类号:

TH165
TP18

郑国梁, 张朝阳, 吉卫喜, 于俊杰. 基于DQN的改进NSGA-Ⅱ求解多目标柔性作业车间调度问题[J]. 现代制造工程, 2025, 540(9): 1-11.

ZHENG Guoliang, ZHANG Chaoyang, JI Weixi, YU Junjie. Solving multi-objective flexible job shop scheduling problems with an improved NSGA-Ⅱ based on DQN[J]. Modern Manufacturing Engineering, 2025, 540(9): 1-11.

参考文献

[1] BRUCKER P, SCHLIE R. Job-shop scheduling with multi-purpose machines[J]. Computing, 1990, 45(4): 369-375.
[2] PEZZELLA F, MORGANTI G, CIASCHETTI G. A genetic algorithm for the Flexible Job-shop Scheduling Problem[J]. Computers & Operations Research, 2008, 35(10): 3202-3212.
[3] LIU Z, WANG J, ZHANG C, et al. A hybrid genetic-particle swarm algorithm based on multilevel neighbourhood structure for flexible job shop scheduling problem[J]. Computers & Operations Research, 2021, 135: 105431.
[4] WANG Z, HE M, WU J, et al. An improved MOEA/D for low-carbon many-objective flexible job shop scheduling problem[J]. Computers & Industrial Engineering, 2024, 188: 109926.
[5] 姜一啸, 吉卫喜, 何鑫,等基于改进非支配排序遗传算法的多目标柔性作业车间低碳调度[J]. 中国机械工程, 2022, 33(21): 2564-2577.
[6] SENG D, LI J, FANG X, et al. Low-Carbon Flexible Job-Shop Scheduling Based on Improved Nondominated Sorting Genetic Algorithm-Ⅱ[J]. International Journal of Simula-tion Modelling, 2018, 17: 712-723.
[7] MENG L, ZHANG C, ZHANG B, et al. MILP modeling and optimization of multi-objective flexible job shop scheduling problem with controllable processing times[J]. Swarm and Evolutionary Computation, 2023, 82: 101374.
[8] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.
[9] HAN B A, YANG J J. Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN[J]. IEEE Access, 2020, 8: 186474-186495.
[10] LEI K, GUO P, ZHAO W, et al. A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem[J]. Expert Systems with Applications, 2022, 205: 117796.
[11] CHEN R, YANG B, LI S, et al. A self-learning genetic algorithm based on reinforcement learning for flexible job-shop scheduling problem[J]. Computers & Industrial Engineering, 2020, 149: 106778.
[12] 李健, 李洹坤, 何鹏博,等.协同智能体强化学习算法的柔性作业车间调度方法研究[J]. 系统仿真学报,2024,36(11):2699-2711.
[13] 张凯, 毕利, 焦小刚. 集成强化学习算法的柔性作业车间调度问题研究[J]. 中国机械工程, 2023, 34(2): 201-207.
[14] SONG W, CHEN X, LI Q, et al. Flexible Job-Shop Scheduling via Graph Neural Network and Deep Reinforcement Learning[J]. IEEE Transactions on Industrial Informatics, 2023, 19(2): 1600-1610.
[15] LI R, GONG W, LU C. A reinforcement learning based RMOEA/D for bi-objective fuzzy flexible job shop scheduling[J]. Expert Systems with Applications, 2022, 203: 117380.
[16] BRANDIMARTE P. Routing and scheduling in a flexible job shop by tabu search[J]. Annals of Operations Research, 1993, 41(3): 157-183.
[17] 唐亮, 程峰, 吉卫喜,等. 改进ICA求解柔性作业车间插单重调度问题[J]. 计算机工程与应用, 2023, 59(21): 303-311.
[18] TIAN Y, XIANG X, ZHANG X, et al. Sampling Reference Points on the Pareto Fronts of Benchmark Multi-Objective Optimization Problems[M].[S.l.]:[s.n.], 2018.
[19] SCHOTT J R. Fault tolerant design using single and multicriteria genetic algorithm optimization[D]. Massachusetts: Massachusetts Institute of Technology,1995.
[20] ZITZLER E, THIELE L. Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach[J]. IEEE Transactions on Evolutionary Computation, 1999, 3(4): 257-271.

基于DQN的改进NSGA-Ⅱ求解多目标柔性作业车间调度问题

Solving multi-objective flexible job shop scheduling problems with an improved NSGA-Ⅱ based on DQN

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 14

编辑推荐

Metrics

[1]	夏良才, 陈仕军. 基于Dueling-DQN的协同进化算法求解绿色分布式异构柔性作业车间调度问题^*[J]. 现代制造工程, 2025, 538(7): 8-19.
[2]	金桥, 杨光锐, 王霄, 徐凌桦, 张芳. 基于A-TD3的码垛机器人轨迹规划^*[J]. 现代制造工程, 2025, 536(5): 42-52.
[3]	黄开启, 胡智强. 矿用自卸车互联油气悬架参数多目标优化[J]. 现代制造工程, 2025, 534(3): 124-131.
[4]	陈广玲, 张天赐, 付涛, 王林涛, 宋学官. 基于NSGA-Ⅱ的智能化电铲多目标最优挖掘轨迹规划[J]. 现代制造工程, 2024, 521(2): 142-149.
[5]	张念, 郑凯, 董兴辉, 柳亦兵. 基于NSGA-Ⅱ与CFD的H型垂直轴风力机翼型优化设计^*[J]. 现代制造工程, 2024, 531(12): 130-136.
[6]	李钊, 温承钦, 黄维忠, 朱海强, 覃丽燕, 周绍鹏, 郑玲. 考虑技能学习差异的多工人协作柔性车间调度^*[J]. 现代制造工程, 2024, 529(10): 9-15.
[7]	刘晋飞;程铖;李杰林;黄华;马学明. 数字孪生环境下考虑可靠性的多目标智能排序策略[J]. 现代制造工程, 2023, 516(9): 28-35.
[8]	陆文丽，黄伟，许恩永，陈萌竹，兰克俊. 基于响应面法的燃料电池商用车储氢系统多目标优化设计[J]. 现代制造工程, 2023, 512(5): 58-64.
[9]	夏建平，李尚强，部绍闵，杨庆福，高德优. 多目标U型顺序相依装配线平衡问题的改进NSGA-Ⅲ[J]. 现代制造工程, 2023, 519(12): 13-21.
[10]	赵鑫，刘晓勇，苏铁熊，任日娜，郭亚新. 基于NSGA-Ⅱ的柴油机主轴承组合结构可靠性提升措施研究[J]. 现代制造工程, 2022, 507(12): 100-110.
[11]	欧学昊,周长城,赵昱,李颂. 卵形截面钢丝螺旋弹簧切应力分析及多目标优化设计[J]. 现代制造工程, 2021, 495(12): 1-6.
[12]	张守京，王彦亭. 基于改进NSGA2的柔性车间多目标智能调度问题研究[J]. 现代制造工程, 2020, 480(9): 23-31.
[13]	李强，张科，康鹏桂，孔蕾，王鑫龙. 基于NSGA-Ⅱ的云制造物流服务方案设计研究[J]. 现代制造工程, 2020, 476(5): 53-58.
[14]	王鹤,滕瑛瑶. 金属玻璃电火花加工多目标优化研究[J]. 现代制造工程, 2020, 481(10): 97-101.