Deep reinforcement learning for production intelligent scheduling of wind turbine extrusion panels

doi:10.16731/j.cnki.1671-3133.2025.01.003

Abstract

Abstract: In order to solve the wind turbine extrusion panel production scheduling problem with complex features such as packaging order flush and product changeover adjustment,a multi-objective collaborative optimization model was constructed to maximize the average utilization rate of the current starting equipment and maximize the order fulfillment rate;the wind turbine extrusion panel production scheduling problem was transformed into a Markov sequence decision-making problem,and 10 different scheduling strategies were designed as the action space,and the appropriate state features and reward functions were refined; a scheduling algorithm based on Dueling Double Deep Q Network (D3QN) was proposed. The effectiveness of the proposed algorithm was verified by comparing with Double DQN and Dueling DQN algorithms through simulation tests on actual data of an enterprise;and the objective values obtained by four different solution methods under 10 algorithms were compared,which verified that the proposed improved D3QN algorithm can get a high-quality solution to the problem,providing an intelligent method and reference for the production scheduling of wind turbine extrusion panel manufacturing enterprises.

Key words: wind power, extrusion panels, production scheduling, deep reinforcement learning, D3QN algorithm

CLC Number:

TB497

YANG Fenghai, YANG Xiaoying, PEI Zhijie, WU Yaqi, ZHANG Zhiwei. Deep reinforcement learning for production intelligent scheduling of wind turbine extrusion panels[J]. Modern Manufacturing Engineering, 2025, 532(1): 23-32.

References

[1] 李成良,杨超,倪爱清,等.复合材料在大型风电叶片上的应用与发展[J].复合材料学报,2023,40(3):1274-1284.
[2] 卫立江,邱锐磊,王乾武,等.大风电叶片拉挤复合材料研究进展[J/OL].复合材料科学与工程:1-6.(2023-10-26)[2024-04-24].http://kns.cnki.net/kcms/detail/10.1683.TU.20231026.1431.004.html.
[3] NEUFELD J S,GUPTA J N D,BUSCHER U.A comprehensive review of flowshop group scheduling literature[J].Computers & Operations Research,2016,70:56-74.
[4] LIN C W,LIN Y K,HSIEH H T.Ant colony optimization for unrelated parallel machine scheduling[J].The International Journal of Advanced Manufacturing Technology,2013,67(1/2/3/4):35-45.
[5] YING K C,LEE Z J,LIN S W.Makespan minimization for scheduling unrelated parallel machines with setup times[J].Journal of Intelligent Manufacturing,2012,23(5):1795-1803.
[6] LIU S,LIU X,PEI J,et al.Parallel-batching machines scheduling problem with a truncated time-dependent learning effect via a hybrid CS-JADE algorithm[J].Optimization Methods and Software,2019(3):1-26.
[7] 郑堃,练志伟,顾新艳,等.采用改进两点交叉算子的改进自适应遗传算法求解不相关并行机混合流水车间调度问题[J].中国机械工程,2023,34(14):1647-1658,1671.
[8] 荀洪凯,陶翼飞,张源,等.多目标启发式狼群算法求解不相关并行机分批调度问题[J].信息与控制,2023,52(1):93-103,114.
[9] 宋君乐,陶翼飞,张源,等.基于两阶段多目标BBO算法的并行机分批调度问题研究[J/OL].控制工程:1-13.(2024-02-26)[2024-04-24].https://doi.org/10.14107/j.cnki.kzgc.20230122.
[10] 高佳,谭园园,李冬,等.离散粒子群算法求解带役龄维护的不相关并行机调度问题 [J/OL].计算机集成制造系统:1-18.(2023-12-28)[2024-04-24].https://doi.org/10.13196/j.cims.2023.0450.
[11] 朱先萌,姜兆亮,魏清月,等.多品种小批量产品冲压计划排程多目标优化[J].计算机集成制造系统,2017,23(9):1907-1916.
[12] 鲁佳俊,高亮,侯国祥.基于果蝇优化算法的船舶管加工车间多目标生产调度研究[J].现代制造工程,2021(8):8-19.
[13] AMALLYNDA I,SANTOSA B.Solving multi-objective Modified Distributed Parallel Machine and Assembly Scheduling Problem(MDPMASP)with eligibility constraints using metaheuristics[J].Production & Manufacturing Research,2022,10(1):198-225.
[14] KONG M,WANG W,DEVECI M,et al.A novel carbon reduction engineering method-based deep Q-learning algorithm for energy-efficient scheduling on a single batch-processing machine in semiconductor manufacturing[J/OL].International Journal of Production Research,2023:1-24[2024-04-24] DOI:10.1080/00207543.2023.2252932.
[15] LI J L,SHI W S,ZHANG N,et al.Delay-Aware VNF Scheduling:A Reinforcement Learning Approach With Variable Action Set[J].IEEE Transactions on Cognitive Communications and Networking,2021,7(1):304-318.
[16] 秦保新,张羽霄,吴思锐,等.基于改进D3QN的煤炭码头卸车排产智能优化方法[J].系统仿真学报,2024,36(3):770-781.
[17] 吴昊泽,李艳武,谢辉.改进PPO算法求解柔性作业车间调度问题[J/OL].计算机集成制造系统:1-20.(2023-06-02)[2024-04-24].http://kns.cnki.net/kcms/detail/11.5946.TP.20230601.0942.006.html.
[18] VAN H,GUEZ A,SILVER D.Deep reinforcement learning with double Q-learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Arizona:AAAI Press,2016.DOI:10.48550/arXiv.1509.06461.
[19] LUO Q,DENG Q,GONG G,et al.An efficient memetic algorithm for distributed flexible job shop scheduling problem with transfers[J].Expert Systems with Applications,2020,160:113721.