基于深度强化学习的高铁枢纽站进站进路自动触发方法Automatic triggering method for hub station arrival routes based on deep reinforcement learning
王振东,李得伟,张芸鹏,赵小莉,赵宏涛,姚帅,潘帅
摘要(Abstract):
针对先到先服务(FCFS)策略难以有效消解枢纽站进站进路冲突并抑制列车晚点的不足,基于深度强化学习,提出1种以总晚点抑制和晚点均衡性为导向的多车进站进路序贯自动触发方法,旨在运行计划执行阶段有效释放枢纽站接车能力。结合调度集中系统信息基础和信号系统控车机制分析,将多阶段进路触发建模为马尔可夫决策过程;优化设计优势行为-批评算法架构,通过多步时序差分算法进行网络参数更新,以适应奖励延迟环境并满足奖励溯源需求;运用历史调度场景数据开展多轮次训练和验证。结果表明:该自动触发方法具有收敛性,多个调度场景的单次决策事件总晚点量均小于20 min;相比基于人工决策的FCFS策略,该方法在3类高频调度场景下的总晚点量均有明显下降;短数据片段训练后,单车晚点时间的四分位距显著缩小,分布均衡性进一步改善;优化目标调节后,对关键线路和重点列车的进路保障能力进一步提升;训练后所建模型对枢纽站相同接车咽喉区的不同决策事件均有泛化能力,单次决策时间小于700 ms。
关键词(KeyWords): 行车调度;调度集中;进路触发;进路冲突缓解;列车运行晚点;深度强化学习
基金项目(Foundation): 国家自然科学基金资助项目(72471023);; 国家重点研发计划项目(2024YFA1012904);; 中央高校基本科研业务费专项资金资助项目(2025JBZX077);; 高速铁路与城轨交通系统技术国家工程研究中心开放项目(2025YJ272)
作者(Author): 王振东,李得伟,张芸鹏,赵小莉,赵宏涛,姚帅,潘帅
参考文献(References):
- [1]TJ/DW 208—2019智能调度集中系统暂行技术条件[S].TJ/DW 208—2019 Provisional technical conditions for intelligent centralized scheduling system[S].
- [2]Li Zhongcan, Huang Ping, Wen Chao, et al. Prediction of train arrival delays considering route conflicts at multi-line stations[J]. Transportation Research Part C:Emerging Technologies, 2022, 138:103606.
- [3]倪少权,龚梓烨,宋宗莹.基于列车群组运行的双线重载铁路群组计划优化研究[J].铁道科学与工程学报,2025,22(6):2479-2492.Ni Shaoquan, Gong Ziye, Song Zongying. Optimization study of group plan on double-track heavy-haul railways based on group train operation[J]. Journal of Railway Science and Engineering, 2025, 22(6):2479-2492.
- [4]向江海,赵军,张思宇,等.铁路编组站到发场技术作业计划编制优化方法研究[J].铁道学报,2025,47(2):11-23.Xiang Jianghai, Zhao Jun, Zhang Siyu, et al. Optimization approach for design of technical operation plan for arrivaldeparture yard of railway marshalling station[J]. Journal of the China Railway Society, 2025, 47(2):11-23.
- [5]叶文龙,徐小明,马静,等.铁路车站列车运行仿真和进路优化算法研究[J].北京交通大学学报,2024,48(6):1-11.Ye Wenlong, Xu Xiaoming, Ma Jing, et al. Algorithm research on train movement simulation and routing optimization in railway stations[J]. Journal of Beijing Jiaotong University, 2024, 48(6):1-11.
- [6]Zhao Yaqiong, Li Dewei, Yin Yonghao, et al. Integrated optimization of demand-driven timetable, train formation plan and rolling stock circulation with variable running times and dwell times[J]. Transportation Research Part E:Logistics and Transportation Review, 2023, 171:103035.
- [7]Yao Zhiyuan, Nie Lei, Fu Huiling. Railway line planning with passenger routing:direct-service network representations and a two-phase solution approach[J]. Transportation Research Part B:Methodological, 2024, 186:102989.
- [8]Jamili A. Computation of practical capacity in single-track railway lines based on computing the minimum buffer times[J].Journal of Rail Transport Planning&Management, 2018, 8(2):91-102.
- [9]田锐,孟令云,王维,等.基于深度Q网络的高速铁路车站到发线运用冲突调整研究[J].铁道学报,2025,47(3):1-9.Tian Rui, Meng Lingyun, Wang Wei, et al. Research on auto conflict-resolution of arrival-departure track utilization in large high-speed railway stations based on deep Q-Network[J]. Journal of the China Railway Society, 2025, 47(3):1-9.
- [10]周敏,顾灏璇,董海荣,等.考虑动车组周转和到发线运用的高铁列车运行多计划协同调整[J].自动化学报,2024,50(8):1577-1588.Zhou Min, Gu Haoxuan, Dong Hairong, et al. Multi-plan collaborative rescheduling of high-speed train operation considering the utilization of rolling stock and arrival and departure tracks[J]. Acta Automatica Sinica, 2024, 50(8):1577-1588.
- [11]张英贵,周婧盈,王宇航.双因素扰动下高铁车站到发线运用调整优化[J].铁道科学与工程学报,2026,23(1):14-26.Zhang Yinggui, Zhou Jingying, Wang Yuhang. Optimization method for arrival and departure track utilization rescheduling in high-speed railway stations under dual-factor interference[J]. Journal of Railway Science and Engineering, 2026, 23(1):14-26.
- [12]李登辉,彭其渊,赵军,等.基于“抽线-插线”的高速铁路网络列车运行调整优化方法[J].铁道学报,2024,46(12):8-21.Li Denghui, Peng Qiyuan, Zhao Jun, et al. Optimization method based on idea of“timetable deletion-insertion” for train rescheduling in a high-speed railway network[J]. Journal of the China Railway Society, 2024, 46(12):8-21.
- [13]Bai Lijie, Bourdeaud’huy T, Castelain E, et al. Sub-problems interaction analysis-based three-level decomposition algorithm for real-time train scheduling and routing problems in railway stations[M]//Proceedings of the third international conference on electrical and information technologies for rail transportation(EITRT)2017. Singapore:Springer Singapore, 2018:517-523.
- [14]Zhang Hengkai, Liu Xiaoyu, Sun Dingshan, et al. Integrated reinforcement learning and optimization for railway timetable rescheduling[J]. IFAC-PapersOnLine, 2024, 58(10):310-315.
- [15]Zhang Liqing, Leong hou U, Zhou Mingliang, et al. An intelligent train operation method based on event-driven deep reinforcement learning[J]. IEEE Transactions on Industrial Informatics, 2022, 18(10):6973-6980.
- [16]Lin Boliang, Shen Yaoming, Wang Zhongkai, et al. An iterative improvement approach for high-speed train maintenance scheduling[J]. Transportation Research Part B:Methodological, 2023, 173:292-312.
- [17]中国铁路总公司.TG/CW 103—2017铁路运输调度规则:高速铁路部分[M].北京:中国铁道出版社,2017.China Railway Corporation. TG/CW 103—2017 Railway transportation dispatching rules:high-speed railway[M]. Beijing:China Railway Publishing House, 2017.
- [18]Q/CR 744—2020 CTCS-3级列控车载设备技术规范[S].Q/CR 744—2020 Technical specification for CTCS-3 onboard ATP[S].
- [19]周敏,顾灏璇,董海荣,等.考虑动车组周转和到发线运用的高铁列车运行多计划协同调整[J].自动化学报,2024,50(8):1577-1588Zhou Min, Gu Haoxuan, Dong Hairong, et al. Multi-plan collaborative rescheduling of high-speed train operation considering the utilization of rolling stock and arrival and departure tracks[J]. Acta Automatica Sinica, 2024, 50(8):1577-1588.
- [20]Mnih V, Kavukcuoglu K, Silver D, et al. Playing atari with deep reinforcement learning[PP/OL]. arXiv.(2013-12-19)[2025-07-10]. https://arxiv.org/abs/1312.5602.
- [21]Mnih V, Badia A P, Mirza M, et al. Asynchronous methods for deep reinforcement learning[PP/OL]. arXiv.(2016-02-04)[2025-07-10]. https://arxiv.org/abs/1602.01783.
- [22]Xu Wei, Zhao Chen, Dai Xingyuan, et al. Parallel dispatching:an ACP-based high-speed railway intelligent dispatching system[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(1):1146-1160.