基于深度强化学习的单线路公交动态驻站控制策略研究

doi:10.16097/j.cnki.1009-6744.2024.05.016

交通运输系统工程与信息 ›› 2024, Vol. 24 ›› Issue (5): 173-184.DOI: 10.16097/j.cnki.1009-6744.2024.05.016

基于深度强化学习的单线路公交动态驻站控制策略研究

刘东^1a，张大鹏^1b，万芸^1a，肖峰^*2

1. 西南财经大学，a. 工商管理学院，b. 管理科学与工程学院，成都611130；2. 四川大学，商学院，成都 610065

收稿日期:2024-07-09 修回日期:2024-08-02 接受日期:2024-08-07 出版日期:2024-10-25 发布日期:2024-10-22
作者简介:刘东(1991- )，男，四川资阳人，博士生。
基金资助:
国家自然科学基金(72301217, 72025104)；四川省自然科学基金(2024NSFSC1055)。

Single-line Bus Operations Dynamic Holding Control Strategy Based on Deep Reinforcement Learning

LIU Dong^1a, ZHANG Dapeng^1b, WAN Yun^1a, XIAO Feng^*2

1a. School of Business Administration, 1b. School of Management Science and Engineering, Southwestern University of Finance and Economics, Chengdu 611130, China; 2. Business School, Sichuan University, Chengdu 610065, China

Received:2024-07-09 Revised:2024-08-02 Accepted:2024-08-07 Online:2024-10-25 Published:2024-10-22
Supported by:
National Natural Science Foundation of China (72301217, 72025104)；Natural Science Foundation of Sichuan Province, China (2024NSFSC1055)。

摘要/Abstract

摘要： 公交运行中，车辆车头时距波动过大会导致公交系统出现串车等运行不稳定现象，针对该问题，本文提出一种基于深度强化学习的动态驻站控制策略，实现公交系统的稳定运行，以及避免出现串车问题。首先，构造线形公交系统，并确定车辆运行和乘客行为规则；然后，介绍基于深度强化学习建立动态控制方法，定义强化学习框架的各要素，并开发事件驱动的模拟器环境，训练和测试智能体；最后，利用仿真模拟对所提方法与基准方法进行大量的数值实验，选取不同评价指标进行对比分析，并实施敏感性分析。实验结果发现，本文方法实现了最稳定的车辆运行轨迹和最小的载客分散度；在车头时距变动上，比无控制策略、基于时刻表控制策略和基于车头时距控制策略分别降低61.90%、60.98%和37.98%；在平均等待时间上，分别降低28.36%、26.53%和23.61%。此外，所提方法在不同行驶时间变异性和车头时距情景下，具有很强的鲁棒性。

关键词: 智能交通, 动态驻站控制, 深度强化学习, 公交系统, 事件驱动

Abstract: Large headway and fluctuations in bus operations can lead to instability of the bus operation system, such as the bus bunching phenomena. This paper proposes a dynamic holding control strategy based on deep reinforcement learning to improve the stability of bus system operations and avoid bus bunching. A linear bus system is established, and the operating rules for vehicles and passenger behavior are defined. Then, a dynamic control method is introduced based on deep reinforcement learning, the elements of the reinforcement learning framework are defined, and an eventdriven simulator environment is developed to train and test the agents. Extensive simulation experiments are conducted to compare the proposed method with traditional methods. Various evaluation metrics are selected for comparative analysis, and the sensitivity analysis is also performed. The experimental results show that the proposed method achieves the most stable vehicle trajectories and the smallest passenger occupancy dispersion. The headway variation was reduced respectively by 61.90%, 60.98%, and 37.98% compared to the no control strategy, the schedule-based control strategy, and the headway-based control strategy. The average waiting time was reduced by 28.36%, 26.53%, and 23.61% compared to the aforementioned strategies. The proposed method also demonstrates strong robustness under varying travel time variability and headway conditions.

Key words: intelligent transportation, dynamic holding control, deep reinforcement learning, bus system, event-driven

中图分类号:

U495

刘东, 张大鹏, 万芸, 肖峰. 基于深度强化学习的单线路公交动态驻站控制策略研究[J]. 交通运输系统工程与信息, 2024, 24(5): 173-184.

LIU Dong, ZHANG Dapeng, WAN Yun, XIAO Feng. Single-line Bus Operations Dynamic Holding Control Strategy Based on Deep Reinforcement Learning[J]. Journal of Transportation Systems Engineering and Information Technology, 2024, 24(5): 173-184.

导出引用管理器 EndNote|Ris|BibTeX

链接本文: http://www.tseit.org.cn/CN/10.16097/j.cnki.1009-6744.2024.05.016

http://www.tseit.org.cn/CN/Y2024/V24/I5/173

参考文献

[1] REZAZADA M, NASSIR N, TANIN E, et al. Bus bunching: A comprehensive review from demand, supply, and decision-making perspectives[J]. Transport Reviews, 2024, 44(4): 1-25.

[2] COMI A, SASSANO M, VALENTINI A. Monitoring and controlling real-time bus services: A reinforcement learning procedure for eliminating bus bunching[J]. Transportation Research Procedia, 2022, 62: 302-309.

[3] GKIOTSALITIS K. Bus operations optimization: A literature review on bus holding, rescheduling and stopskipping[C]. Washington: In 99th Transportation Research Board (TRB) Annual Meeting, 2020.

[4] 金梦宇, 何胜学, 张思潮. 考虑尾气污染的公交跳站策略研究[J]. 华东交通大学学报, 2020, 37(5): 53-59. [JIN M Y, HE S X, ZHANG S C. Study on bus stop-skip strategy considering bus emission[J]. Journal of East China Jiaotong University, 2020, 37(5): 53-59.]

[5] LIU M, ZHAO J, HOOGENDOORN S P, et al. An optimal control approach of integrating traffic signals and cooperative vehicle trajectories at intersections[J]. Transportmetrica B: Transport Dynamics, 2022, 10(1): 971-987.

[6] 翁剑成, 李文杰, 林鹏飞, 等. 考虑车速调控和乘车引导的公交运行优化控制策略[J]. 交通运输系统工程与信息, 2023, 23(6): 165-175. [WENG J C, LI W J, LIN P F, et al. Bus operation optimization control strategy considering speed regulation and boarding guidance[J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(6): 165-175.]

[7] HE S X, LIANG S D, DONG J, et al. A holding strategy to resist bus bunching with dynamic target headway[J]. Computers & Industrial Engineering, 2020,140: 106237.

[8] 姜瑞森, 胡大伟, 孙倩, 等. 基于实时车辆信息共享的公交动态控制策略[J]. 长安大学学报(自然科学版), 2023, 43(6): 95-105. [JIANG R S, HU D W, SUN Q, et al. Dynamic control strategy for public transportation based on real-time information sharing[J]. Journal of Chang'an University (Natural Science Edition), 2023, 43 (6): 95-105.]

[9] 黄青霞, 贾斌, 强生杰, 等. 基于驻站和限流的组合公交控制策略研究[J]. 交通运输系统工程与信息, 2018, 18(4): 103-109. [HUANG Q X, JIA B, QIANG S J, et al. Integrated bus control strategy considering holding and limited-boarding[J]. Journal of Transportation Systems Engineering and Information Technology, 2018, 18(4): 103-109.]

[10] SIRMATEL I I, GEROLIMINIS N. Mixed logical dynamical modeling and hybrid model predictive control of public transport operations[J]. Transportation Research Part B: Methodological, 2018, 114: 325-345.

[11] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Humanlevel control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.

[12] YING C S, CHOW A H, CHIN K S. An actor-critic deep reinforcement learning approach for metro train scheduling with rolling stock circulation under stochastic demand[J]. Transportation Research Part B: Methodological, 2020, 140: 210-235.[13] ALESIANI F, GKIOTSALITIS K. Reinforcement learningbased bus holding for high-frequency services[C]. Maui: International Conference on Intelligent Transportation Systems, 2018.

[14] WANG J, SUN L. Dynamic holding control to avoid bus bunching: A multi-agent deep reinforcement learning framework[J]. Transportation Research Part C: Emerging Technologies, 2020, 116: 102661.

[15] FU L, YANG X. Design and implementation of bus-holding control strategies with real-time information[J]. Transportation Research Record, 2002, 1791(1): 6-12.

基于深度强化学习的单线路公交动态驻站控制策略研究

Single-line Bus Operations Dynamic Holding Control Strategy Based on Deep Reinforcement Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	李哲, 苟杨扬, 李震尧, 李敖, 岑威, 高建平. 车联网公交系统动态时空优先控制研究[J]. 交通运输系统工程与信息, 2024, 24(5): 56-64.
[2]	刘淼淼, 刘晓晨, 朱明月, 魏泽平, 邓辉, 姚民坤, 吴思霖, 李昂, 石赞, 龚筱萸. 基于动态轨迹规划的自动驾驶车辆协同换道方法[J]. 交通运输系统工程与信息, 2024, 24(5): 65-78.
[3]	胡正华, 周继彪, 毛新华, 张敏捷. 基于图像序列分析的城市道路交通事故预测[J]. 交通运输系统工程与信息, 2024, 24(5): 91-102.
[4]	刘美岐, 金楷然, 李雅澜, 郭戈. 城市道路智能网联车辆轨迹鲁棒控制方法[J]. 交通运输系统工程与信息, 2024, 24(4): 31-40.
[5]	温惠英, 张昕怡, 黄俊达, 许鹏鹏. 考虑动态交互作用的智能车辆轨迹预测[J]. 交通运输系统工程与信息, 2024, 24(4): 60-68.
[6]	赵朔, 奇格奇, 李培豪, 关伟. 基于脑电通道注意力机制的驾驶行为识别研究[J]. 交通运输系统工程与信息, 2024, 24(4): 283-291.
[7]	马千里, 王通, 邵帅, 贾鹏. 路测环境下无人驾驶小巴复杂场景运行评价[J]. 交通运输系统工程与信息, 2024, 24(4): 300-310.
[8]	董红召, 杨嘉炜, 全程. 车联网下公交专用道复用的动态清空控制方法[J]. 交通运输系统工程与信息, 2024, 24(3): 12-20.
[9]	刘江, 张楚, 蔡伯根, 王剑, 陆德彪. 基于样本增强的列车卫星定位伪距欺骗检测方法[J]. 交通运输系统工程与信息, 2024, 24(3): 32-42.
[10]	李建市, 徐友春, 齐尧, 谢德胜, 李华. 起伏道路条件下无人车纵向速度控制方法研究[J]. 交通运输系统工程与信息, 2024, 24(3): 43-52.
[11]	张金雷, 杨健, 刘晓冰, 陈瑶, 杨立兴, 高自友. 基于计算机视觉的轨道交通站内火灾检测与定位[J]. 交通运输系统工程与信息, 2024, 24(3): 53-63.
[12]	陈喜群, 朱奕璋, 谢宁珂, 耿茂思, 吕朝锋. 基于异构多智能体自注意力网络的路网信号协调顺序优化方法[J]. 交通运输系统工程与信息, 2024, 24(3): 114-126.
[13]	辛琪, 王嘉琪, 杨文科, 徐猛, 袁伟. 信控路段混行交通生态驾驶深度强化学习模型[J]. 交通运输系统工程与信息, 2024, 24(3): 127-139.
[14]	倪浩原, 余贵珍, 李涵, 陈鹏, 刘喜, 王文达. 露天矿作业区无人矿车协同通行决策方法研究[J]. 交通运输系统工程与信息, 2024, 24(3): 277-289.
[15]	张磊, 彭金栓, 陈晓利. 接管场景驱动下非驾驶任务对驾驶人视觉与生理特征的影响[J]. 交通运输系统工程与信息, 2024, 24(3): 290-298.