A Combination Model for Connected and Autonomous Vehicles
Lane-changing Decision-making Under Multi Connectivity Range

doi:10.16097/j.cnki.1009-6744.2023

Abstract

Abstract: In order to improve the lane-changing efficiency of intelligent connected vehicles (ICV) under different network connection ranges, combined with deep reinforcement learning and molecular dynamics theory, a double deep Q network lane-changing decision model integrating the masking mechanism and attention mechanism (MAQ) is proposed. Firstly, in the Simulation of Urban Mobility (SUMO) simulation environment, the driving status information of connected vehicles and human drive vehicles (HDV) within the network range is collected. Secondly, the MAQ model is built, the mask mechanism and attention mechanism are adopted to achieve fixed model input size and displacement invariance. Thirdly, in order to quantify the degree of influence between vehicles, the relative speed and the relative position between vehicles are used as parameters, and the molecular dynamics theory is used to give weights to HDV information within the connectivity range. Finally, different lane- changing decision models and weighting methods are compared in different traffic density simulation environments. The effect of lane change decision is tested under different connectivity ranges (80~330 meters, with an interval of 50 meters). The simulation results show that, taking 40 HDVs and a 100-meter connectivity range as an example, the MAQ model has a 90.2% improvement in fitting accuracy compared with the DeepSet-Q model; compared with the linear weighting method, the molecular dynamics weighting method increases the total reward value by 5.5% , and the average speed of ICV by 4.4%; with the expansion of the connectivity range, the average speed of ICV shows a change rule of first increasing, then decreasing, and then tending to be stable.

Key words: intelligent transportation, intelligent connected vehicles, lane change decision, deep reinforcement learning, molecular dynamics, connectivity range

摘要： 为提升不同网联范围下智能网联车(Intelligent Connected Vehicles, ICV)的换道效率，结合深度强化学习和分子动力学理论，提出一种融合掩码机制和注意力机制的双深度Q网络(MaskAttention-DDQN, MAQ)换道决策模型。首先，在SUMO (Simulation of Urban Mobility)仿真环境中采集网联范围内ICV及人工驾驶车辆(Human Drive Vehicles, HDV)的行驶状态信息。其次，搭建MAQ模型，采用掩码机制和注意力机制方法，实现固定模型输入大小，以及实现置换不变性。第三，为实现车辆间影响程度的数值化，以车辆间相对速度和相对位置为参数，使用分子动力学理论为网联范围内HDV信息赋予权重。最后，分别在不同交通密度仿真环境中对不同换道决策模型和赋权方法进行对比，并测试ICV在不同网联范围(80~330 m，以50 m为间隔)下的换道决策效果。仿真结果表明，以40辆HDV、100m网联范围为例，MAQ模型比DeepSet-Q模型拟合精度提高了90.2%；分子动力学赋权方法相比线性权重赋权方法总奖励值提高了5.5%，ICV平均车速提高了4.8%；ICV平均车速随着网联范围的扩大，呈现出先增大、再减小、后趋于平稳的变化规律。

关键词: 智能交通, 智能网联车, 换道决策, 深度强化学习, 分子动力学, 网联范围

CLC Number:

U495

ZHAO Jian-dong, HE Xiao-yu, YU Zhi-xin, HAN Ming-min. A Combination Model for Connected and Autonomous Vehicles Lane-changing Decision-making Under Multi Connectivity Range[J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(1): 77-85.

赵建东, 贺晓宇, 余智鑫, 韩明敏. 多网联范围下的智能网联车换道决策组合模型研究[J]. 交通运输系统工程与信息, 2023, 23(1): 77-85.

Add to citation manager EndNote|Ris|BibTeX

URL: http://www.tseit.org.cn/EN/10.16097/j.cnki.1009-6744.2023

http://www.tseit.org.cn/EN/Y2023/V23/I1/77

References

[1] 冯焕焕, 邓建华, 葛婷. 引入驾驶风格的熵权法多属性换道决策模型[J]. 交通运输系统工程与信息, 2020, 20 (2): 139-144. [FENG H H, DENG J H, GE T. Multiattributes lane-changing decision model[J]. Transportation System Engineering and Information, 2020, 20(2): 139-144.]

[2] YU H, TSENG H E, LANGARI R. A human-like game theory-based controller for automatic lane changing[J]. Transportation Research Part C: Emerging Technologies, 2018, 88: 140-158.

[3] 赵建东, 赵志敏, 屈云超, 等. 轨迹数据驱动的车辆换道意图识别研究[J]. 交通运输系统工程与信息, 2022, 22(4): 63-71. [ZHAO J D, ZHAO Z M, QU Y C, et al, Vehicle lane change intention recognition driven by trajectory data[J]. Transportation System Engineering and Information, 2022, 22(4): 63-71.]

[4] 陈力, 殷时蓉, 罗天洪, 等. 基于BP神经网络的智能车辆换道决策模型研究[J]. 汽车工程学报, 2022, 12(1): 83-89. [CHEN L, YIN S R, LUO T H, et al. Research on lane-changing decision model for intelligent vehicles based on BP neural network[J]. Chinese Journal of Automotive Engineering, 2022, 12(1): 83-89.]

[5] 裴晓飞, 莫烁杰, 陈祯福, 等. 基于 TD3 算法的人机混驾交通环境自动驾驶汽车换道研究[J]. 中国公路学报, 2021, 34(11): 246-254. [PEI X F, MO S J, CHEN Z F, et al. Lane changing of autonomous vehicle based on TD3 algorithm in human-machine hybrid driving environment[J]. China Journal of Highway and Transport, 2021, 34(11): 246-254.]

[6] HUEGLE M, KALWEIT G, MIRCHEVSKA B, et al. Dynamic input for deep reinforcement learning in autonomous driving[C]. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2019: 7566-7573.

[7] DONG J Q, CHEN S K, LI Y J, et al. Space-weighted information fusion using deep reinforcement learning: The context of tactical control of lane-changing autonomous vehicles and connectivity range assessment [J]. Transportation Research Part C: Emerging Technologies, 2021, 128: 103192.

[8] 陈晨. 智能网联条件下高速公路事故路段车辆协同通行模型 [D]. 广州: 华南理工大学, 2021. [CHEN C. Cooperative traffic model of vehicles on highways accident section under intelligent network connection[D]. Guangzhou: South China University of Technology, 2021.]

[9] 宗芳, 王猛, 贺正冰. 考虑多车影响的分子动力学智能网联跟驰模型[J]. 交通运输系统工程与信息, 2022, 22 (1): 37-48. [ZONG F, WANG M, HE Z B. A molecular dynamics-based car-following model for connected and automated vehicles considering impact of multiple vehicles[J]. Transportation System Engineering and Information, 2022, 22(1): 37-48.]

[10] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30(1): 5998- 6008.

[11] 王畅, 付锐, 张琼, 等. 换道预警系统中参数TTC 特性研究[J]. 中国公路学报, 2015, 28(8): 91-100, 108. [WANG C, FU R, ZHANG Q, et al. Research on parameter TTC characteristics of lane change warning system[J]. China Journal of Highway and Transport, 2015, 28(8): 91-100, 108.]

[12] HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double Q-learning [C]. New York:Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 2016.

[13] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J]. Arxiv Preprint Arxiv: 1409.0473, 2014.

[14] 石蕊. 信号控制交叉口行车场建立及车辆通行行为优化[D]. 长春: 吉林大学, 2021. [SHI R. The construction of risk field and optimization of driving behaviors for signalized intersections[D]. Changchun: Jilin University, 2021.]

[15] KRAJZEWICZ D, ERDMANN J, BEHRISCH M, et al. Recent development and applications of SUMOsimulation of urban mobility[J]. International Journal on Advances in Systems and Measurements, 2012, 5(3/4): 128-138.

[16] TREIBER M, KESTING A. Traffic flow dynamics[J]. Traffic Flow Dynamics: Data, Models and Simulation, Springer-Verlag Berlin Heidelberg, 2013: 983-1000.

[17] ERDMANN J. SUMO's lane-changing model[M]// Gereon M, Modeling Mobility with Open Data. Berlin, Germany: Springer, Cham, 2015: 105-123.

[18] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Humanlevel control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.

A Combination Model for Connected and Autonomous Vehicles Lane-changing Decision-making Under Multi Connectivity Range

多网联范围下的智能网联车换道决策组合模型研究

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	LIU Run-kun, YU Hai-yang, CHENG Meng-yue, REN Yi-long. Platoon Control Strategy for Connected and Automated Vehicles Under Intermittent Communication Failures [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(2): 54-66.
[2]	ZHANG Peng, GU Yun-xiang, SUN Chao, LI Wen-quan. Collaborative Optimization Model of Truck Speed and Signal Timing Based on Minimum Fuel Consumption [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(2): 84-91.
[3]	LU Tong, YU Zu-jun, GUO Bao-qing, RUAN Tao. Semantic Segmentation of Railway Scene Based on Reticulated Multi-scale and Bidirectional Channel Attention [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(2): 233-241.
[4]	TANG Jin-jun, FU Qiang, WANG Cheng-cheng, YUAN Chen, ZHANG Xuan, XU Chang-jing. Multi-objective Optimization of Variable Speed Limit Control Strategy on Expressway [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(2): 252-261.
[5]	ZHANG Bing-tao, CHANG Wen-wen, LI Xiu-lan. Fatigue Driving Detection Based on Spatial-temporal Electroencephalogram Features and Parallel Neural Networks [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(2): 315-325.
[6]	SUN Wei, ZHANG Meng-ya, MA Cheng-yuan, ZHU Ji-chen, YANG Xiao-guang. Coordination of Signal and Vehicle Trajectory at Intersections for Mixed Traffic Flow [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(1): 97-105.
[7]	CHEN Xi-qun, ZHU Yi-zhang, LV Chao-feng. Signal Phase and Timing Optimization Method for Intersection Based on Hybrid Proximal Policy Optimization [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(1): 106-113.
[8]	HE Wen-wu, PEI Bo-yu, LI Ya-ting, LIU Xiao-yu, XU Shao-bing. Traffic Flow Forecasting Based on Bi-directional Adaptive Gating Graph Convolutional Networks [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(1): 187-197.
[9]	HAO Wei, GONG Ya-xin, ZHANG Zhao-lei, YI Ke-fu. Cooperative Merging Strategy for Freeway Ramp in a Mixed Traffic Environment [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(1): 224-235.
[10]	XIE Xin-lin, LUO Chen-yan, XU Xin-ying, XIE Gang. Dual Attention Guided Cross-layer Optimized Traffic Scene Semantic Segmentation [J]. Journal of Transportation Systems Engineering and Information Technology, 2023, 23(1): 236-244.
[11]	YANG Liu, YANG Ying, SONG Yun-zhou, ZHANG Yu. A Review of Influencing Factors and Identification Methods of Driver Stress [J]. Journal of Transportation Systems Engineering and Information Technology, 2022, 22(6): 40-50.
[12]	XIANG Xin-jian, HU Hai-bin, YAO Jia-na, DING Yi, ZHENG Yong-ping, JIN Li. A Road Small Target Tracking Approach Based on Object-shadow Matching Algorithm [J]. Journal of Transportation Systems Engineering and Information Technology, 2022, 22(6): 85-94.
[13]	CHEN Zhi-jun, HU Jun-nan, LENG Yao, QIAN Chuang, WU Chao-zhong. Intelligent Vehicle Target Fast Recognition Based on Lightweight Network and Attention Mechanism [J]. Journal of Transportation Systems Engineering and Information Technology, 2022, 22(6): 105-113.
[14]	ZHANG Wen-juan, YANG Hao-zhe, ZHANG Bin, LI Xiu-jie. Short-time Passenger Flow Prediction Model of Urban Rail Transit Considering Multi-timescale Features [J]. Journal of Transportation Systems Engineering and Information Technology, 2022, 22(6): 212-223.
[15]	PENG Hong-qin , ZHANG Guo-wu. Theoretical Integration and Practice of System Engineering and Transportation in the New Era [J]. Journal of Transportation Systems Engineering and Information Technology, 2022, 22(5): 1-6.