关键词: 交通工程, 非机动车行为, 强化学习, 生成对抗模仿学习, 自动驾驶测试, 微观交通仿真

Abstract: In order to accurately reproduce the interaction behavior of bicycles to meet the needs of autonomous driving simulation testing, a Position Reward Augmented Generative Adversarial Imitation Learning (PRA-GAIL) method is proposed. In urban roads, since the disturbance behavior is mainly generated by electric bicycles, electric bicycles are selected as the research object. In the constructed simulation environment, Generative Adversarial Imitation Learning (GAIL) is used to make the simulated trajectories approximate the real trajectories, while Position Reward and Lagrangian Constraint methods are added to solve the homogenization and uncontrollable behaviors of existing simulation methods. In the test set validation, the average displacement error of the GAIL and PRA-GAIL methods decreased by 61.7% and 65.8% , respectively, compared to the behavioral cloning method. In the behavioral performance validation, the KL divergence of acceleration distributions between simulation and reality was significantly reduced in PRA-GAIL compared to GAIL, and the percentage error of overtaking and illegal lane changing behaviors decreased by 7.2% and 20.2%, respectively. Using the Lagrangian method to add constraints resulted in a 75.8% reduction in the number of agents with risky behavior compared to commonly used reward augmentation methods. In trajectory validation, in the simulation environment, the average displacement error of PRAGAIL is reduced by 17.5% compared to GAIL. The resulting model realistically reproduces the overtaking maneuver space of cyclists. The results show that the method adopted in this paper is suitable for bicycle behavior simulation, the proposed modifications effectively enhance the simulation performance, and the obtained simulation model accurately reproduces the disturbance behavior of bicycles on road segments, which can be applied to automated vehicle simulation tests.

Key words: traffic engineering, bicycle behavior, reinforcement learning, generative adversarial imitation learning, automatic vehicle test, micro traffic simulation