HCPP:一种数据高效的分层车辆跟随方法

王旭; 商尔科; 苗启广; 戴斌; 刘泱

doi:10.19665/j.issn1001-2400.20230309

您当前的位置：

首页 >

文章列表页 >

HCPP:一种数据高效的分层车辆跟随方法

信息与通信工程&计算机科学与技术 | 更新时间：2024-01-22

- HCPP:一种数据高效的分层车辆跟随方法
- 暂无标题
- 西安电子科技大学学报 2023年50卷第6期页码：161-171
- 作者机构：
  
  1. 西安电子科技大学计算机科学与技术学院,陕西西安 710071
  2. 军事科学院国防科技创新研究院,北京 100071
  3. 国防科技大学智能科学学院,湖南长沙 410073
- 作者简介：
  
  [ "王旭(1993—),男,西安电子科技大学博士研究生,E-mail:[email protected];" ]
  [ "商尔科(1984—),男,副研究员,E-mail:[email protected];" ]
  [ "戴斌(1975—),男,研究员,E-mail: [email protected]。" ]
  刘泱(1994—),男,国防科技大学博士研究生,E-mail:[email protected]
- 基金信息：
  
  国家自然科学基金重大项目重点报告(61790565);国家自然科学基金(62272364);国家自然科学基金(61902296);陕西省重点研发计划(2020LSFP3-15);广西可信软件重点实验室研究课题(KX202061);青岛市科技计划重点研发专项(21-1-2-18-xx)
- DOI：10.19665/j.issn1001-2400.20230309
  中图分类号：
扫描看全文
王旭, 商尔科, 苗启广, 等. HCPP:一种数据高效的分层车辆跟随方法[J]. 西安电子科技大学学报, 2023,50(6):161-171.
王旭, 商尔科, 苗启广, 等. HCPP:一种数据高效的分层车辆跟随方法[J]. 西安电子科技大学学报, 2023,50(6):161-171. DOI： 10.19665/j.issn1001-2400.20230309.

DOI：

摘要

现有的基于控制理论的车辆跟随控制方法需要对两辆车的速度和距离等建立模型,缺乏泛化性并难以实现稳定平滑的控制效果。为了解决这个问题,提出了一种不依赖车辆运动学模型的数据高效的分层车辆跟随控制方法。该方法的上层利用车辆坐标、速度和其他车载传感器的感知结果构建数据集,并训练深度强化学习模型,避免了依赖先验知识的问题,并不需要在真实世界中进行训练。训练过程中从数据集中随机采样样本,提高了数据利用率。该方法的下层通过比例积分微分控制算法实时控制车辆的加速度和角速度,避免了深度强化学习策略不稳定导致的控制抖动,从而使车辆的控制更加平滑。为了验证该算法的性能,进行了仿真实验和实车实验。实验结果表明,该算法可以使跟随车与目标车之间的距离始终保持在安全合理的范围内。通过对比实验,证明了该算法在横向和纵向上都实现了更稳定、平滑和安全的车辆跟随控制。

Abstract

The current car following control methods based on the control theory rely on models for both car speed and distance,which suffer from a lack of generalization and achieve stable and smooth control results with difficulty.To address this problem,we propose a data-efficient hierarchical car following control method that does not depend on car kinematic models.The upper layer of the proposed method constructs a dataset based on the perception results of car coordinates,speed,and other onboard sensors.A deep reinforcement learning model is trained to perform car following,avoiding the reliance on prior knowledge and eliminating the need for real-world training.Training samples are randomly selected from the dataset,which improves data utilization.The lower layer of the method implements real-time control of the car acceleration and angular velocity using a PID controller,which avoids the control jitter caused by the instability of deep reinforcement learning policies,resulting in smoother control.To verify the performance of the algorithm,both simulation and real-world experiments are conducted.Experimental results show that the proposed algorithm can keep the distance between the following car and the target car within a safe and reasonable range.The comparative experiments demonstrate that the proposed algorithm achieves more stable,smoother,and safer car following control in both lateral and longitudinal directions.

关键词

车辆跟随控制深度强化学习离线训练分层控制比例积分微分控制

Keywords

car-following controldeep reinforcement learningoffline traininghierarchical controlproportion integral differential(PID) control

references

PIPES L A. An Operational Analysis of Traffic Dynamics[J]. Journal of Applied Physics, 1953, 24(3):274-281. DOI:10.1063/1.1721265http://doi.org/10.1063/1.1721265https://pubs.aip.org/jap/article/24/3/274/160330/An-Operational-Analysis-of-Traffic-Dynamicshttps://pubs.aip.org/jap/article/24/3/274/160330/An-Operational-Analysis-of-Traffic-Dynamics

FORBES T W. Human Factor Considerations in Traffic Flow Theory[J]. Highway Research Record, 1963(15):60-66.

任殿波, 张继业. 基于 Lyapunov 函数方法的时滞车辆纵向跟随控制[J]. 控制与决策, 2007, 22(8):918-921.

REN Dianbo, ZHANG Jiye. Lyapunov Function Approach to Longitudinal Following Control of Vehicles in Platoon with Delays[J]. Control and Decision, 2007, 22(8):918-921.

LIN Y, MCPHEE J, AZAD N L. Comparison of Deep Reinforcement Learningand Model Predictive Control for Adaptive Cruise Control[J]. IEEE Transactions on Intelligent Vehicles, 2021, 6(2):221-231. DOI:10.1109/TIV.2020.3012947http://doi.org/10.1109/TIV.2020.3012947https://ieeexplore.ieee.org/document/9152161/https://ieeexplore.ieee.org/document/9152161/

李腾, 曹世杰, 尹思薇, 等. 应用Q学习决策的最优攻击路径生成方法[J]. 西安电子科技大学学报, 2021, 48(1):160-167.

LI Teng, CAO Shijie, YIN Siwei, et al. Optimal Method for the Generation of the Attack Path Based on the Q-Learning Decision[J]. Journal of Xidian University, 2021, 48(1):160-167.

张英, 韦闽峰, 王世会, 等. 飞行器强化学习多模在轨控制[J]. 西安电子科技大学学报, 2020, 47(2):75-82.

ZHANG Ying, WEI Minfeng, WANG Shihui, et al. Aircraft Reinforcement Learning Multi-Mode Control in Orbit[J]. Journal of Xidian University, 2020, 47(2):75-82.

朱冰, 蒋渊德, 赵健, 等. 基于深度强化学习的车辆跟驰控制[J]. 中国公路学报, 2019, 32(6):53-60. DOI:10.19721/j.cnki.1001-7372.2019.06.005http://doi.org/10.19721/j.cnki.1001-7372.2019.06.005

ZHU Bing, JIANG Yuande, ZHAO Jian, et al. A Car-Following Control Algorithm Based on Deep Reinforcement Learning[J]. 2019, 32(6):53-60.

LIAO Y, YU G, CHEN P, et al. Modelling Personalised Car-Following Behaviour:A Memory-Based Deep Reinforcement Learning Approach(2022)[J/OL].[2022-12-31]. https://www.tandfonline.com/doi/full/10.1080/23249935.2022.2035846https://www.tandfonline.com/doi/full/10.1080/23249935.2022.2035846https://www.tandfonline.com/doi/full/10.1080/23249935.2022.2035846.

COLOMBARONI C, FUSCO G, ISAENKO N. Modeling Car Following with Feed-Forward and Long-Short Term Memory Neural Networks[J]. Transportation Research Procedia, 2021, 52:195-202. DOI:10.1016/j.trpro.2021.01.022http://doi.org/10.1016/j.trpro.2021.01.022https://linkinghub.elsevier.com/retrieve/pii/S235214652100051Xhttps://linkinghub.elsevier.com/retrieve/pii/S235214652100051X

SHI K, WU Y, SHI H, et al. An Integrated Car-Following and Lane Changing Vehicle Trajectory Prediction Algorithm Based on a Deep Neural Network[J]. Physica A:Statistical Mechanics and Its Applications, 2022, 599:127303. DOI:10.1016/j.physa.2022.127303http://doi.org/10.1016/j.physa.2022.127303https://linkinghub.elsevier.com/retrieve/pii/S0378437122002503https://linkinghub.elsevier.com/retrieve/pii/S0378437122002503

ZHOU Y, FU R, WANG C. Learningthe Car-Following Behavior of Drivers Using Maximum Entropy Deep Inverse Reinforcement Learning[J]. Journal of Advanced Transportation, 2020, 4752651:1-13.

ZHU M, WANG X, WANG Y. Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning[J]. Transportation Research Part C:Emerging Technologies, 2018, 97:348-368. DOI:10.1016/j.trc.2018.10.024http://doi.org/10.1016/j.trc.2018.10.024https://linkinghub.elsevier.com/retrieve/pii/S0968090X1830055Xhttps://linkinghub.elsevier.com/retrieve/pii/S0968090X1830055X

JOHNSONM A, MORADI M H. PID Control[M]. London: Springer-Verlag London Limited, 2005:1-559.

SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal Policy Optimization Algorithms(2017)[J/OL].[2022-01-01]. https://arxiv.org/abs/1707.06347https://arxiv.org/abs/1707.06347https://arxiv.org/abs/1707.06347.

SHAH S, DEY D, LOVETT C, et al. Airsim:High-Fidelity Visual and Physical Simulation for Autonomous Vehicles[C]// Field and Service Robotics.Berlin:Springer, 2018:621-635.

QUIGLEY M, GERKEY B, CONLEY K, et al. ROS:An Open-Source Robot Operating System[C]// ICRA Workshop on Open Source Software.Piscataway:IEEE, 2009:1-6.

PASZKE A, GROSS S, MASSA F, et al. Pytorch:An Imperative Style,High-Performance Deep Learning Library[C]// Advances in Neural Information Processing Systems. San Diego: NIPS, 2019:1-12.

KESTING A, TREIBER M, HELBING D. Enhanced Intelligent Driver Model to Access the Impact of Driving Strategies on Traffic Capacity[J]. Philosophical Transactions of the Royal Society A:Mathematical,Physical and Engineering Sciences, 2010, 368(1928):4585-4605. DOI:10.1098/rsta.2010.0084http://doi.org/10.1098/rsta.2010.0084https://royalsocietypublishing.org/doi/10.1098/rsta.2010.0084https://royalsocietypublishing.org/doi/10.1098/rsta.2010.0084

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

采用DDPG的联合波束成形和功率控制算法