To reduce the fuel consumption (FC) of heavy duty logistic vehicles (HDLVs), $P2$ parallel hybridization is a promising solution, and deep reinforcement learning (DRL) is a promising method to optimize energy management strategies (EMSs). However, the complicated discrete-continuous hybrid action space lying in the $P2$ system presents a challenge to achieve real-time optimal control. Thus, this article proposes a novel DRL algorithm combining auto-tune soft actor–critic (ATSAC) with ordinal regression to optimize the engine torque output and gear shifting simultaneously. ATSAC can adjust the update frequency and learning rate of SAC automatically to improve the generalization, and ordinal regression can convert discrete variables into samplings in continuous space to handle the hybrid action. Moreover, a multidimensional scenario-oriented driving cycle (SODC) is established through naturalistic driving big data (NDBD) as the training cycle to further improve the EMS generalization. By comprehensive comparison with the widely used twin-delayed deep deterministic policy gradient (TD3)-based EMSs, ATSAC achieves significant improvement with 53.70% higher computational efficiency and 12.31% lower negative total reward (NTR) in the training process. Application analysis in unseen real-world driving scenarios shows that only ATSAC-based EMS can obtain real-time optimal control in the testing process. Furthermore, the EMS trained through SODC obtains 81.73% lower NTR than the standard China world transient vehicle cycle (CWTVC), which demonstrates that SODC can represent the real-world driving scenarios much more accurately than CWTVC, especially in low-speed high-load conditions, which are crucial for HDLVs.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Deep Reinforcement Learning-Based Energy Management for Heavy Duty HEV Considering Discrete-Continuous Hybrid Action Space


    Beteiligte:
    Liu, Zemin Eitan (Autor:in) / Li, Yanfei (Autor:in) / Zhou, Quan (Autor:in) / Li, Yong (Autor:in) / Shuai, Bin (Autor:in) / Xu, Hongming (Autor:in) / Hua, Min (Autor:in) / Tan, Guikun (Autor:in) / Xu, Lubing (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    01.12.2024


    Format / Umfang :

    3997340 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Learning of EMSs in Discrete-Continuous Hybrid Action Space

    Li, Yuecheng / He, Hongwen | Springer Verlag | 2022


    UAS CONFLICT RESOLUTION IN CONTINUOUS ACTION SPACE USING DEEP REINFORCEMENT LEARNING

    Hu, Jueming / Yang, Xuxi / Wang, Weichang et al. | TIBKAT | 2020


    UAS Conflict Resolution in Continuous Action Space Using Deep Reinforcement Learning

    Hu, Jueming / Yang, Xuxi / Wang, Weichang et al. | AIAA | 2020


    Obstacle Avoidance for UAS in Continuous Action Space Using Deep Reinforcement Learning

    Hu, Jueming / Yang, Xuxi / Wang, Weichang et al. | ArXiv | 2021

    Freier Zugriff