Deep Reinforcement Learning (DRL) is widely used in path planning with its powerful neural network fitting ability and learning ability. However, existing DRL-based methods use discrete action space and do not consider the impact of historical state information, resulting in the algorithm not being able to learn the optimal strategy to plan the path, and the planned path has arcs or too many corners, which does not meet the actual sailing requirements of the ship. In this paper, an optimized path planning method for coastal ships based on improved Deep Deterministic Policy Gradient (DDPG) and Douglas–Peucker (DP) algorithm is proposed. Firstly, Long Short-Term Memory (LSTM) is used to improve the network structure of DDPG, which uses the historical state information to approximate the current environmental state information, so that the predicted action is more accurate. On the other hand, the traditional reward function of DDPG may lead to low learning efficiency and convergence speed of the model. Hence, this paper improves the reward principle of traditional DDPG through the mainline reward function and auxiliary reward function, which not only helps to plan a better path for ship but also improves the convergence speed of the model. Secondly, aiming at the problem that too many turning points exist in the above-planned path which may increase the navigation risk, an improved DP algorithm is proposed to further optimize the planned path to make the final path more safe and economical. Finally, simulation experiments are carried out to verify the proposed method from the aspects of plan planning effect and convergence trend. Results show that the proposed method can plan safe and economic navigation paths and has good stability and convergence.


    Access

    Download


    Export, share and cite



    Title :

    An Optimized Path Planning Method for Coastal Ships Based on Improved DDPG and DP


    Contributors:
    Yiquan Du (author) / Xiuguo Zhang (author) / Zhiying Cao (author) / Shaobo Wang (author) / Jiacheng Liang (author) / Fengge Zhang (author) / Jiawei Tang (author)


    Publication date :

    2021




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown





    Collision avoidance path planning of tourist ship based on DDPG algorithm

    Yi ZHOU / Chuanping YUAN / Haicheng XIE et al. | DOAJ | 2021

    Free access

    Towards Safe and Efficient Modular Path Planning using Twin Delayed DDPG

    Hebaish, Marawan Azmy / Hussein, Ahmed / El-Mougy, Amr | IEEE | 2022


    Autopilot Strategy Based on Improved DDPG Algorithm

    Zuo, Xiaochao / Li, Xiaoning / Tian, Zhewen | SAE Technical Papers | 2019


    Unmanned Vehicle Path Planning Based on DDPG Reinforcement Learning Algorithm in Forest Land Environment

    Li, Luhao / Li, Yongliang / Du, Miaomiao et al. | Springer Verlag | 2025