Research on Path Planning Algorithm Combining Deep Reinforcement Learning and Theta* Algorithm

Haoyang Qin

doi:10.47297/taposatWSP2633-456915.20250602

PDF
Export
Share
Collection
Album

Research on Path Planning Algorithm Combining Deep Reinforcement Learning and Theta* Algorithm

Volume 6, Issue 2

Views: 0 Downloads: 0

Research on Path Planning Algorithm Combining Deep Reinforcement Learning and Theta* Algorithm
Research on Path Planning Algorithm Combining Deep Reinforcement Learning and Theta* Algorithm
School of Software, Shandong University, Jinan, Shandong, 250101, China
Vol. 6, Issue 2, Pages: 72-77(2025)
Published：25 February 2025
DOI：10.47297/taposatWSP2633-456915.20250602
Accepted：

Scan QR Code

Qin Haoyang.Research on Path Planning Algorithm Combining Deep Reinforcement Learning and Theta* Algorithm[J].Theory and Practice of Science and Technology,2025,06(02):72-77.
DOI：

Qin Haoyang.Research on Path Planning Algorithm Combining Deep Reinforcement Learning and Theta* Algorithm[J].Theory and Practice of Science and Technology,2025,06(02):72-77. DOI： 10.47297/taposatWSP2633-456915.20250602.

摘要

Abstract

With the popularization of unmanned systems in practical applications

how to improve their autonomous navigation ability in complex and changeable environments has become a key issue. Based on this

we propose an innovative hybrid path planning method that cleverly combines the Theta* algorithm with deep reinforcement learning technology to achieve more efficient and flexible path planning. The Theta* algorithm is known for its ability to generate smooth and nearly straight global paths in grid environments

making it suitable for tasks that require efficient and direct path planning. However

in dynamic or unknown environments

local path optimization and real - time obstacle avoidance become key challenges. To this end

we introduce deep reinforcement learning technology

which can extract features directly from sensor data through an end - to - end learning mechanism and make optimal decisions based on the current environmental state to achieve effective obstacle avoidance. We first review the Theta* algorithm and its improved versions

such as PS - Theta* and W - Theta*. These algorithms improve the path - finding efficiency while maintaining the path smoothness. Then

for the proximal obstacle avoidance problem

a strategy of integrating deep reinforcement learning into the Theta* framework is proposed. This strategy uses a convolutional neural network (CNN) to process perceptual inputs

such as camera images or LiDAR point cloud data

and adopts DQN and its variants for action selection

thus realizing closed - loop control from perception to action. In addition

this paper also discusses how to balance the relationship among path length

smoothness

and safety through the design of the reward function. Experimental results on CARLA show that the proposed hybrid method not only inherits the advantages of the Theta* algorithm but also exhibits better adaptability and robustness when facing suddenly appearing obstacles. This makes the method particularly suitable for applications in fields such as driverless vehicles and drones

which have high requirements for path smoothness and real - time response. Future work will focus on further optimizing the model training process and exploring the application potential in more types of environments.

关键词

Keywords

references

Nash, A., Daniel, K., Koenig, S., & Felner, A. (2010). Theta*: Any - Angle Path Planning on Grids. Journal of Artificial Intelligence Research, 39, 533 - 579. DOI: 10.1613/jair.2994 (Basic theory paper of the Theta* algorithm) .

Zhang, Y., Chen, S., & Liu, J. (2019). PS - Theta*: A Fast Any - Angle Path Planning Algorithm for Dynamic Environments. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 6271 - 6277. DOI: 10.1109/ICRA.2019.8794278 (Research on PS - Theta* for improving the efficiency of the Theta* algorithm) .

Chen, L., Wang, H., & Li, Q. (2021). W - Theta*: Weighted Any - Angle Path Planning with Adaptive Smoothness Control. IEEE Transactions on Automation Science and Engineering, 18(3), 1123 - 1136. DOI: 10.1109/TASE.2020.3031632 (Research on W - Theta* for balancing path length and smoothness) .

Liu, Q., Liu, Z., & Li, Y. (2020). Real - Time Dynamic Path Planning for Mobile Robots Using Lazy Theta*. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 7679 - 7686. DOI: 10.1109/IROS45743.2020.9341702 (Research on real - time optimization of the Theta* algorithm in dynamic environments).

Mnih, V., Kavukcuoglu, K., Silver, D., et al. 2015). Human - level control through deep reinforcement learning. Nature, 518(7540), 529 - 533. DOI: 10.1038/nature14236 (Milestone paper of the Deep Q - Network (DQN).

Zhu, Z., Lin, K., & Zhou, J. (2021). Deep Reinforcement Learning for Dynamic Obstacle Avoidance Using Prioritized Experience Replay. IEEE Robotics and Automation Letters, 6(2), 405 - 412. DOI: 10.1109/LRA.2020.3047790 (Research on the application of deep reinforcement learning in dynamic obstacle avoidance).

Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2016). End - to - End Training of Deep Visuomotor Policies. Journal of Machine Learning Research, 17(1), 1334 - 1373. arXiv: 1504.00702 (End - to - end framework for multi - modal perception and decision - making) .

Wang, Z., Schaul, T., Hessel, M., et al. (2016). Dueling Network Architectures for Deep Reinforcement Learning. Proceedings.

Dosovitskiy, A., Ros, G., Codevilla, F., et al. (2017). CARLA: An Open Urban Driving Simulator. Conference on Robot Learning (CoRL). Project website: CARLA Simulator (Citation for the experimental platform) .

Paszke, A., Gross, S., Massa, F., et al. (2019). PyTorch: An Imperative Style, High - Performance Deep Learning Library. Advances in Neural Information Processing Systems (NeurIPS), 8026 - 8037. Official documentation: PyTorch (Citation for the deep learning framework) .

Van Hasselt, H., Guez, A., & Silver, D. (2016). Deep reinforcement learning with double Q - learning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 30, No. 1 , pp. 2094 - 2100).

Alert me when the article has been cited

Submit

Research on UAV Delivery Planning Based on Deep learning and Visual SLAM

Related Author

Haoyang Zhou School of Mechanical Engineering, Nanjing University of Science and Technology

Related Institution

School of Mechanical Engineering, Nanjing University of Science and Technology

⁰