Deep Reinforcement Learning for Mobile Robot Path Planning
DOI:
https://doi.org/10.53469/jtpes.2024.04(04).07Keywords:
Mobile robot, Deep reinforcement learning, Path planning, NavigationAbstract
Path planning is an important problem with the the applications in many aspects, such as video games, robotics etc. This paper proposes a novel method to address the problem of Deep Reinforcement Learning (DRL) based path planning for a mobile robot. We design DRL-based algorithms, including reward functions, and parameter optimization, to avoid time-consuming work in a 2D environment. We also designed an Two-way search hybrid A* algorithm to improve the quality of local path planning. We transferred the designed algorithm to a simple embedded environment to test the computational load of the algorithm when running on a mobile robot. Experiments show that when deployed on a robot platform, the DRL-based algorithm in this article can achieve better planning results and consume less computing resources.
References
Liu, T., Xu, C., Qiao, Y., Jiang, C., & Chen, W. (2024). News Recommendation with Attention Mechanism. ArXiv. https://doi.org/10.5281/zenodo.10635481
Ma D, Li S, Dang B, et al. Fostc3net: A Lightweight YOLOv5 Based On the Network Structure Optimization[J]. arXiv preprint arXiv:2403.13703, 2024.
Liu T, Cai Q, Xu C, et al. Image Captioning in news report scenario[J]. arXiv preprint arXiv:2403.16209, 2024.
Bidmeshki M M, Zhang Y, Zaman M, et al. Hunting security bugs in SoC designs: Lessons learned[J]. IEEE Design & Test, 2020, 38(1): 22-29.
Liu T, Xu C, Qiao Y, et al. Particle Filter SLAM for Vehicle Localization[J]. arXiv preprint arXiv:2402.07429, 2024.
Tong W, Guan X, Kang J, et al. Normal assisted pixel-visibility learning with cost aggregation for multiview stereo[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(12): 24686-24697.
Zhang X, Liu H, Xue L, et al. Multi-objective Collaborative Optimization Algorithm for Heterogeneous Cooperative Tasks Based on Conflict Resolution[C]//International Conference on Autonomous Unmanned Systems. Singapore: Springer Singapore, 2021: 2548-2557.
Wang X, Hu B, Zhou M. OGBPS: orientation and gradient based path smoothing algorithm for various robot path planners[C]//2019 IEEE international conference on robotics and biomimetics (ROBIO). IEEE, 2019: 1453-1458.
Zhang Y, Makris Y. Hardware-based detection of spectre attacks: a machine learning approach[C]//2020 Asian hardware oriented security and trust symposium (AsianHOST). IEEE, 2020: 1-6.
He W, Jiang Z. A survey on uncertainty quantification methods for deep neural networks: An uncertainty source perspective[J]. arXiv preprint arXiv:2302.13425, 2023.
Zhang Y, Zhu M, Gong Y, et al. Optimizing science question ranking through model and retrieval-augmented generation[J]. International Journal of Computer Science and Information Technology, 2023, 1(1): 124-130.
Wang X, Qiao Y, Xiong J, et al. Advanced Network Intrusion Detection with TabTransformer[J]. Journal of Theory and Practice of Engineering Science, 2024, 4(03): 191-198.
Arief H A, Arief M, Zhang G, et al. Sane: smart annotation and evaluation tools for point cloud data[J]. IEEE Access, 2020, 8: 131848-131858.
Zhou L, Zhang Y, Makris Y. TPE: A hardware-based TLB profiling expert for workload reconstruction[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2021, 11(2): 292-305.
Li S, Dong X, Ma D, et al. Utilizing the LightGBM Algorithm for Operator User Credit Assessment Research[J]. arXiv preprint arXiv:2403.14483, 2024.
Zhang Y, Zhou L, Makris Y. Hardware-based real-time workload forensics via frame-level TLB profiling[C]. IEEE 37th VLSI Test Symposium, 2019.
Liu T, Cai Q, Xu C, et al. Rumor Detection with a novel graph neural network approach[J]. arXiv preprint arXiv:2403.16206, 2024.
Tong K W, Sun P Z H, Wu E Q, et al. Adaptive cost volume representation for unsupervised high-resolution stereo matching[J]. IEEE Transactions on Intelligent Vehicles, 2022.
Su J, Jiang C, Jin X, et al. Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review[J]. arXiv preprint arXiv:2402.10350, 2024.
Zhang Y, Zhu M, Gui K, et al. Development and Application of a Monte Carlo Tree Search Algorithm for Simulating Da Vinci Code Game Strategies[J]. arXiv preprint arXiv:2403.10720, 2024.
Huang W, Wan G, Ye M, et al. Federated graph semantic and structural learning[C]. Int. Joint Conf. Artif. Intell., 2023.
Wan G, Huang W, Ye M. Federated Graph Learning under Domain Shift with Generalizable Prototypes[C]. AAAI Conference on Artificial Intelligence, 2024.
Ru J, Yu H, Liu H, et al. A Bounded Near-Bottom Cruise Trajectory Planning Algorithm for Underwater Vehicles[J]. Journal of Marine Science and Engineering, 2022.
Li S, Zhang G, Lei X, et al. Trajectory tracking control of a unicycle-type mobile robot with a new planning algorithm[C]. IEEE International Conference on Robotics and Biomimetics, 2017.
Huang Y, Yang J J. Symmetric contrastive learning for robust fault detection in time-series traffic sensor data[J]. International Journal of Data Science and Analytics, 2024.
Chen Z, Jing L, Li Y, et al. Bridging the domain gap: Self-supervised 3d scene understanding with foundation models[J]. Advances in Neural Information Processing Systems, 2024, 36.
Xie H, Deng T, Wang J, et al. Robust incremental long-term visual topological localization in changing environments[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 72: 1-14.
Jaladi S R, Chen Z, Malayanur N R, et al. End-To-End Training and Testing Gamification Framework to Learn Human Highway Driving[C]//2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2022: 4296-4301.
Deng T, Xie H, Wang J, et al. Long-Term Visual Simultaneous Localization and Mapping: Using a Bayesian Persistence Filter-Based Global Map Prediction[J]. IEEE Robotics & Automation Magazine, 2023, 30(1): 36-49.
Tai, L.; Liu, M. Towards Cognitive Exploration through Deep Reinforcement Learning for Mobile Robots. arXiv 2016, arXiv:1610.01733.
Pfeiffer, M.; Schaeuble, M.; Nieto, J.; Siegwart, R.; Cadena, C. From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 29 May–3 June 2017; pp. 1527–1533.
Tai, L.; Liu, M. A robot exploration strategy based on Q-learning network. In Proceedings of the 2016 IEEE International Conference on Real-time Computing and Robotics (RCAR), Angkor Wat, Cambodia, 6–10 June 2016; pp. 57–62.
Tai, L.; Paolo, G.; Liu, M. Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017; pp. 31–36.
Faust, A.; Ramírez, O.; Fiser, M.; Oslund, K.; Francis, A.; Davidson, J.O.; Tapia, L. PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-Based Planning. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 5113–5120.
Francis, A.; Faust, A.; Chiang, H.L.; Hsu, J.; Kew, J.C.; Fiser, M.; Lee, T.E. Long-Range Indoor Navigation With PRM-RL. IEEE Trans. Robot. 2020, 36, 1115–1134.
Zeng, J.; Qin, L.; Hu, Y.; Yin, Q.; Hu, C. Integrating a Path Planner and an Adaptive Motion Controller for Navigation in Dynamic Environments. Appl. Sci. 2019, 9, 1384.
Iyer, A.; Mahadevan, A. Collision Avoidance Robotics Via Meta-Learning (CARML). arXiv 2020, arXiv:2007.08616.
Sánchez-López, J.L.; Wang, M.; Olivares-Méndez, M.A.; Molina, M.; Voos, H. A Real-Time 3D Path Planning Solution for Collision-Free Navigation of Multirotor Aerial Robots in Dynamic Environments. J. Intell. Robot. Syst. 2019, 93, 33–53.
Kim, M.; Han, D.K.; Park, J.; Kim, J.S. Motion Planning of Robot Manipulators for a Smoother Path Using a Twin Delayed Deep Deterministic Policy Gradient with Hindsight Experience Replay. Appl. Sci. 2020, 10, 575.
Yao J, Li C, Sun K, et al. Ndc-scene: Boost monocular 3d semantic scene completion in normalized device coordinates space[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, 2023: 9421-9431.
Yao J, Pan X, Wu T, et al. Building lane-level maps from aerial images[C]//ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024: 3890-3894.
Yan C. Predict Lightning Location and Movement with Atmospherical Electrical Field Instrument[C]//2019 IEEE 10th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON). IEEE, 2019: 0535-0537.
Yan C, Qiu Y, Zhu Y. Predict Oil Production with LSTM Neural Network[C]//Proceedings of the 9th International Conference on Computer Engineering and Networks. Springer Singapore, 2021: 357-364.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Hao Liu, Yi Shen, Shuangjiang Yu, Zijun Gao, Tong Wu
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
