复杂环境中基于强化学习的无人机路径规划研究

Reinforcement learning (RL) is one class of important methods for solving complex sequential decision making problems. RL has becoming one of the research hotspots in machine learning and widely studied in a variety of areas, including robotics, control engineering, games and so on. However, it is still difficult for RL to solve problems with large or continuous state and action spaces, which limits the applications of RL in real engineering problems. This project intends to focus on the path planning of unmanned aerial vehicles (UAVs) using reinforcement learning in complex environments and aims to make innovation and breakthrough on state feature representation, online RL algorithm and generalization ability of RL through: 1) proposing a state feature representation method using neural network for reinforcement learning with continuous spaces; 2) proposing an online reinforcement learning algorithm with approximate policy iteration to improve the efficiency of reinforcement learning; 3)proposing a hierarchical path planning method using reinforcement learning for UAVs in complex environments; 4) designing and achieving a UAV system to evaluate the performance of the proposed hierarchical path planning method using reinforcement learning. The research of this project will play a significant role in improving the efficiency of reinforcement learning in continuous spaces and expanding the applications of reinforcement learning in real problems.

强化学习是求解复杂序贯决策优化问题的一类重要方法，具有广泛的应用前景，近年来成为机器学习的研究热点，然而，大规模连续空间中的状态特征表示与算法效率问题仍制约着强化学习在实际应用中的推广。本项目以复杂环境中基于强化学习的无人机路径规划问题为背景，研究强化学习在连续空间中的状态特征表示方法和快速在线学习算法，克服现有方法面临的连续空间逼近与泛化能力问题；在无人机路径规划问题中引入分层规划机制，应用强化学习方法提高规划效率和规划结果的合理性，并对所提方法进行仿真实验验证。本项目的成果将有利于解决强化学习现有方法面临的连续空间中算法计算效率和泛化性能受限等问题，促进强化学习在实际问题中的应用推广，同时为无人机路径规划提供高效的学习优化方法，对于提高无人机自主学习能力具有十分重要的理论意义和实用价值。

强化学习可通过与环境交互求解最优控制问题，是机器学习的研究热点之一，近年来得到广泛关注。目前强化学习面临的一大挑战是大规模或连续状态空间中的快速求解问题。本项目以复杂环境中基于强化学习的无人机路径规划为背景，研究大规模或连续状态空间中强化学习的状态特征表示方法和快速学习算法，克服现有方法面临的连续空间逼近与泛化能力问题，并将强化学习方法应用于无人机路径规划，提升无人机的自主学习能力。项目提出了一种基于随机神经元的状态特征表示方法，减少了需要人为调整优化的参数，提高了状态特征表示效率；提出了一种基于超限学习机的近似策略迭代（ELM-API）算法，提高了算法学习效率，理论分析与仿真实验结果表明该算法具有良好的特征表示能力和泛化能力；提出了一种基于策略迭代强化学习的无人机分层路径规划方法，使得路径规划模块能够处理一定的环境不确定性，提高了无人机对环境的自适应能力；构建了一个用于路径规划算法实验验证的无人机测试系统，很好支撑了强化学习的应用研究。本项目成果提升了强化学习在解决大规模或连续空间问题时的算法效率，对于提高无人机的自主性具有重要学术意义和应用价值。