基于博弈学习的通信抗干扰智能决策方法研究

With the rapid development of the jamming attack technologies, the style of jamming is increasing, and the level of intelligence is improving. Meanwhile, various new radio technologies and services are widely used, and the number of wireless devices has increased dramatically. Thus, the mutual interference among users is also becoming serious. Therefore, the new jamming environments will show some obvious characteristics, such as “intelligent jammer” and “complex constitution” (mutual interference among users and external malicious jamming exist at the same time) in the future. This project starts from the key problem of intelligent decision-making for anti-jamming communication, and we carry out an in-depth study of intelligent decision-making algorithms for anti-jamming communication. Some technologies are employed, such as game theory, game-theoretic learning, multi-agent reinforcement learning, and we mainly aim to solve the following three problems: Constructing game model for anti-jamming communication, performing robust decision-making under the condition of dynamic and incomplete information constraints, and achieving effective decision-making in complex jamming environments. Specifically, the main contributions are summarized as follows: (1) The conflict and competition among users and between user and jammer are analyzed in depth, and the anti-jamming game model is constructed to provide theoretical support for algorithm design; (2) To cope with the dynamic and incomplete information constraints in jamming environments, intelligent anti-jamming decision-making approaches based on dynamic and incomplete information game model are proposed; (3) Taking the coordination among users into consideration, a collaborative anti-jamming decision-making framework is constructed, and intelligent anti-jamming decision-making approaches based on stochastic game are proposed.

随着电子进攻技术的迅速发展，干扰样式不断增加、智能化水平不断提升。同时，各种无线电新技术、新业务广泛应用，无线设备数量急剧增加，相应的互扰现象变得异常严重。因此，未来的新型干扰环境将呈现出“干扰智能”、“构成复杂”（即同时存在外部恶意干扰和用户间互扰）等典型特征。本项目从通信抗干扰智能决策这一核心问题出发，综合运用博弈论、博弈学习、多智能体强化学习等多种理论和方法，主要围绕抗干扰博弈模型构建、动态不完全信息约束下的稳健决策以及复杂干扰条件下的有效决策三个关键问题展开，深入研究抗干扰智能决策方法。具体内容包括：（1）深入分析用户间以及用户和干扰间的冲突和竞争关系，构建抗干扰博弈模型，为算法设计提供理论支撑；（2）针对干扰环境中的动态不完全信息约束，提出基于动态不完全信息博弈的抗干扰智能决策方法；（3）考虑用户间协作，构建协作抗干扰决策架构，提出基于随机博弈的协作抗干扰智能决策方法。

通信抗干扰能力是军事无线通信最基本的要求，也是战场生存能力的重要特征。本项目着眼于通信抗干扰智能决策这一核心问题，研究基于博弈学习的通信抗干扰智能决策方法。通过构建抗干扰博弈模型，建模分析用户与恶意干扰之间的干扰关系，以及用户间的冲突和竞争关系，并设计智能学习算法求解最佳抗干扰策略，从而实现抗干扰策略与干扰环境的最佳匹配。.本项目主要研究内容和结果如下：（1）抗干扰博弈模型。从博弈学习抗干扰基本架构出发，通过抗干扰博弈模型构建和智能学习算法设计两个方面进行分析，并以Stackelberg博弈和随机博弈为例进行分析。针对干扰作为博弈参与者的情况，构建了Stackelberg博弈抗干扰模型，同时刻画用户和恶意干扰之间的干扰关系以及用户间的冲突和竞争关系；针对干扰作为环境的情况，构建了随机博弈抗干扰模型，同时刻画用户间的协作与竞争特性。（2）基于动态不完全信息博弈的抗干扰智能决策方法。针对干扰环境的动态不完全信息约束，基于Stackelberg博弈，将用户和恶意干扰分别建模为博弈的领导者和跟随者，并设计了智能学习算法求解博弈均衡解。（3）基于随机博弈的协作抗干扰智能决策方法。考虑信息交互层面的协作，将用户建模为随机博弈的参与者，干扰作为环境，并设计多用户Q学习协作抗干扰决策算法，实现干扰模式未知条件下干扰环境与抗干扰策略的匹配。.受本项目支持，在IEEE Communications Magazine 、IEEE Transactions on Vehicular Technology等期刊上发表SCI论文12篇（其中IEEE期刊论文6篇），中文期刊论文1篇，会议论文1篇，出版专著1部；授权国家发明专利2项，受理3项。培养硕士生3名，培养博士生1名。