鲁棒与风险敏感的高维序列决策过程理论与算法研究

With the tremendous development of Artificial Intelligence (AI), the demand for high-stakes applications has become increasing prominent. The research difficulties and hotpots are these high-stakes AI systems how to make robust and safe decisions in order to avoid the extreme policies and prevent the catastrophic consequences. Most real-world high-stakes AI systems can be modeled by high-dimensional sequential decision processes. Therefore, this project takes the high-dimensional sequential decision processes as the research objects, and aims to propose robust and risk-sensitive policy learning methods. Then, it plans to carry out research from three aspects: basic theory, key technologies and practical examples. Firstly, build a novel theory, which is to explore the equivalent relationship between the robust high-dimensional sequential decision processes and the risk-sensitive high-dimensional sequential decision processes, and investigate their unifying representation framework. Secondly, based on deep reinforcement learning, study the following key technologies to realize the policy robustness and avoid risks: robust policy learning algorithms, dynamic risk-sensitive policy learning algorithms, and robust and risk-sensitive policy learning algorithms of integrating the distributional reinforcement learning advantages for high-dimensional sequential decision processes. Finally, develop a dual-arm electric power robot of splicing and building wires prototype system to verify the proposed theory and key technologies. In conclusion, it is of great significance to explore the robust and risk-sensitive high-dimensional sequential decision processes theory and key technologies for sequential decision processes and deep reinforcement learning theoretical researches and engineering applications.

随着人工智能技术的巨大发展，高风险应用的需求日益明显。如何让这些高风险智能系统做出安全鲁棒的决策，规避极端策略，防止灾难性后果发生，是研究的难点和热点。很多高风险智能系统都可以用高维序列决策过程来建模表示。故本项目以高维序列决策过程为研究对象，旨在提出鲁棒与风险敏感的策略学习方法，从基础理论、关键技术和实例例证三方面展开研究。首先，挖掘鲁棒的高维序列决策过程与风险敏感的高维序列决策过程之间的等价关系，研究二者统一表示的核心理论框架；其次，基于深度强化学习，研究高维序列决策过程的鲁棒、动态风险敏感决策算法，以及融合分布式强化学习优势的鲁棒与风险敏感决策算法等关键技术，实现鲁棒决策，规避风险；最后，开发接搭线双臂电力机器人原型系统，验证提出的理论和关键技术。通过上述研究，探索鲁棒与风险敏感的高维序列决策过程的理论和关键技术，对高维序列决策过程以及深度强化学习的理论研究和工程应用具有重要意义。

随着人工智能技术的巨大发展，高风险的需求日益明显，如何让这些高风险的智能系统做出安全鲁棒的决策，规避极端策略，防止灾难性后果发生，是研究的难点和热点。.本项目针对这些问题，以高维序列决策过程为研究对象，主要从以下几个方面开展研究：1）高维鲁棒判别学习研究 2）高维分位数回归的分布式鲁棒优化 3）鲁棒聚类研究 4）利用雷达反射率图像序列过程，空气回声分类问题研究 5）高速铁路极点数识别研究 6）高维图像特征鲁棒提取研究。本项目将部分研究成果应用在图像以及检测任务中，获得较好的性能。.三年来，课题总体进展顺利，发表国际会议、期刊7篇，正在投稿论文4篇。