面向多方协同深度学习系统的脆弱性检测与防护

Multi-party collaborative deep learning system connects cloud server and end users to train a deep model collaboratively via protocol. It has become a new intelligent system in the Internet-of-Things. However, in multi-party collaborative context, deep learning system has multiple vulnerabilities such as training data poisoning, information leakage and adversarial example attack. Now researches in security community often use homomorphic encryption technique to prevent information leakage, which lack of detection and defense on poisoning and adversarial examples. Researches in AI community focused on security problems of centralized deep learning system, and lack of security defense study for multi-party collaborative deep learning system. This project aims to address the security threats in the training and execution process of multi-party collaborative deep learning system. The main contents are as follows: (1) the vulnerability detection on training example poisoning by examples generation and gradient perturbation; (2) the vulnerability detection on information leakage by generative adversarial network; (3) auto adversarial examples generation and attacking detection technique via feature space transformation; (4) the comprehensive system defense and reinforcement to multiple vulnerabilities. This project discovers new methods to the vulnerability detection and defense of multi-party collaborative deep learning system comprehensively and effectively, and provides supports on the information assurance of the Internet-of-Things and intelligent systems.

多方协同深度学习系统通过协议连接云服务器和终端进行协同训练，成为物联网中新的人工智能应用系统。然而深度学习系统在多方协同环境下，具有训练数据污染、信息泄漏以及对抗样本攻击等多重脆弱性。目前网络和安全领域通常采用同态加密方法防止信息泄漏，缺少对训练数据污染和对抗样本攻击的检测与防护；而人工智能领域重在关注集中式深度学习系统的安全性问题，缺少对多方协同深度学习系统的安全防护研究。本项目针对多方协同深度学习系统面临的安全威胁，围绕模型训练和系统运行过程，重点研究：（1）基于样本生成和梯度扰动的训练数据污染脆弱性检测技术；（2）基于生成对抗网络的隐私泄漏脆弱性检测技术；（3）基于特征空间变换的对抗样本自动生成与攻击检测技术；（4）面向多重脆弱性的深度学习系统防护与加固机制。本项目为多方协同深度学习系统脆弱性的全面高效检测和安全防护的提供了思路和方法，为物联网及人工智能系统安全保障提供技术支持。

多方协同深度学习系统是由多个数据拥有者利用自身数据进行协同训练的新型学习系统，在物联网、边缘计算等领域具有十分重要的应用前景。然而包括深度学习在内的机器学习模型和算法，在多方协同环境下具有训练数据污染异常、隐私泄漏以及对抗样本攻击等多重脆弱性。本项目针对多方协同的机器学习和深度学习系统面临的隐私和安全问题，围绕模型预处理、训练和推理过程，研究了以下内容：基于概率图模型和吉布斯采样推理的训练数据群体异常脆弱性检测技术；基于堆叠自编码器网络的对抗样本攻击脆弱性检测技术；基于随机同义词替换的深度学习模型对抗性攻击防护和鲁棒加固技术；面向隐私安全的多方协同无监督学习框架。本项目的实施为建立安全的多方协同深度学习系统，以及为数据隐私安全计算在工业领域中的应用提供了理论和技术基础。