大数据环境下保护用户隐私的半监督学习算法研究

In the big data era, it is more convenient to use the machine learning algorithms in massive data mining. However, the process of data analysis is often accompanied by the disclosure of sensitive individual information. Recently, a lot of semi-supervised learning algorithms have been proposed in the machine learning field. But how to protect users' privacy in semi-supervised learning has not yet been addressed. This project is devoted to the study of privacy-preserving semi-supervised learning algorithms. The main subjects of this project include the design and sparse analysis of the graph regularized semi-supervised algorithm with privacy preserving property, and the theory analysis for their prediction error; The design of entropy regularized semi-supervised algorithms. At the same time, we will provide the error analysis of entropy regularized semi-supervised algorithm, and obtain their convergence rates. Finally, the privacy-preserving performance of the proposed formulations will be measured. The most challenging part in the graph-based semi-supervised algorithm is how to achieve the balance between sparsity and accuracy, and improve the learning rate with the help of the large amount of unlabeled data. While in the design of privacy preserving entropy based semi-supervised algorithms, the challenge is how to give an error analysis scheme. The expected result of the project is to protect the privacy of users in semi-supervised learning under the graph and entropy regularization, and give an analysis for their sparsity, convergence and stability.

在大数据环境下，机器学习算法能够更便捷地被用来获取和挖掘海量数据。然而，数据被学习的过程往往也伴随着用户隐私的泄漏。目前，在机器学习中涌现出了大量的半监督学习算法，但是如何在半监督学习的过程中保护用户的隐私还是一个有待解决的问题。本项目致力于研究保护用户隐私的半监督学习算法，其主要内容包括：保护用户隐私的图正则化半监督算法的设计与稀疏性研究，并从理论上给出误差分析；构造保护用户隐私的熵正则化半监督算法。同时，提出熵正则化半监督算法的误差分析方案，获得其收敛速率。最后，展开对设计算法的隐私保护性能分析。研究的关键问题包括如何在基于隐私保护的图正则化半监督算法中实现稀疏性与预测准确性的平衡，并提高学习速率，以及提出基于隐私保护的熵正则化半监督算法的误差分析方案。项目预期成果是在图正则化与熵正则化的半监督学习中保护用户的隐私，并从理论上分析算法的稀疏性、收敛性与稳定性。

本项目研究大数据环境下保护用户隐私的半监督学习算法的设计与理论分析。传统的半监督学习算法主要是集中式地学习数据，进而训练出最优的学习算法。因此，这类半监督学习算法在对所有的数据进行处理过程中存在隐私泄露的风险。同时，由于采用了均方误差准则，使得这类半监督学习算法在处理非高斯分布的噪声时表现差强人意。因此，我们提出了一种新的具有隐私保护性能的半监督学习算法，其采用了最大相关熵准则。该算法将训练数据随机划分到不同的子学习机器上进行训练，降低了运算复杂度，而且避免了对数据的共享。在大数据环境下，这使得半监督学习算法降低了对敏感信息的共享，从而达到了具有隐私保护性能的目的。并且，区别于已有的半监督学习算法，这一算法未采用传统的平方损失函数，而是建立在最大相关熵函数的标准上。其在处理非高斯分布的噪声时仍然具有较好的学习性能。我们从理论的角度说明了这种具有隐私保护性能的半监督学习算法具有较快的收敛速率，同时呈现出有效的隐私保护性能。在合成数据与真实数据集合上的实验表明，该算法相较于已有的半监督学习算法具有良好的回归学习性能和隐私保护性能。