基于多域离散数据的深度学习

Deep learning has achieved great success in cognitive pattern recognition tasks such as image and speech recognition by leveraging the capacity of deep neural network and the big data. However, on more widely data mining applications such as recommender systems and computational advertising etc., the advantages of deep learning have not been obviously revealed yet. The reason is the data type of most data mining tasks is multi-field categorical and sequential discrete data, on which few deep learning models are proposed to work. Therefore, this project aims to study the capability of deep learning models working on multi-field categorical and sequential discrete data. The main contributions are summarized as follows. (1) To design the end-to-end neural network architecture specifically working on multi-field categorical and sequential discrete data in order to address the problem of learning representation and mining inter-field data interaction patterns and thus improve the prediction performance. (2) Based on the studied neural network architecture, to propose efficient learning algorithm and automatic hyperparameter search methods to reduce the model complexity and get the model learning process accelerated and hands-free. (3) To deploy the proposed neural network model and learning algorithm onto at least two real-world large-scale-data intelligent applications, including editor article recommendation and user ad click prediction, to evaluate the efficacy and robustness of the solution. Overall, the output of this project would provide useful and insightful guidelines for the deep learning solutions for the various intelligent systems based on multi-field discrete data.

深度学习通过构建深层的神经网络模型结合大数据训练从而在图像和语音识别等感知任务中取得卓越的模式识别和预测效果。而深度学习的优势还并未明显体现在更多数据挖掘领域，例如推荐系统和计算广告等应用中，原因是这些问题的数据大多是多域的类别型或序列型的离散数据，之前少有神经网络模型直接工作在此类数据上。因此，本项目拟对基于多域离散数据的深度学习模型的有效性进行分析，具体包括：（1）设计面向此类数据的端到端神经网络架构，解决多域离散数据的表示及其交互模式的自动挖掘，提高深度学习模型的预测精准性。（2）基于新型神经网络架构，提出快速学习算法和超参数搜索算法，减低模型计算复杂度，提高模型的收敛效率和自动化程度。（3）将提出的模型和学习算法在至少两个大数据场景中落地实践，包括编辑稿件推荐和用户广告点击预测，并探讨模型的有效性和鲁棒性。研究成果将为深度学习在基于多域离散数据上的多种智能应用提供有益参考和借鉴。

本项目主要对基于多域离散数据的深度学习模型的有效性进行分析并提出新型深度学习模型。项目首先设计面向此类数据的端到端神经网络架构，解决多域离散数据的表示及其交互模式的自动挖掘，提高深度学习模型的预测精准性。其次，基于新型神经网络架构，提出快速学习算法和超参数搜索算法，减低模型计算复杂度，提高模型的收敛效率和自动化程度。此外，将提出的模型和学习算法在至少两个大数据场景中落地实践。在项目期2018年至2020年，项目组在本项目研究方向发表国际学术论文47篇，其中CCF-A类论文36篇；申请5项技术发明专利；培养了8名研究生；开源工程代码8项以上到Github。项目研究产出的深度学习模型落地在华为手机应用市场推荐系统平台和阿里巴巴广告推荐平台上，皆取得了令人满意的效果，提升了平台的业务指标。