稀疏与低秩表征的关键理论及其在大数据中的应用研究

The Internet has great effect on people’s social affairs in 21st century by new means of social network, internet finance, e-commerce and so on. Internet-based data increase so incredible that the era of big data is at hand, which means data are everything. In recent years, the theory of sparse and low rank representation is becoming a powerful tool to deal with high-dimensional, non-linear and non-structural data. Based on this paradigm, this project focuses on three different but closely-related aspects. The first one is the model selection of sparse and low rank representation. Regularization based on data property, discrimination and graphical structure is the key at this stage of the project. The second one is to formulate fast and efficient algorithm, based on randomly approximate theory, to solve the new model built in the first step. Randomized sampling and randomized projection will be taken into account when designing new randomized methodology. The last step is to develop verification platform under IBM cluster with Mapreduce/GraphLab architecture for vision-based big data in order to make our new theory into reality. This open platform also provides data interface, application interface and feedback to the proposed model and theory.

进入二十一世纪，互联网以新的形式，如社交网络、互联网金融、电子商务等，深刻影响着人们的工作和生活方式。数据的海量增长使得数据为王的大数据时代到来。基于稀疏与低秩表征的理论是近年涌现出来的一种鲁棒高效地处理高维、非线性、非结构化数据的新方法。围绕“大数据的稀疏与低秩表征”这一科学问题，项目的研究内容分为紧密联系而侧重不同的三个方面：（1）研究基于稀疏与低秩表征的模型选择，重点在基于数据特性、判别性和图结构的正则化设计方面开展工作。（2）研究稀疏与低秩模型的随机近似快速求解算法，重点在随机化采样和随机化投影方面开展研究工作。（3）构建稀疏与低秩表征面向视媒体大数据的开放系统平台（集群+Mapreduce/GraphLab ），提供开放式的数据接口、应用接口，为完善稀疏与低秩表征的算法理论体系和设计方法提供反馈。

本项目以压缩传感、稀疏与低秩表征理论为基础，面向高维、非线性、非结构化大规模数据，构建了稀疏与低秩表征的新模型与新算法，设计并实现了原型系统平台，为基于互联网的视媒体大数据处理提供新方法与新技术。研究成果体现在：.1.基础理论研究方面，提出了基于视媒体大数据的稀疏与低秩表征的新理论和新方法，结合多任务/多示例学习、半监督学习、迁移学习等机器学习方法，用于指导视媒体大数据的处理，如语义理解、综合检索、目标分类等任务。.2.关键技术研究方面，面向视媒体大数据，以随机算法为核心，设计了快速有效的稀疏与低秩模型的求解新算法，在稀疏与低秩表征的基础理论与解决视媒体大数据的实际问题之间搭建扎实的桥梁，提供了技术保障。.项目组在CCF A/B类期刊与会议上发表论文20余篇，申请发明专利15项，为企业解决技术难题多项。