基于云计算的视频智能处理与编码

Cloud computing, with its powerful capability of computing and storage and with its large-scale data resource, provides unprecedented opportunities for IT industry. However, as an emerging computing infrastructure, various heterogeneities in services, devices, networks, plus the requirements of instantaneous responses to the bursts of simultaneous large-scale data access and processing, bring about great challenges to the cloud-based multimedia applications and services. How to support complex and even unpredictable multimedia services as well as manipulate large scale data at the same time to provide good quality of experience (QoE), becomes one of the crucial and fundamental problems for cloud-based multimedia processing. This proposed project includes two main research issues: 1) cloud-based video intelligent processing technology; and 2) cloud-aware video representation and coding method. For the sake of breaking the tight coupling between multimedia processing and application, we propose in this project: 1) to investigate multimedia intelligent processing method based on latent machine learning (LML) and latent data mining (LDM), which is able to achieve scalable recognition; 2) to develop a type of cloud-aware video representation, dubbed Intermedia, based on which the fast generation of scalable video coding bit stream adaptable to diverse end devices, highly heterogeneous networks, as well as complicated service needs can be easily realized. Besides, we further propose to establish a general framework for video application and services based on cloud. In the proposed framework, video data originated from various users are first uploaded to the cloud. Then, the cloud computing platform utilizes the idle resources in the cloud to intelligently process the video data to generate its corresponding Intermedia. As a type of compact, structured, and intelligent video representation, Intermedia is amenable to various multimedia applications involving both signal level and semantic level processing. Moreover, the target video bit stream with specific frame rate, bit rate, resolution, and format can be generated from Intermedia with low complex operation. The aim of this project is to make a breakthrough in the area of cloud-based large-scale data processing and to lay the foundation for the standardization of cloud-aware media.

云巨大的计算、存储、数据资源和云中复杂应用、多样终端、异构网络、海量数据等为多媒体应用带来了机遇和挑战。本课题拟研究的关键科学问题：云环境中如何同时应对海量数据和大规模复杂应用，并能即时提供高品质多媒体服务。本课题的研究内容包括：适应云环境的视频智能处理和视频表达与编码。本课题突破多媒体智能处理和视频编码传统紧耦合的研究方式，创新性地提出：1)基于潜机器学习和潜数据挖掘的智能处理方法，可伸缩地实现高层语义的模式识别；2)基于中间媒体的视频表达形式，其通过简单处理即可快速生成适配网络环境、终端设备和复杂应用需求的可伸缩码流。本课题拟基于云平台，通过对视频数据进行潜智能处理，生成简洁化、结构化、智能化的中间媒体，根据具体应用需求，将中间媒体即时编码为适配码流，通过互联网快速流畅地传给用户。本课题预期在海量多媒体数据云处理的理论、方法、技术上有较大突破，为制定适应云环境的多媒体国际标准奠定基础。

云巨大的计算、存储、数据资源和云中复杂应用、多样终端、异构网络、海量数据等为多媒体应用带来了机遇和挑战。本项目研究的关键科学问题：云环境中如何同时应对海量数据和大规模复杂应用，并能即时提供高品质多媒体服务。本项目主要研究内容包括：适应云环境的视频智能处理和视频表达与编码。经过4年研究，项目组在上述2项研究内容取得了重要进展。在视频智能处理方面：提出了稳健的基于时空分解的视频处理方法，提出了基于移动视频定位、场景文字检测等面向移动视频的智能处理方法，提出了视觉特征提取、视觉码本学习、特征融合、几何校验等面向大规模图像检索的智能处理方法，提出了基于深度学习神经网络的视频智能处理方法等。在视频表达与编码方面：提出了多参考帧最优预测编码方法及参考帧质量分配策略，提出了面向可伸缩高性能视频编码的λ域码率控制方法，提出了高性能视频编码中的最优码率分配技术，提出了基于可伸缩编码的自适应错误弹性编码方法，提出了基于数字模拟混合编码的信源信道联合编码方案等。这些成果丰富了海量多媒体数据云处理的理论和方法，在相关关键技术上有较大突破和创新，为制定适应云环境的多媒体编码国际标准奠定了基础。项目执行期间项目组发表学术论文42篇（包含已录用2篇），包括IEEE/ACM Transactions论文17篇、CCF A类会议4篇；申请专利26项，其中13项已获授权；有5项技术提案被视频编码国际标准JCTVC (MPEG/ITU-T)接受。项目负责人2013年获得国家基金委杰出青年基金资助，2016年入选国家万人计划科技领军人才，2015年获得国家自然科学二等奖（排名2）。该项目培养博士毕业生6人，硕士毕业生10人，其中1人获得中科院院长特别奖，2人获得中科院优秀博士学位论文，1人获得中国电子学会优秀博士学位论文。项目组圆满完成了项目计划书制定的研究计划，达到了预期的研究目标。