云环境下的移动视频适配技术研究

Along with the popularization of intelligent mobile devices and the technical development of cloud computing, the combination of mobile video and cloud platform is the future trend of multimedia services. The mobile video adaptation technique under the cloud environment is the core technology to solve the contradiction between the mass video content in cloud and the limited resources for mobile device and network. The related research has theoretical and practical value. This project will introduce the human visual perception and cognitive characteristics, and mine the association of cloud data, in order to achieve the adaptation of resources and video content. The research includes three layers: the theoretical model, the key technology and the verification platform. By the fusion of spatial-temporal features and the association of multimodal information, a spatial-temporal attention model is established. The construction of multi-level tree structure is employed for multi task learning and information sharing mechanism, and the multiple source information is used to overcome the variety of cloud data categories and the information asymmetry. Thus an efficient semantic analysis model is build. Through the cloud platform, crowd data of subjective visual quality assessment are obtained to do online learning for the construction of visual quality assessment reference library. Then the library is used to guide the construction of the subjective metric model for visual distortion assessment. Based on above models, some key technologies are studied, such as visual distortion-computational complexity-rate joint optimized video coding, and the scalable content extraction with user preferences. Finally, a video adaption prototype system for mobile cloud is established for the integration and verification of above models and methods.

随着智能移动设备普及和云计算技术发展，移动视频与云平台的结合是未来多媒体服务的趋势。云环境下移动视频适配技术是解决云中海量视频内容与移动终端和网络有限资源之间矛盾的核心技术，具有理论和应用价值。本项目拟通过引入人的视觉感知和认知特性、挖掘云中群体数据关联两个突破口，实现云环境下的资源适配和内容适配。研究内容包括：拟融合时空域特征，关联多模态信息，进行时空关注建模；构建树状结构多层次多任务学习与信息共享机制，利用多源信息克服云中数据类别不均衡性和信息分布不对等性，构建高效语义分析模型；通过云端采集群体主观评测数据并在线学习，构建视觉质量主观评价参考库，指导构建与主观一致的视觉失真客观度量模型；最后基于上述模型，研究视觉失真-计算复杂度-码率联合优化的自适应编码、符合用户偏好的可伸缩内容适配等关键技术。

云环境下移动视频适配技术是解决云中海量视频内容与移动终端和网络有限资源之间矛盾的核心技术，具有理论价值和应用价值。本项目围绕海量视频数据的“视觉感知特性分析及其可计算模型构建”、“与主观感知一致的视觉失真度量模型”、“视觉失真和用户体验优化的鲁棒网络传输和服务”等关键问题，按照计划和目标进行了研究，取得了相应的研究进展和技术突破。并搭建了面向移动视频应用的验证测试平台，对所提出的“基于优化光流和重心偏向的视频显著性预测方法”、“基于双流卷积神经网络融合的视频显著性预测”、“融合语义信息的非自然视频显著性预测”、“针对网页注视点分布的显著性预测”、“基于生成质量特征图的无参考图像质量评价”、“基于CNN和多回归的无参考视频质量评价”、“数据驱动与经验驱动融合的无参考视频质量评价”、“基于视觉失真优化的视频编码动态码率控制”、“基于视觉关注对象分割的视频编码传输优化”、“动态背景模型和轮廓修正”、“基于软件定义网络(SDN)的多路径QoS解决方案”、“基于视觉关注分段MDP值迭代自适应码率控制”等一系列相关理论模型和技术方法进行了验证、测试和改进，为实际应用提供了有价值的参考解决实例。