适应移动云计算的自由视点合成、自适应卸载与功耗优化

Cloud computing has demonstrated its advantages in enduing the thin terminals with the capability of enjoying multimedia applications, which are expensive in computational complexity and storage, by offloading. However, for free-viewpoint video, which draws the greatest attention nowadays, applying offloading to the existing view synthesis techniques, which involve large intermediate data and high complexity under the constraint of synthesis quality, cannot guarantee the energy saving for the mobile devices. Consequently, targeting energy minimization of mobile devices, this project researches on novel view synthesis techniques for mobile cloud computing. By theoretical analysis, statistical learning and feedback optimization, novel view synthesis techniques are proposed, using character sampling, image，depth and geomatry prediction, based on representation of depth DCR contours, based on inter-view and temporal motion compensation and based on texture characteristic clustering to realize fine-granularity workload-decomposability from algorithm level to system level under the synthesis quality constraint. Then, through global mapping of multi-dimensional workload and the definition of energy characteriscts for the terminals, the terminal energy model of view synthesis based on mobile cloud computing is established together with real-time prediction methods. Based on the research of optimizing and controlling the power consumption under dynamic multi-constraint, adaptive offloading between the mobile devices and the cloud can be realized to minimize the energy consumption of mobile devices for free-viewpoint video applications. The research results will serve for the next generation multimedia applications moving to thin terminals and cloud, which has obvious theoretical significance and broad application prospect.

云计算通过卸载终端负载，使移动终端享用高复杂度的多媒体应用成为可能。但是，对于当前备受关注的自由视点视频，因其现有的视点合成技术在合成质量约束下生成的中间数据量大、技术复杂度高且不易拆分，直接应用云计算卸载无法保证移动终端功耗的降低。本课题研究适应于移动云计算的视点合成技术，以实现移动终端功耗最小化为目标，利用特征抽样、图像、深度与几何信息预测，研究基于深度DCR等高表征、基于视间、时域运动补偿以及基于图像特征聚类的新型视点合成技术，在合成质量约束下实现系统层到算法层的跨层数据/负载精细可拆分。进而通过多维负载的全局映射和终端功耗特征定义，建立面向移动云计算的视点合成终端功耗模型和实时动态多约束优化与控制方法，实现视点合成负载在移动终端和云端间的自适应卸载，使终端实现自由视点合成的功耗最优化。该研究服务于下一代多媒体应用向轻型终端的演进和"云"化发展，具有重要的理论意义和广泛的应用前景。

研究针对移动终端的实时交互式FTV这一未来的新型应用，以实现移动终端功耗最小化为目标，利用特征抽样、图像、深度与几何信息预测，提出了一系列新型视点合成技术，建立基于移动云计算的终端功耗模型和自适应卸载与优化方法，构建基于移动云计算的交互式FTV架构，达到终端功耗最小化的目标。项目执行完成全部预定目标：发表SCI检索高水平国际期刊论文6篇，EI检索国际重要学术会议论文25篇；授权国家发明专利9项，公开相关国家发明专利8项，申请PCT 2项，美国发明专利1项；培养2016年“广东特支计划”科技创新青年拔尖人才1名、高质量的博士和硕士10名；项目执行期间，荣获多项学术荣誉，包括：2016年国家科技进步二等奖、2016年广东省科学技术一等奖和清华大学2015年优秀硕士学位论文；从2014年到2017年期间，项目组成员参加图像处理领域的著名国际会议ICIP、ICME、ISCAS、VCIP、PCM 27人次，报告学术研究成果，提升研究成果的国际影响力和知名度。