真实场景全视觉信息重建与生成

Full visual information stands for the information related to human vision in a real scene, containing the geometry, motion and appearance. The reconstruction and generation of visual information are becoming hot topics in multimedia processing, and are gradually being applied to various scenarios such as military, education, medical operation, industry design, and cultural industry. However, only partial visual information can be processed with existing techniques, which largely restricts the ability to capture the real world and synthesize a virtual world..This project will treat all the aspects of the full visual information as a whole and aim to achieve efficient and general representation of the full visual information and the ways to accurately reconstruct and vividly synthesize it. First, by investigating the relationship of different types of visual information, we aim to build an efficient and general representation of the full visual information based on multi-scale voxel. Second, we plan to build a multi-source multi-view capturing system and reconstruct the full visual information by integrating both data-driven and physical-driven technologies. Third, based on the previously reconstructed results and the rules among the different types of visual information, we propose methods to generate realistic geometry, motion and appearance with full flexibility in modifying each of them..This project focus on the basic principles, algorithms and applications related to the full visual information. Thus will contribute to promoting the fundamental theory, developing technology and extending the applications of the full visual information.

全视觉信息定义为真实场景中与人类视觉相关的信息，包括几何、运动和外观。对这些信息的重建与生成，已成为多媒体信息处理的前沿热点方向，逐渐应用于军事，教育，医学，工业设计，文化传媒等领域。然而，当前技术只能对部分视觉信息进行处理，大大局限了人类对现实场景的感知能力和对虚拟场景的生成能力。.本项目拟将全视觉信息作为整体进行研究，旨在实现全视觉信息的高效统一表达，高精度重建和高真实感渲染生成。首先，研究全视觉信息各要素的内在关联机制，建立基于多尺度体素的全视觉信息高效、统一表达框架；其次，搭建多源多视数据采集平台，研究数据、物理联合驱动的高精度全视觉信息解耦重建；最后，基于真实场景重建结果和各视觉信息的相互作用机理，实现几何、运动、外观自由可控的高真实感渲染生成。.本项目将围绕全视觉信息的基本原理，实现方法和实际应用三个方面展开工作，为全视觉信息处理技术的发展提供理论基础、技术支撑和应用拓展。

全视觉信息定义为真实场景中人类视觉可观察到的信息，包括几何、运动和外观。对这些信息的重建与生成，对虚拟现实中的真实对象数字化和人机交互中的人类表情和姿态识别具有重要的意义和应用价值。本项目从基础理论，核心算法和前沿应用方面做出了卓有成效的研究，建立了基于三维体素的视觉信息高效表达框架，提出了针对人脸、人体、人手和场景的快速重建与生成方法，并在多个厂家的智能终端系统中得到实际引用。项目发表CCF A类论文15篇，申请国家发明专利10项，项目成果作为创新点之一获得电子学会技术发明奖一等奖一项，项目技术实现直接知识产权转化200余万元。