高维材质外观高效采集的设备、理论与方法研究

With the rapid development of virtual / augmented reality industry, the demand for high-precision, high-dimensional material appearance is surging, in order to support the immersive experience that is seamlessly integrated with the real world. High-dimensional material appearance essentially characterizes the interactions between light and materials, so that it can be used to render the complex appearance variations that change with lighting and view conditions as in the physical world. However, one fundamental difficulty with material modeling is the sheer complexity. The efficiency of current approaches cannot meet the massive demand on realistic materials, which hinders the development of virtual / augmented reality industry. In order to tackle this problem, we first conduct research on high-angular-resolution illumination, multi-view, desktop-sized high-precision acquisition device for general high-dimensional materials, in order to capture a large number of high-quality appearance data from a variety of daily materials. Starting with these data, we analyze their features as a prior, and derive optimal sampling theorem as well as related sampling-reconstruction algorithms on high-dimensional materials. Finally, we explore related applications. We expect that the research products of this project will increase the efficiency of digital material modeling and decrease its cost, pushing forward the development of virtual / augmented reality industry in our country.

随着虚拟与增强现实产业的高速发展，人们对高精度高维材质外观的需求不断增加，以支持与真实世界无缝对接的沉浸式体验。高维材质外观从本质上刻画光与材料之间的交互作用，能真实地呈现物理世界中随光照和视角变化的复杂外观变化。然而，材质建模存在着高复杂度的根本困难，现有方法的效率无法满足对高真实感材质的大量需求，阻碍了虚拟与增强现实产业的发展。针对这个问题，我们首先将有针对性地研发高光源角度分辨率、多视角、桌面型通用高维材质的高精度采集设备，用来获取大量不同种类日常材料的高质量材质数据；从这些数据出发，我们将分析其特征作为先验，进而推导高维材质最优采样理论和相关采集重建算法，并以此为基础探索相关应用。我们相信本项目的研究成果将提高材质数字化建模的效率和实用性，降低建模成本，有力推动我国虚拟与增强现实产业的进一步发展。

针对高维材质外观的高效高质量采集重建问题，我们在项目执行期内主要沿着两个主要方向进行了探索：在高维外观采集建模方向上，分别对结构化/非结构化输入的平面/非平面材质外观开展了研究；在外观与几何的联合采集建模方向上，分别对复杂物体的高效联合采集建模以及复杂材质下的高精度三维重建开展了研究。.项目组在国际上率先提出了可微分外观采集建模架构，揭示了外观建模与跨领域编解码之间的联系，首次把采集重建过程完整映射到了神经网络，创新性地实现了全自动、端到端的软硬件联合优化，把整体效率提升了近1个数量级，解决了高维外观建模效率低这一根本性难题。针对非结构化输入数据，揭示了自由式外观扫描和非结构化几何学习之间的联系，设计了能有效聚合非规则测量信息并同时优化采集光照条件的新型神经网络，实现了各向异性复杂外观的自由式采集重建。针对未知几何与外观联合建模问题高度欠定的挑战，提出了可微分联合采集建模方法，通过自动学习到的采集光照，在图像域中精确解耦了几何与外观，分离了视角/光照不变的关键性质，并以此来自动计算鲁棒光度特征信息，提高了联合建模的精度与效率，实现了近场采集装备的非平面各向异性外观采集建模，整体效率提高了1个数量级。项目组还研制了4套具有自主知识产权的高密度光源阵列采集设备，在边长0.8米的立方体中实现了对超7万个光源的高速并行控制，角度分辨率比现有设备高1到2个数量级，在核心指标上达到了国际领先。.上述方向的探索为高维材质外观的高效高质量采集重建提供了有效的思路。研究成果发表于计算机图形学高水平国际期刊上（包括ACM TOG和IEEE TVCG），并申请授权了多项专利。项目负责人还因为在这个方向上的工作获得了2020年度国家自然科学基金优秀青年基金的资助。