无约束场景下端到端深度视觉匹配研究

As one important preliminary for multiple 3D vision and machine perception tasks, visual correspondence is a long-lasting research interest in the community of computer vision researchers. However, either traditional methods that depend on hand-craft feature engineering or existing deep alternatives are vulnerable under unconstrained scenarios. To achieve sufficient robustness and reliability for visual correspondence in complex and unconstrained visual scenarios, this project focuses on the end-to-end learning of feature representations and correspondence models that are adaptive to unconstrained visual changes. It is achieved by (1) building a large-scale CG dataset for unconstrained visual correspondence and studying task-driven deep visual correspondence model designs and their training schemes based on the synthetic dataset; and further improved by (2) learning adaptive matchable features based on self-supervised feature disentanglement scheme, and (3) jointly constraining the feature representation and correspondence distribution according to the physically sound criterion of cycle consistency. The research achievements of this project will enrich the theories and methodologies in the area of deep visual correspondence and improve the performance and practicality of correspondence-based 3D vision algorithms. Moreover, they are also valuable in increasing the compatibility of deep learning models in a much broader area of computer vision.

作为三维视觉和多种机器认知任务的基础，视觉匹配技术一直是计算机视觉研究领域的热点之一。然而，不论是依赖于人工特征工程的传统匹配技术，还是现存深度匹配技术，往往难以胜任无约束复杂视觉场景下的视觉匹配任务。面对视觉匹配鲁棒性和合理性的需求，本项目深入开展适应无约束视觉条件的深度特征和匹配模型的端到端学习研究：（1）建立大规模基于计算机渲染的无约束视觉匹配数据集，并利用该数据集研究任务驱动的端到端深度视觉匹配模型设计和训练机制。为了进一步增强视觉匹配的性能，（2）克服特征表达对场景匹配无关视觉信息的敏感性，研究基于特征解构的自适应匹配特征学习；（3）强化匹配结果的物理合理性，研究利用匹配循环一致性来联合约束特征表达和匹配分布。本项目的研究成果可以丰富深度视觉匹配理论和方法的研究，改善基于匹配的三维视觉算法的性能和实用性，对提升深度学习在计算机视觉领域的兼容性也有重要意义。

在项目执行期内，主要针对无约束场景下计算机视觉任务中视觉匹配鲁棒性和合理性问题，研究任务驱动的深度视觉匹配模型设计和端到端训练机制，从自适应匹配特征学习和强化匹配物理合理性两个角度，结合下游丰富的二维和三维生成、配准、重建和跨模态感知等应用任务，提出一系列面向无约束场景、以视觉匹配为核心模块、可端到端训练的计算机视觉应用算法。作为本项目负责人，一共发表论文12篇，含发表人工智能和计算机视觉领域的国际知名学术期刊4篇，国际知名学术会议8篇；其中包含CCF推荐A类论文9篇。研究成果申请国家发明专利4项，其中已授权2项。参与CVPR 2021三维物体定位挑战赛获得冠军。