视觉显著性引导的前景物体分割及其语义解析

The explosive growth of Internet images and videos bring new challenges to the techniques of visual content analysis. It is required that the analysis techniques should be highly efficient, accurate and intelligent so that the large-scale Internet images and videos with complex contents can be processed as the human-being does. Toward this end, this project focuses on a fundamental problem in research areas such as computer vision, neuro-informatics, machine learning and big data. That is, how to describe the common features shared by saliency, foreground objects and its semantics so as to construct the correlation model, and how to guide the segmentation and semantic analysis of foreground objects with visual saliency. To address this problem, this project will develop models and approaches for saliency-guided segmentation and semantic analysis of foreground objects, including visual saliency modulation towards foreground object segmentation, saliency-guided foreground object segmentation in images and videos, foreground object parts analysis and activity understanding. In addition, several system prototypes for important industrial domains will be developed to validate the effectiveness of the proposed models and approaches. Eventually, a fundamental supporting set of models, approaches and prototypes will be systematically constructed for foreground object segmentation and semantic analysis. A series of scientific papers will be published on the top journals and conferences. The products of this project, e.g., patents, software copyrights, benchmarks, will have independent intellectual property rights.

互联网图像视频数据规模的爆炸式增长，对视觉内容解析技术提出了新的挑战。为了解析规模庞大、内容复杂的互联网图像视频场景，并生成符合人类认知的解析结果，需要高效率、高精准的视觉内容解析技术。本项目立足计算机视觉、神经信息学、机器学习和大数据等研究领域的国际前沿，针对基础科学问题“如何描述显著性、前景物体及其语义的共性特征并构建其关联关系模型，以显著性引导前景物体分割及其语义解析”，深入开展视觉显著性引导的前景物体分割及其语义解析模型方法研究，力争取得面向前景物体分割的视觉显著性调制、显著性引导的图像视频前景物体分割、前景物体部件级语义解析及行为理解等方面的模型方法创新与技术突破，并面向重要行业领域应用需求研制技术验证原型系统，初步形成一套具有基础支撑作用的前景物体分割及语义解析模型方法系统。预期将发表系列高水平论文，以发明专利、软件著作权、基准数据集等形式，形成具有自主知识产权的研究成果。

本项目针对视觉显著性引导的前景物体分割及其语义解析开展研究，重点对视觉显著计算、前景物体分割、物体语义理解三个难点开展科研攻关，取得了基于耦合知识蒸馏的超高速视频显著计算、边界敏感的图像前景物体分割、近邻可逆的视频前景物体序列分割、面向大规模有噪声图像分类的多粒度泛在重加权方法、部件规范化的近似重复车辆重识别、循环先验指导的有序多任务部件分割等理论方法创新。在本项目资助下，发表/录用高水平论文27篇，含CCF A类期刊和会议论文20篇（2篇IEEE TPAMI，5篇CCF A类会议的Oral/Spotlight论文）；获图像识别国际挑战赛WebVision 2018冠军；申请中国发明专利22项（授权5项）、美国发明专利5项（授权4项），成果在国家计算机网络与信息安全管理中心等3家单位得到应用；培养博士生8人，硕士研究生24人，其中6人获国家奖学金，2人获北京市优秀毕业生。项目负责人入选北京市科技新星，获国家自然科学基金优秀青年科学基金资助，并获中国电子学会自然科学二等奖（排名1）。