渐进式预测与深度嵌入的显著性检测方法研究

Confronted with the meet of accurate, robust and fast salient object detection, we will systematically analyze the applications of deep learning theory in machine vision, explore and explain the semantic behavior and prediction logic of convolutional neural networks, and find the internal relations between deep convolution process and human visual attention mechanism. We will address some important problems, for example, existing methods don't fully exploit the local-to-global, coarse-to-fine semantic cues embodied in different convolution layers, large scaling of feature maps easily results in inaccurate or noisy features, the non-structural loss functions that measures pixel-wise difference between predicted map and ground-truth are difficult to capture the structural information of salient object. We will propose progressive prediction architecture, and deeply investigate residual learning, boundary-boosting learning and adversarial learning to complementarily utilize the multi-scale features across different layers, combine semantic and spatial information, and develop structural supervision, thereby enhancing the capability of globally precise location and locally fine prediction. Considering that the deep neural networks that are pre-trained on image classification data possibly make the learned end-to-end mapping difficultly adapt to saliency prediction task in the unknown scene, we will use embedding learning to generalize the end-to-end mapping. Through the above research work, we hope to achieve unconstrained and efficient saliency detection even in complex scene.

面向显著性目标准确、鲁棒且快速检测的实际需求，本课题将系统地分析深度学习理论在机器视觉任务中的应用，探索理解卷积神经网络的语义行为和预测逻辑，发现深度卷积过程与人视觉注意机制的内在联系。针对现有深度方法未能深入挖掘并充分利用卷积网络各层级特征所描述的局部到全局、精细到粗旷的语义线索，大的特征尺度变换容易产生错误的或有噪声的通道特征，以及依据非结构性损失的监督训练难以捕获目标丰富的结构化信息等重要问题，拟搭建渐进式预测的网络架构，通过残差学习、边界增强学习以及对抗学习解决网络深浅层特征间的互补利用、语义与空间信息的融合以及结构化损失的对抗训练，增强卷积网络对显著性目标全局准定位与局部精预测的能力。针对用分类数据集预训练网络学习到的端到端映射难以鲁棒地适应未知场景的显著性预测问题，通过深度嵌入学习来提高网络映射的泛化推理能力。通过上述研究，最终实现复杂场景下无约束的显著性目标快速精准检测。

面向显著性目标准确、鲁棒且快速检测的实际需求，本课题探索理解卷积神经网络的语义行为和预测逻辑，发现深度卷积过程与人视觉注意机制的内在联系，构建了渐进式预测的网络框架，充分利用各层级特征的语义线索，提出了多尺度交互网络（CVPR2020）、门控网络（ECCV2020）、多视角（ICCV2019）、多阶段优化（TIP2020）、混合尺度（CVPR2022）的显著性检测模型。本课题发现不同模态、任务、监督信息的内在联系，构建了多模态融合的网络框架，充分利用多种模态的语义线索，提出了动态滤波（ECCV2020）、双向关系对齐（CVPR2021）、多源融合（ACM MM2021）、多任务学习（TNNLS2021/TIP2022）、自监督学习（AAAI2022）、弱监督学习（ICCV2019/TNNLS2022）、主动学习（TIP2020）的推理模型。在本项目资助下，课题组共发表学术论文21篇，其中SCI检索9篇，中科院一区5篇，在顶级会议CVPR/ICCV/ECCV/AAAI/ACM MM发表论文11篇。申请发明专利3项。