基于显著性和信任传递的动态场景主题发现

Natural scenes are usually composed of several dynamic entities. Foreground objects often move amid complicated backgrounds that are themselves moving, e.g., swaying trees or other objects such as a crowd, a flock of birds, moving water, wave's snow, rain, and smoked-filled environments. Even for static scenes, egomotion of the imaging sensor can cause a highly variable background. In the most extreme situations, egomotion and scene motion combine to produce very complex motion patterns. We refer to scenes with any of these types of variability as dynamic scenes. Because both spatial and temporal dependencies between moving objects are relevant, so study of the extraction algorithm of content and semantic in dynamic scenes is critical significant to scene understanding.. Human visual system has a remarkable ability to quickly grasp salient regions in static and dynamic scenes. Efficient belief propagation of early vision, generative models and the Bayesian brain hypothesis provide a framework that can quantitatively model interaction and inference between prior knowledge and sensory evidence to represent the physical and statistical properties of the environment. In this work, we address dynamic scene understanding and want to automatically answer questions such as: What are the typical objects in the scene? How do they relate to each other? What are the rules governing the scene? How to structure the semantic-visual scene hierarchy? To this end, we present a novel method to automatically discover topic based on saliency and belief propagation inference algorithm. The main contributions of the proposed method can be summarized as follows.. First, we present a method that builds on a mixture spatiotemporal saliency detection model from down-top and top-down visual pathway by belief propagation and hierarchical Bayesian networks by robust local and global static and motion feature as well as geometry context information, and predict the future positions of moving objects in a dynamic scene using intrinsic dimensionality and flow-based feature.. Second, we propose a novel way to analyze dynamic scenes from the perspective of social relationships. To appropriately describe objects' relationships, we devise a method to quantify relations and construct objects' social networks in order to perform semantic analysis. Social relations between objects are used to be the context information, and leading objects and corresponding communities can be automatically determined.. Last, we present a novel generative model by saliency and belief propagation from single and multiple frame using spatial learning topic model and dynamic relational topic model by belief propagation respectively,in order to discover dynamic scene topic.Semanti-visual scene hierarchy can be constructed to segment story in.dynamic scenes.

动态场景因运动复杂、多变，研究其内容和语义提取对理解场景有非常重要的意义。. 本项目针对海量信息快速抽取、低层特征和高层语义之间存在鸿沟等问题，在视觉注意和选择性的基础上，结合主题模型和信任传递机制，提出基于显著性和信任传递的动态场景主题发现机理。. 研究内容及创新点：1）具有预测性的混合时空显著计算模型。提出基于信任传递和贝叶斯网络的混合时空显著模型，并利用本征维和流失量信息预测后续显著位置；2）场景社会关系学习。提出以场景片段中显著对象出现和共现关系建立社会关系网络结构，发现场景重要显著位置；3）动态场景主题发现。提出空间信任传递主题模型及动态信任传递关系主题模型分别学习单帧和场景的语义可视层次结构，建立动态场景的总的语义可视层次结构，完成场景故事层面的分割。

项目是针对复杂动态场景中自我运动和场景运动导致的显著图表示有噪声、不完整，缺乏后续显著对象预测和场景层次组织结构关系描述，以及主题学习演变等问题，提出显著性和信任传递学习主题模型相结合的场景主题发现方法。研究内容主要包括显著性计算模型、主题模型、复杂场景语义描述以及特征描述和提取4个方面的。. 在显著性模型计算方面，主要从静态图像的空间显著性检测入手，逐渐拓展到动态图像中的时空显著性检测和协同显著性检测的研究。研究成果主要体现在：在静态图像的显著性检测方面，彩色图像中显著对象的自动抠取方法和非清晰区域抑制的空间显著性检测方法获得中国发明专利授权，发表SCI论文1篇；在时空显著性检测方面，融合时间显著性和空间显著性的显著性检测方法以及基于慢特征的时空显著性检测方法申请发明专利。. 主题模型研究方面，主要研究了PLSA模型和LDA模型的在线学习算法以及基于TMBP模型的动态场景分类、人物行为识别。研究成果主要体现在：在线主题模型学习算法方面，发表SCI论文1篇，计算机学报论文1篇，申请发明专利1项；基于TMBP模型的动态场景分类方面，发表SCI论文1篇，申请发明专利1项；基于TMBP人物行为识别方面，申请发明专利1项。. 在复杂场景语义分析方面，主要研究了固定摄像头下复杂行为语义解析、时空显著性下视频场景解析以及有关社会结构分析的研究。研究成果主要体现在社会结构分析方面发表SCI论文1篇；基于HSV空间在线聚类的视频背景提取方法获得发明专利授权；一种运动目标检测方法获得发明专利授权；用于脱机手写汉字识别的汉字字符切分方法获得发明专利授权。. 在特征描述和特征提取方面，主要研究了特征降维、基于SIFT流的动态场景描述、深度学习下的自动特征表达以及强化梯度下降曲线形状拟合的神经结构描述。研究成果主要体现在特征降维方面的鉴别稀疏保持嵌入的算法发表在自动化学报上；基于SARSA的强化梯度下降曲线形状拟合的形状描述方法发表在SCI期刊上。