基于水下视觉的海洋鱼类细粒度识别和鲁棒跟踪研究

The species, quantity, distribution and behavior of marine fish recorded in the underwater videos are important information source for marine ecological monitoring and biodiversity analysis. In recent years, with the rapid increase of underwater videos, automatic fish recognition and tracking is becoming an urgent technique for underwater monitoring. .This project aims at investigating the key technologies of fine-grained recognition and robust tracking of marine fishes based on underwater vision in real underwater environment. However, due to the complexity of natural underwater environment, wide variety of marine fishes, high appearance similarity between different fish species, and diverse appearances caused by different viewpoints and non-rigid deformation, accurate fish recognition and robust tracking is a challenging task. In this project, we attempt to solve these problems through the perspective of multimodal information mining and multiple cues fusion. The main research content includes: image dataset expansion based on random local regions, fine-grained recognition of marine fish based on multi-modal appearance features, robust tracking based on multiple clues fusion, and coordination of marine fish recognition and tracking based on multitask multiview model, and these are significant to deal with crucial challenges such as insufficient of diversity training images, indistinguishable appearance of fishes under the same family and strong randomness of fish motion direction. In summary, our research will enrich the theory and application of fish recognition and tracking in the complex and real underwater environment, and provide practical algorithms and technologies for automatic analysis of underwater videos.

水下视频中记录的海洋鱼类种类、数量、分布和行为数据为海洋生态监测和生物多样性分析提供重要的数据支撑。近年来随着水下视频数量快速增长，对水下视频自动分析技术的需求日益凸显。.本申请拟面向真实水下环境开展鱼类细粒度识别和鲁棒跟踪关键技术研究。水下环境复杂、鱼类种类繁多、外观相似性高及体态视角多样化等为鱼类识别和跟踪带来了极大挑战。针对该问题，本申请从多模态信息挖掘和多线索融合的角度对其进行深入系统的研究，研究内容包括：基于随机局部区域的海洋鱼类图像数据扩展、基于多模态外观特征的海洋鱼类细粒度识别、基于多线索融合的海洋鱼类鲁棒跟踪和基于多任务多视图模型的鱼类协同识别与跟踪，旨在解决水下鱼类视频和图像多样化训练数据缺失、同科鱼类外观差异不显著和鱼类运动方向强随机性等为鱼类识别和跟踪带来的切实困难及科学问题。研究成果将丰富复杂真实水下环境中鱼类识别和跟踪的理论与应用，为水下视频自动分析提供关键技术。

基于视觉的水下目标识别和跟踪可用于实现对海洋鱼类等目标的种类、数量、分布与行为等的智能分析，为海洋生态监测、海洋生物多样性分析和精细化水产养殖等领域应用提供技术方法支撑，具有重要的研究价值和广阔的市场应用前景。当前，复杂场景中的目标细粒度识别与鲁棒跟踪等方面依然存在诸多技术挑战，难以满足实际应用场景中对目标的精准识别和跟踪需求。.本项目对基于视觉的目标细粒度识别和鲁棒跟踪中的关键科学问题开展深入研究，主要包括：复杂场景下基于视觉的细粒度目标识别、复杂场景下基于视觉的目标鲁棒跟踪、基于多任务学习的目标识别和基于深度哈希的目标检索。本项目的主要贡献包括：通过创新关键区域定位和区分性特征提取方法等显著提高了复杂场景下目标细粒度识别的准确率；通过有效融合时空机制等多线索，强化目标表征建模，提高了目标跟踪的准确率和鲁棒性；通过强化模型的空间表达能力和对目标形状差异的适应能力，提高了多任务学习中目标识别的准确率；通过提高哈希编码的质量，提高了目标检索的准确率。.本项目研究期间，发表学术论文26篇，其中SCI收录的国际期刊论文23篇，获发明专利授权7项，形成的视觉分析关键算法和模型等成果进一步丰富了目标识别和跟踪的方法体系，拓展了应用场景。项目培养相关领域方向硕博士毕业生10余名，进一步凝聚了科研方向，提升了团队的系统化科研能力。