基于多穿戴式相机的视频行为检测

Video activity detection has wide applications in public security and our daily life. The current technology is focused on the videos that are collected using a fixed camera or a camera network made up of multiple fixed cameras. As a result, the collected videos can only cover limited spatial areas from pre-specified view angles. In recent years, the rapid advancement of the wearable-camera technology, such as GoPro and Google Glass, provides a new perspective for wider-range video activity detection. The videos collected by wearable cameras not only expand the spatial coverage for the activities of interest, but also well reflect the intelligence and perception of the camera wearers. However, the continuous motion and the possible sudden view-angle change of the wearable cameras also bring new challenges to video activity detection. The goal of this project is to systematically investigate the video activity detection using multiple wearable cameras, reveal and address the underlying key scientific problems, and finally implement an effective and flexible prototype system for video activity detection using multiple wearable cameras. Specific proposed research topics include: crowded human tracking on wearable camera videos; cross-video person identification; detection of human activities with varying durations; activity recognition from incomplete videos; and collaborative activity recognition by integrating videos from multiple, temporally synchronized wearable cameras. The proposed research will extend the theory and technology of video activity detection from fixed cameras to wearable cameras and significantly expand its future prospects.

视频行为检测在社会公共安全和民众日常生活中有着广泛的应用。目前常用技术主要使用固定相机（单个或相机网络）来实现视频数据采集，这将导致固定的、有限的视频覆盖角度和区域问题。近年来穿戴式相机的快速发展和应用为大范围视频行为检测提供了崭新的途径，它们不仅极大地拓展了视频覆盖的空间，同时能很好体现穿戴人的智能感知。然而，穿戴式相机的常态化的移动和时常的视角突变为视频行为分析带来全新的挑战。本项目旨在系统性地研究基于多穿戴式相机的视频行为检测，发现和解决其中关键的科学问题，并实现一个有效且可扩展的穿戴式相机视频行为检测系统。具体的研究内容包括：基于穿戴式相机视频的人群跟踪；跨视频人的识别；不同持续时间的人体行为检测；不完整视频的行为识别；基于多穿戴式相机同步视频的集成行为检测。本项目的研究将视频行为检测的理论和技术从固定相机推广到穿戴式相机，极大地扩展了视频行为检测的应用前景。

视频行为检测是有着广泛应用前景的基础研究课题。目前常用技术主要使用固定相机（单个或相机网络）来实现视频数据采集，这将导致固定的、有限的视频覆盖角度和区域问题。本项目利用多个穿戴式相机组成的移动相机网络来克服这些局限性。多穿戴式相机可以从不同视角获取更丰富的视频信息，但是其常态化的移动和时常的视角突变为视频行为检测带来全新的挑战。基于此，本项目研究了以下几个关键问题：1）跨视角人的识别，2）视角突变下的视频跟踪，3）移动相机动作识别，4）降质图像对识别和跟踪的影响，5）同一场景的协同显著性。其中，本项目采用深度神经网络技术集成表观、运动和姿态这些互补特征大大改善了跨视角人的识别精度。对于穿戴式相机使用中常见的视角突变所带来的目标丢失，本项目发现可以利用跟踪目标之间的空间分布特征来实现重新检测和跟踪。移动相机动作识别主要研究的是降低对动作过程中视角变化的敏感性，对此本项目通过视频分段建模，辅以多视角固定相机视频的训练数据，来达到目的。对于降质图像的影响，本项目通过系统化的实验得到丰富的定量结果。同时还发现当前先进的提高图像质量方法，如去模糊算法，不一定有助于最终的识别和跟踪。同一场景的协同显著性则是本项目提出的一个新问题，在中层视觉层面上检测同一场景中特征相似的目标组成的群体，对相关场景的分析可以起到促进作用。这些具体问题的研究，将促进视频行为检测的理论和技术从固定相机推广到穿戴式相机，扩展视频行为检测的应用前景。