基于视频的城市交通场景理解与建模方法研究

Much effort has been devoted recently to understanding and modeling urban traffic scenes, as solving this issue is a critical challenge for emerging applications such as autonomous driving and intelligent transportation systems. Most existing work has focused on detecting and tracking objects of interest, or on creating segmentations of a scene into semantic labels, and to a lesser extent, on scene understanding and modeling in controllable indoor environments and standardized highways. However, actual urban traffic scenes are cluttered and dynamic with various objects, which may be in varying states and interact with each other, and in situations such as variable illumination in which the objects have different appearances. Moreover, in urban scenes, objects such as pedestrians and cars may move with large difference of scale. .The objectives of this project focus on two basic problems: understanding scale-variant scenes with comprehensive data analysis (not only data fusion); and understanding and modeling objects' activities and their interactions using visual data captured in dynamic urban traffic scenes. Based on recent progress in computer vision, mechanisms of human cognition, and machine learning, we will work on how to estimate a 3D layout of urban scenes for a movable platform, and on how to model a hierarchical distribution of objects, their activities and their interactions in such scenes..The main components of our project are: (1) Tracking progress in principles of human visual recognition and machine learning, and of methods for extracting features and their descriptors of objects, their activities and their interactions. (2) Developing methods for estimating a 3D layout of urban scenes, building a 3D video space into which the videos surrounding a platform can be adaptively registered, describing spatial distributions of objects, their activities and their interactions, and interpreting semantic contexts of the signs and texts in the scenes. (3) Carrying out experiments for scene understanding, both on simulated urban systems and on a movable platform, to verify the theory and methods proposed in the project..We have been working on data fusion, object detection and recognition in previous projects, and have equipped a movable platform with sensors for data capture, which led to our winning the first "IV Future challenge 2009" competition held by NSFC. The members in our research group are accomplished in visual computing, machine learning and system simulation. Note that Professors Yoichi Sato at The University of Tokyo and Yoshitsugu Manabe at Chiba University are to join us, and that Shiying Li is currently working for one year in Professor Sato's laboratory on the principles and methods related to this project. The objectives and proposal are reasonable to ensure that this project will success in boosting our scientific creativity and holding patents for autonomous vehicles.

城市交通场景理解和建模是智能信息处理和计算机视觉领域的热门研究课题。现有研究主要关注城市场景理解的部分内容，例如运动物体的检测跟踪和场景中的物体分类标注。然而，复杂的城市交通场景中各类物体混杂、动态变化、彼此之间有很强的交互关系，而且场景理解尺度跨度大，具有很多不确定性，需要系统有序的协同分析和内容理解，挑战性很大。本项目围绕城市交通场景数据在时空上的显著差异性和整体理解问题，深入研究城市场景理解和建模的理论和方法。主要内容包括：（1）跟踪人类认知机理、机器学习和计算理论方面的新成果；（2）借助这些理论，分析城市交通场景的整体结构、场景中各类物体的视觉特征、空间布局和行为以及它们之间的交互关系，研究表达和描述这些场景特征的方法和模型；（3）通过仿真平台和实验室已有的车载式移动平台进行验证。期望通过本项目的研究，丰富该领域的理论基础，使我国掌握无人驾驶车辆核心技术上的自主知识产权。

城市交通场景理解和建模是智能信息处理和计算机视觉领域的热门研究课题。本项目围绕城市交通场景结构复杂、目标多样、动态变化等问题，研究城市交通场景理解和建模方法，取得了以下几个方面的研究成果。（1）提出基于消失点和道路主方向估计的道路分割算法，提出应对颜色失真、形状失真和尺度变化的交通标志检测和识别算法，理解城市交通场景中的主要结构要素。（2）提出适用于室外场景的多角度视频拼接算法，构建三维动态城市交通场景。（3）改进基于部件结构的刚性和非刚性目标检测算法，提出联合非刚性目标检测和姿态估计的层次化模型，提高复杂城市交通场景中目标检测精度。（4）针对城市交通场景背景嘈杂的特点，提出运动目标检测算法，能够同时检测运动目标及其阴影区域。（5）分析城市交通场景中运动目标与其阴影之间、以及与其他目标之间的相互关系，提出多特征融合的目标分离算法，分离运动目标及其阴影，以及处于阴影附近或沉浸于阴影中的其他运动目标。项目组发表相关论文9篇，其中EI检索7篇，组织和参加20多次国际国内学术活动，培养研究生5名、在读5名。通过该项目的研究，发展了城市交通场景理解和建模的理论和方法，能够提供关键技术方面的支持。