基于位置的多视角实时渲染和异形大屏幕投影显示关键技术研究

In recent years, 3D realistic scene with crowd interaction on large screens of different shapes has been widely applied to the fields of military training, sport training, product exhibit, education and entertainment business. Most of current display systems are based on rendering and displaying the scene at a fixed view position, which only results in a sense of limited reality. In this project, our research faces 3D realistic scene with crowd interaction, particularly focusing on solving its core issues such as real time photo-realistic rendering and projected camera calibration for large screens of different shapes, trajectory tracing and crowd clustering. The goal here is to enable users receive different virtual experiences according to different view positions or different interaction contents, which in turn will raise the sense of reality to a higher level. In addition, we will provide effective system solutions and software tools for virtual crowd shooting based on 3D realistic scene with crowd interaction, which will promote the research of human-computer interaction and virtual reality in research depth and application width.

基于异形大屏幕(半球、圆柱、弧形等)投影的立体真实感群体互动场景在军事训练、体育训练、产品展示、教育、娱乐等领域的应用越来越广泛。目前的显示系统多是基于固定视点渲染和显示场景画面，缺乏一定的真实感。 为此，本项目重点研究基于位置的多视角实时渲染与异形大屏幕投影的实时变形校正，群体用户位置跟踪与聚类等关键技术，使不同位置的用户实时观看到基于其所在位置和交互内容的不同画面，使得对于现实世界的模拟更真实；并研发群体虚拟射击系统进行技术集成和验证，为基于异形大屏幕投影的立体真实感群体互动场景的构建提供有效系统方案、关键技术和工具软件，推动人机交互与虚拟现实技术的深入研究和广泛应用。

课题组重点研究了基于位置的多视角实时渲染与投影变形校正、群体用户位置的实时跟踪等问题，开展了基础理论研究和科学应用研究，提出了多画面立体投影显示方法、异形屏幕的实时投影校正方法、异形屏幕多投影拼接融合与实时视频播放方法、实时渲染技术和群体位置跟踪方法。并对虚拟现实环境中的用户体验机制进行了研究，提出了基于生理信号的互动系统最佳用户体验评估方法，构建了一种用以解释虚拟现实环境中用户心流指标与实际活动表现之间的弱关系产生机制的模型。基于上述研究技术，研发了群体虚拟射击系统，进行系统验证。课题共发表学术论文21篇，其中SCI收录9篇，EI收录9篇，CCF-A论文6篇。出版《人机交互基本教程（第三版）》1 部。申请国家发明专利18项，美国专利2项，其中授权国内发明专利15项、授权美国发明专利1项；获得软件著作权11项。培养博士后2人，博士研究生4人，硕士研究生14人。参加国际国内会议10次。课题组按计划完成了课题计划任务。