融入直觉化交互机理的人机界面全息设计与评价理论研究

The display and control system of cockpit is an important human computer interface between the controller and equipment. With the emergence of intelligent interaction technology, the information transferred between controller and equipment has increased substantially, which necessatres the exploration of the comprehensive human cognitive characteristics. Therefore the design of multimodal human computer interface based on intuitive interaction mechanism becomes a new branch. This study will focus on the design and evaluation of multimodal human computer interface based on intuitive interaction mechanism through the following objectives. Firstly, intuition thinking features and the process of information transfer will be characterized based on neurophysiology and cognitive psychology, and the mapping between the intuitive interaction mechanism and multimodal information are explored. Secondly, based on the theory of Multimodal fusion and Polyehtomatic Sets, different sensory signals from the multimodal information will be extracted and expressed, by now a multimodal intuitive human computer interaction model can be established. Finally, based on the above research, holographic theory of designning and evaluation of multimodal human computer interface is proposed for the complex mechatronic equipment cockpit. This project aims to improve the cognitive accuracy, efficiency and workload of controller when facing complex information, to provide theoretical guidance for the design and evaluation of multimodal human computer interface and to enhance the ability of independent research and development of complex mechatronic equipment cockpit.

舱室的显示与控制系统是操控者和装备间传递信息的重要人机交互界面。随着智能交互技术的出现，通过人机界面传递给操控者的信息量大幅增加，如何全面的探究人体认知特性，设计具有直觉化交互用户体验模式的多通道人机界面将是一个新的发展方向。本申请拟开展面向直觉化交互的多感知通道人机界面设计方法研究，通过神经生理学和认知心理学理论，分析直觉思维特点以及信息加工过程，探索直觉化交互机理与多通道间的响应模式，根据多通道信息融合理论和多色集合理论，对不同属性信号进行提取、表达和整合，构建多通道直觉化人机交互模型。基于以上研究，最终提出一套面向复杂机电装备舱室的多通道人机界面全息设计与评价理论。本项目研究旨在提高操控者面对复杂信息的认知精度和效率，降低认知负荷，为多通道舱室显控界面设计与评价提供理论指导，而且对于提升复杂机电装备舱室的自主研发能力，形成系统配套的高新技术装备体系有着重要意义

随着智能交互技术的出现，通过人机界面传递给操控者的信息量大幅增加，如何全面的探究人体认知特性，设计具有直觉化交互用户体验模式的多通道人机界面是当今研究热点。本项目依据人体直觉化交互机理，研究多维信息如图像、语音、运动等因素与人体多通道（视觉、听觉、触觉）的认知关系，发现了直觉化交互与视听联觉存在内在联系，也与动作行为有着密切关系，因此首先从神经科学的角度研究并得出了基于视听联觉的直觉化交互机理；又通过对多通道交互工效的研究，提出了基于任务的多通道优先级模型以及悬垂手势交互方式，使得交互过程更加符合人体直觉化认知交互习惯和用户体验规律；最后提出了以生理信号、神经信号以及工效评价指标如准确度、反应时、可靠性相结合的全息评价方法，对不同任务界面如功能型、行为型、情感型界面进行实验验证，对人的注意感知、决策行为、情感激发进行评价，总结出直觉化交互方式对于不同任务界面的注意导向、行为诱导、情绪干预等影响效果显著。项目研究成果已在公共服务展示系统、数字导航系统、无人机地面控制站中得到应用。随着研究的深入，项目成果还可推广到更多特殊环境下的显控系统，如航天、航海（潜艇）、地质勘探、油气矿开采等，一方面通过直觉化交互方式增强用户体验，提高使用者认知效率，增强操控安全性；另一方面可以对特殊环境下人们负面心理进行干预，如视听联觉交互可以有效干预人们在幽闭空间情绪问题，减轻精神压力和工作负荷。因此，本项目提出的面向人机界面的全息评价理论，对复杂装备舱室的显控系统设计与评价有着广阔的应用前景。