多源融合视角研究孤独症儿童非语言社会性注意与行为反应

The early diagnosis and intervention of autism are critical. Research on the characteristics of social communication impairment in children with autism will meet the major needs of early screening and assessment. Recognition of nonverbal social attention and behavioral responses in children with autism is the basic means of revealing the characteristics of social communication impairments. The current researches based on single model data such as scales, eye-gaze, physiology, EEG and so on are not sufficient to form the comprehensive objective quantitative model. However, the multi-model fusion method faces the difficulties of large differences in representation characteristics of multi-model data and data heterogeneity. The effect needs to be improved. In response to the above issues, this research will establish a recognition model of social communication impairment that includes three dimensions of attention, emotion and activity. The multi-model data on autistic children's eye-gaze, head pose, facial expressions, body movements, interactions, and physiological reactions will be collected and analyzed. Time-synchronized feature extraction and feature correlation are used to reduce the diversity of multi-model data representation characteristics. And the hierarchical feature fusion algorithm will be used to reduce the difficulty of heterogeneous data fusion. Then the weighted clustering of mixed data attributes will finally realize the effective recognition of social communication impairment characteristics in children with autism. The result of this research will provide innovative theories and methods for early screening and assessment of children with autism, and promote the effective assessment of autistic children, as well as the development of personalized intervention training.

孤独症的早期诊断和干预至关重要。研究孤独症儿童的社交沟通障碍特征是孤独症早期筛查评估的重大需求。对孤独症儿童非语言社会性注意和行为反应的识别是揭示其社交沟通障碍特征的基本手段。当前基于量表和眼动、生理、脑电等单一模态数据的研究不足以形成综合的客观量化模型，而多源融合的方法面临多源数据表征特性差异大和数据异构的困难，效果有待提高。针对上述问题，本项目将建立包含注意力、情感、活跃度三个维度的社交沟通障碍识别模型，围绕孤独症儿童的眼睛注视、头部姿态、面部表情、肢体动作、交互行为、生理反应等方面采集多源数据，以时序同步动态特征提取和特征关联作为减小多源数据表征特性差异的途径，以层级式特征融合算法降低异构数据融合难度，通过混合数据属性加权聚类实现对孤独症儿童社交沟通障碍特征的有效识别。本项目的研究将为孤独症儿童早期筛查评估提供创新理论与方法，促进孤独症儿童的有效评估以及个性化干预训练的开展。

社交沟通障碍是孤独症的核心症状之一。研究孤独症儿童的社交沟通障碍特征是孤独症早期筛查评估的重大需求。对孤独症儿童非语言社会性注意和行为反应的识别是揭示其社交沟通障碍特征的基本手段。本项目以多源融合视角研究孤独症儿童非语言社会性注意与行为反应，建立包含注意力、情感、活跃度三个维度的社交沟通障碍识别模型，围绕视觉注意、面部表情、生理反应、交互行为等方面进行数据采集与特征提取，多源融合实现孤独症儿童障碍特征的有效识别，促进其在孤独症儿童评估与教育干预领域的转化及应用。.项目的主要研究成果包括：1）提出一种基于强度标签分布的面部表情分析方法，实现表情识别和表情强度的联合估计，其准确率指标达到较高水平。2）深入研究孤独症儿童对面部表情、社会性信息的注意特性，深入分析孤独症儿童共情能力缺陷，揭示内在加工机制，为社交沟通障碍评估提供有效的指标依据。3）深入研究视觉注意、面部表情、生理反应、认知等多种模态数据，探讨不同模态数据识别效果与信息互补性，提出基于多源数据融合的孤独症儿童识别方法，对数据进行分层次混合融合，有效提升了孤独症儿童识别效果。研究成果已应用到孤独症儿童评估领域，服务于广大孤独症儿童的早期筛查评估，取得了良好的效果。.经过3年研究，项目组圆满完成了预期任务。共发表学术论文8篇，其中SCI收录2篇（含一区论文1篇），EI收录5篇；申请发明专利3项，其中1项为国际专利并已获授权；获批软件著作权4项；获得湖北省级奖励3项，其中科技进步奖一等奖1项、社会科学优秀成果奖二等奖2项；培养硕士研究生5人，协助培养博士研究生2人。