手势动作对言语理解的启动效应以及练习效应的脑机制研究

Investigation of brain mechanisms underpinning the effect of sensorimotor information has on language comprehension is an important approach for the understanding of information processing. In this project, with the use of TMS, fMRI and behavioral experiment, based on the hypothesis that gesture-speech integration is the priming effect of gesture comprehension has on the lexical retrieval of speech, we intend to investigate the top-down effect of the left Inferior frontal gyrus (IFG) has on the posterior temporal gyrus (pMTG); the anterior temporal lobes (ATL) as the hub for the common coded representation of both gesture and speech; the effect of practice has on the connection of IFG, pMTG and ATL in the priming effect of gesture has on speech. It is the first time for this project that pointed out how gesture-speech integrate with each other: feature codes of gesture and speech that are stored in both IFG and pMTG would form a temporal composition with the effect of practice, the composition the formed would store in ATL as a common coded representation of both gesture and speech. The results of this project would enrich the understanding of how sensorimotor information could influence the linguistic process and the cognitive process, in the meanwhile, the results could also shed light on cuing of aphasia.

解析感知运动信息对言语活动的影响的脑机制对理解大脑的信息加工具有重要意义。本项目拟结合TMS、fMRI和心理物理实验以手势的理解对语义提取的启动效应来考察手势与言语整合的脑网络和神经机制。重点研究在启动效应中，左额下回对左后颞回的自上而下的调节作用；左前颞叶作为手势与言语共同编码表征源的存储作用；左额下回、左后颞回和左前颞叶在启动效应中连接建立的后天练习作用。本项目首次大胆提出，手势与言语的整合的本质是存储在左额下回、左后颞回的手势和言语的特征编码之间通过练习形成的信息集合，并作为手势与言语共同的编码表征源存储于左前颞叶。本项目研究成果将加深对感知运动信息对言语理解以及高级认知活动影响的理解，同时对失语症的治疗和干预具有很大启示和应用作用。

人脑中语义信息的存储和表征是多模态的，并且以语义网络的形式广泛分布在大脑中，从而实现了大脑皮层的感知觉与运动系统之间密切而复杂的联系。其中来自手势的信息格外重要。手势信息和言语信息交互作用，共同影响着我们对交流信息的理解。以往的研究发现，跨模态的手势信息和言语信息的加工过程也会涉及语义加工的两个系统：用于语义存储的语义表征系统和语义控制系统,以使语义加工更灵活、动态地符合当前的情景和任务要求，然而这两个系统是如何在手势与言语的跨模态语义加工中动态交互的尚不明确。. 在本项目中，我们结合了行为学方法、fMRI技术和非侵入性精密计时双脉冲经颅磁刺激（double-pulse TMS）技术，将手势和言语信息的整合过程分成以40ms为单位的时间窗，在每个时间窗上分别对多模态语义表征核心脑区施加刺激，以期获得两个系统的因果性时间动态加工机制。研究结果发现了手势-言语的两阶段整合环路。在第一个阶段，手势对言语的调控先发生于pMTG，后出现于IFG，提示存在pMTG向IFG的前馈投射。第二个阶段中，IFG比pMTG更早参与到整合过程，提示存在IFG向pMTG的反馈调节（zhao et al., 2021,JN）。该研究在国际上首次为多模态语义整合的动态脑环路机制提供了因果性证据，揭示了额叶语义控制系统与颞叶语义表征系统的动态交互过程。