基于脑血管疾病颅脑影像的医学报告自动生成及病灶定位研究

Automatic generation of medical report is an indispensable part of intelligent medical treatment in the future, which is also a new hot.topic in the cross-domain of artificial intelligence and medical application. This project intends to study the automatic generation of medical reports and the localization of lesions on the craniocerebral images of cerebrovascular diseases, so as to provide reliable and effective technical support for the auxiliary diagnosis. There are lots of layers which are internally related in the 3D space, with the sparsity and uncertainty of the lesions. The text structure of the report is complex, which leads to great challenges for the feature representation, lesion localization, and language generation. Therefore, this project focuses on two key scientific issues, namely, the structured visual feature representation of 3D images and the semantic spatial consistency matching of the heterogeneous data, to perform the following researches: Considering the sparsity and uncertainty of the lesions, this project proposes a structured feature representation method based on multi-scale superpixel and multi-layer attention; According to the layer dependence of 3D space, the project introduces a novel memory attention model to guide the medical report generation. With the unlabeled lesion data, weak supervised lesion localization is performed based on the reverse matching of the attention weights. Lesion localization and report generation are combined with the consistent matching of text-visual heterogeneous data. The project will promote the clinical application of medical reports generation technology in the 3D medical image, thus has a very important theoretical significance and application value.

医学报告自动生成是未来智能医疗不可或缺的一部分，也是人工智能结合医疗应用的新兴热点问题。本课题拟针对脑血管疾病颅脑影像进行医学报告自动生成和病灶定位研究，为辅助医生诊断提供可靠有效的技术支撑。颅脑三维影像层数多、层关系密切；病灶分布稀疏、不确定性大；报告文本结构复杂。这些特点给特征表示、定位识别和语言学习带来了极大的挑战。因此，本项目围绕三维影像结构化视觉特征表示和异构数据语义空间一致性匹配这两个关键科学问题开展如下研究：针对三维空间病灶的稀疏和不确定性，研究基于多尺度超像素和多层注意力的结构化特征表示方法；根据三维空间的层依赖关系，提出基于记忆注意力的医学报告自动生成模型；在无标记病灶的样本空间下，基于注意力权重反向匹配实现弱监督病灶定位，病灶定位和报告生成通过文本-视觉异构数据一致性匹配进行联合学习。项目将推动医学报告生成技术在三维影像中的临床应用，具有重要的理论意义和应用价值。

医学报告自动生成是未来智能医疗不可或缺的一部分，也是人工智能结合医疗应用的新兴热点问题。本课题拟针对脑血管疾病颅脑影像进行医学报告自动生成和病灶定位研究，为辅助医生诊断提供可靠有效的技术支撑。颅脑三维影像层数多、层关系密切；病灶分布稀疏、不确定性大；报告文本结构复杂。这些特点给特征表示、定位识别和语言学习带来了极大的挑战。因此，本项目围绕三维影像结构化视觉特征表示和异构数据语义空间一致性匹配这两个关键科学问题开展如下研究：针对三维空间病灶的稀疏和不确定性，提出基于多尺度超像素的结构化特征表示方法；根据三维空间的层依赖关系，提出基于记忆注意力的医学报告自动生成模型；在无标记病灶的样本空间下，基于注意力权重反向匹配实现弱监督病灶定位，病灶定位和报告生成通过文本-视觉异构数据一致性匹配进行联合学习。项目将推动医学报告生成技术在三维影像中的临床应用，具有重要的理论意义和应用价值。