多模态、多属性甲状腺超声影像大数据自主标注及学习

Ultrasound examination is preferred mean of imaging methods for thyroid nodules. Ultrasound diagnosis is often questioned by clinicians mainly due to lack of quantitative and stable analysis techniques. Two-dimensional ultrasound, contrast-enhanced ultrasound and elastography are the most effective three kinds of data in ultrasound diagnosis. In this project, we are going to study, under multi-modal and multi-attribute environments, the problem of ultrasound image automatic annotation and learning for thyroid nodule diagnose. Furthermore, the theory and method of quantitative assessment of benign and malignant thyroid nodules are going to be built. The main contents are as follows: 1) Study the small data learning model based on the knowledge transfer learning, achieve multi-attribute automatic labeling. So we can construct big data of multi-modal thyroid ultrasound; 2) Study the multi-modal ultrasound data feature fusion computing model to solve the problem of multiple sub-feature definition and learning for pathological attributes; 3) Exploring the enhancement and visual representation model of ultrasonic attribute features to realize the visualization of learning model, sub-feature and dynamic feature; 4) Study the problems of joint modeling, trend prediction, feature validity evaluation and heterogeneous data analysis across time stages, and finally construct an efficient thyroid nodule malignant risk assessment system. The key issues of the project include heterogeneous big data driven combined with expert knowledge, cross time feature modeling and association analysis for specific diseases, definition and description of sub-features corresponding to pathological properties.

超声检查是甲状腺结节的首选影像学手段，但超声医生诊断通常受到临床医生的质疑，主要源于超声数据欠缺定量、稳定的分析手段。二维超声、造影和弹性图是超声诊断中最有效的三种不同模态数据，本项目研究多模态甲状腺超声影像大数据多属性自主标注及学习问题，进而建立甲状腺结节的良恶性风险定量评估理论和方法。内容包括：1）提出基于知识迁移的小数据学习模型、实现多属性自动标注，构建多模态甲状腺超声大数据；2）研究多模态超声数据特征融合计算模型，解决对应病理属性的语义子特征定义及学习问题；3）探讨超声属性特征的增强及可视化表示模型，达到学习模型可视化、子特征可视化和动态可视化；4）实现跨时间阶段超声数据联合建模、趋势预测、特征有效性评价、异质数据综合分析，构建高效的甲状腺结节恶性风险评估体系。本课题关键问题包括：融合专家知识的异构大数据驱动学习、特定病种跨时间特征建模和关联分析、病理属性子特征定义及描述。

以乳腺及甲状腺多模态超声影像大数据学习为主题，针对申报书提出的4大研究内容，本团队从8个方向开展了系统的研究：1) 系统调研，形成主流方向及算法综述报告和专著成果；2) 噪声条件下大规模影像数据集构建；3) 图像分割、病灶自动标注及定位；4) 少样本学习、跨域问题研究；5) 网络模型设计及特征工程；6) 多模态超声数据融合；7) 困难样本挖掘、噪声数据处理；8) 时空特征挖掘、信息融合。团队在项目执行期间，发表了包括AAAI、ECCV、IEEE Transactions on Multimedia (TMM)在内的学术论文34篇，SCI期刊论文13篇，其中科院二区以上期刊论文6 篇；包括软件学报、自动化学报、电子学报在内的中文高水平论文14篇。授权发明专利10项，受理3项，出版学术专著1部。项目组在B站上开设了人工智能科普网站，对算法进行了讲解，相关算法源码已经在Gitee上开源，以超声为主题作大会报告5次。以项目成果开发的超声影像分析软件在合作医院进行了试用，并制定了后续合作研究计划。总体上，项目达到原定目标。