基于多模态眼底影像深度学习的白内障术后视力预测研究

Artificial intelligence is a national strategy. The combination of ophthalmology and artificial intelligence is a focus of attention in society and academia. Some top journals, such as Cell, JAMA, et al, had published research papers about ophthalmic artificial intelligence. Senile cataract is a blind disease in ophthalmology, and it only can be cured by surgery. Visual prediction of cataract surgery has always been an urgent problem in the field of ophthalmology. The traditional methods are functional examinations, then the subjective results can be got, and it is difficult to get accurate prediction results. In this project, routine inspections are used to obtain fundus photography and OCT images. Multimodality fundus image deep learning is applied for obtaining accurate prediction of best corrected visual acuity after cataract surgery. Therefore, the local information fusion DCGAN model is proposed to turn the preoperative fundus images and OCT images into clear fundus photography and OCT images, which are supposed to be closed to the real postoperative images. The DCGAN model is used to solve the problem of blurry fundus image sharpness. Then the local fusion features of the macular image in the fundus image are acquired by merging the deep features which are generated by CNN and the shallow features. Based on the fundus photography (mainly the local fusion features of the macular area) and OCT images generated by the model, a multimodal deep learning model can be trained to correlate the images with best corrected visual acuity; it can accurately predict the best corrected visual acuity before surgery, assisting both doctors and patients in surgical decision making.

人工智能是国家战略，眼科与人工智能结合是社会和学术界关注的热点。《Cell》《JAMA》等顶级期刊已发表多篇眼科人工智能研究成果。老年性白内障是眼科常见致盲性眼病，唯有通过手术复明。白内障术后视力预测是眼科领域亟待解决的问题，传统方法通过功能性检查获得主观结果，难以进行精准预测。本项目通过眼科常规检查设备获得患者眼底影像，将多模态眼底影像深度学习应用于白内障术后最佳矫正视力的精准预测。采用局部信息融合DCGAN模型，通过术前模糊眼底影像生成清晰眼底影像，使生成图像无限接近真实术后图像，解决白内障模糊眼底图像清晰化问题；通过CNN提取生成眼底照片黄斑区深层特征，并融合其浅层特征获取黄斑区局部融合特征；通过眼底照片黄斑区局部融合特征和生成OCT图像，训练多模态深度学习模型，将术前眼底影像与术后最佳矫正视力关联，能够在手术前精准预测术后最佳矫正视力，辅助医患双方手术决策。

人工智能技术在国内外学术界和产业界获得了广泛的关注，我国人工智能在国家的大力支持下发展迅速。眼科领域专业医生极度缺乏且结构化数据较多，适合人工智能技术的应用，可以辅助医生快速诊断疾病并对患者及时转诊，减轻医生负担。本项目针对眼底常见病、翼状胬肉、黄斑病变等疾病构建智能辅助诊断分类分级模型，通过数据增强、模型轻量化、可解释性和长尾学习等技术的研究，提高模型灵敏度和特异度，最终多数模型诊断疾病的灵敏度和特异度可达95%左右。智能辅助诊断通常可以获得疾病的类别及程度，但很难获得病灶精确信息，因此本项目针对眼前节图像对翼状胬肉病灶区域进行精准分割，通过跳跃连接融合图像浅层特征，并结合多头注意力机制探寻翼状胬肉边缘区域与正常眼部区域关系，最终模型在翼状胬肉分割任务上的IOU和MIOU分别达到了79.44%和87.43%。同时，本项目利用分割技术获得眼底图像视盘区和虚拟黄斑区，通过二者中心点自动测量视盘-黄斑中心夹角，辅助斜视诊断。眼科人工智能技术发展迅猛，但缺乏相关领域伦理的指导，本项目早期通过调查问卷的方式，评估医疗工作者和其他专业技术人员对眼科人工智能的态度和担忧，发现多数受访者支持眼科人工智能并对未来持乐观态度，但仍认为需要对眼科人工智能伦理给予更多关注，所以本项目主持人参与了眼科人工智能临床应用伦理专家共识的制定，为眼科人工智能的长期发展奠定伦理基础。本项目通过对数据增强、模型轻量化、长尾学习、和注意力机制等技术的研究，优化了智能辅助诊断模型和病灶分割模型，并对模型的可解释性进一步研究，未来将融合符号神经网络深入研究模型可解释性问题，同时结合图像和文本研究病因推理及溯源等相关问题。