内镜引导颅底手术中多模态影像虚实融合问题的研究

The location of the skull base tumor is usually deeply inside the brain and close to vital tissues, such as the brainstem, major arteries and cranial nerves. Therefore, the clinical surgery of removing these kinds of tumors is particularly challenging. Endoscopic technique is widely used for the diagnosis and treatment of the skull base tumor. While multi-modality image fusion guidance is the critical technique to minimize unnecessary trauma in invasive and precision surgeries. As the endoscopic image is with narrow vision and without three-dimensional structural information, current endoscopic guided surgery is unable to fully meet the needs of the clinical operation. Therefore, the significant scientific issues to be urgently solved for endoscopy technology is fusion enhanced display of multi-modality images by breaking through the limitation of endoscopic guided surgery. This project aims to study the key theories and technologies of the endoscopic image enhancement and distortion correction; the reconstruction of the nasal cavity structure with endoscopic images and pose estimation of the endoscope; the registration of the nasal cavity structure; the visual field expansion for the endoscopic enhanced display with mixture reality fusion. The theoretical framework of the endoscope-CT/MR image fusion is constructed to provide an accurate surgical solution for the skull base tumor surgery. Finally, the correctness and validity of the proposed theories and technologies are evaluated by constructing a series of experimental systems. The proposed multi-modality image fusion theory will effectively promote the development of the mixed reality, and further promote the development of the theory of virtual fusion in the clinical application of endoscopic guided surgery.

颅底肿瘤往往位置较深，毗邻脑干、颅神经及重要血管，临床手术操作难度较大。内镜技术是颅底肿瘤诊疗的主要手段，而多模态影像融合引导是实现微创、精准诊疗的核心。现有内镜引导手术技术存在内镜图像术野狭小、无直观三维结构信息的缺陷，尚无法完全满足临床手术需求。如何在理论模型与关键技术上突破内镜引导手术的局限，实现多模态影像的融合增强显示是该领域亟需解决的重要科学问题。本项目拟对内镜引导颅底手术进行延拓和创新，研究内镜图像的增强和畸变校正、内镜图像结构重建和位姿估计、鼻腔结构配准、内镜图像视野扩张和虚实融合增强显示等关键问题，构建内镜-CT/MR影像虚实融合理论框架，提供一套基于混合现实的颅底肿瘤精准手术解决方案，并将在理论研究成果的基础上搭建实验系统验证所提理论和方法的正确性和有效性。提出的多模态影像融合计算理论将有力推动混合现实理论的发展，进而促进虚实融合理论在内镜引导手术临床应用的深入发展。

内镜引导手术是微创诊疗最为重要的发展方向之一。然而，在诊疗过程中，由于内镜图像术野狭小，无法获得皮下内部器官组织的整体结构信息，医生无法直接通过内镜图像判断病灶与周围重要器官组织的位置关系。由此，研究内镜引导颅底手术中多模态影像虚实融合，将术前多模态影像与内镜图像相结合获取腔体组织整体结构信息，实现内镜图像的术野虚拟扩张与融合增强显示，为医生提供直观立体的颅底组织结构信息，并为突破现有内镜引导颅底手术的局限提供新的思路。本项目重点突破了以下关键技术难题：.（1）内窥镜图像质量增强；（2）内窥镜图像特征匹配；（3）单帧内镜图像深度估计；（4）内窥镜视频图像手术阶段识别；（5）内窥镜图像运动与相机位姿估计；（6）视神经分割；（7）表面和体数据混合渲染深度增强；（8）术中快速无标配准；（9）内窥镜旋转实时跟踪；（10）内窥镜引导术中导航关键部位预警；（11）内窥镜手术导航自动定位人脸标识点。.本项目在国际知名期刊上发表SCI论文37篇，EI检索论文10篇。相关关键技术成果申报国家发明专利12项，其中授权专利1项。获软件著作权4项。成果转化获国家医疗器械注册证1项。作为主要完成人获中国图象图形学学会科技一等奖和中华医学科技二等奖。