基于内镜视频图像的胃内壁动态表面建模研究

A novel dynamic 3D surface reconstruction method for human soft cavity organs will be researched in this project, which is only based on video images captured by endoscopy. Compared with the conventional reconstruction method of static 3D surface model, which is usually generated from medical tomographic images, the method adopted by this project generates dynamic 3D surface model with the endoscopic video images only, which makes the method has the potential of realizing real-time dynamic 3D surface model generating during endoscopy procedure. This project takes gastroscopy as a study object, and attempts to reconstruct the dynamic 3D surface model of stomach wall based on gastroscopic optical images. The research scheme can be described as following: 1) research on establishing space coordinate system for gastroscopy procedure based on medical space sensor, and develop novel traceable gastroscopy lens which can detect the real-time position information with 6 degrees of freedom. The problem of camera calibration and correction of distortion image resulting from the fisheye lens of gastroscope will be resolved in this project; 2. research on dense feature point detection method in weakly featured gastroscopic optical image, and finding feature points correspondence among gastroscopic video images according to the traceable gastroscopic lens; 3) by introducing the computer vision technique, Structure From Motion (SFM), a method of generating partial 3D surface patch will be provided, which is based on overlapped gastroscopic images captured by the single traceable gastroscopic lens from different viewpoint; 4) research on non-complete gastric surface fusion method to stitch those surface patches into a complete 3D surface, and using video texture mapping technology to realize the dynamic 3D surface model of stomach wall with the same visual effect of gastroscopy; 5) finally, an image guided intervention system with enhanced reality for gastroscopy will be developed to verify the 3D modeling method presented in this project. The research results of this project can also be widely applied to dynamic modeling and image guided intervention for other soft cavity organs, such as bronchoscopy, colonoscopy, and so on.

本项目主要研究完全基于内镜视频图像重建人体软性腔体器官的三维动态表面的技术方法，并以胃镜为切入点，尝试构建基于胃镜光学图像的胃内壁动态表面模型。与常规通过断层扫描成像重建瞬时静态模型的方法不同，本项目拟采用的方法仅依赖内镜检查过程中的光学图像实现动态模型的构建。本项目拟采用的技术方案具体如下：1）研究建立基于内镜检查的空间定标体系，研制具有实时定位功能的胃镜镜头；2）研究弱特征下胃镜视频图像的密集特征点提取和序列图像间的配准方法；3）利用运动恢复结构方法，研究实现基于单目可追踪镜头下的胃镜视频图像重建胃内壁的几何结构；4）研究实现胃镜检查下非完备表面的融合技术以及视频纹理映射技术，以形成具有真实视觉效果的、完整的胃内壁动态表面；5）研制一种具有增强现实效果的胃镜检查介入引导系统，对本项目的建模方法进行验证。本项目的研究成果可以在结肠等其它人体软性器官的动态建模和介入引导手术得到应用。

本项目主要研究了完全基于内镜视频图像重建胃腔这种人体软性腔体器官的三维动态表面，并通过构建图像引导胃镜检查系统进行了实践验证。具体来讲，主要包括：.1）将电磁空间定位装置的传感器与胃镜设备绑定，通过胃镜镜头的手眼定标算法实现了可进行实时位置追踪的胃镜镜头，并且构建了基于内镜检查的空间定标体系。.2）研究出了适用于腹腔软性器官内窥镜检查的图像配准算法Homograph Patch Feature Transfrom (HPFT)。本项目基于传统的影像纹理的配准算法，结合腹腔软性器官内壁表面平滑的特征，通过迭代的方法，不断地在当前的配准结果中检测满足空间单应性映射关系的匹配面片。实验结果表明，HPFT相对于其他经典配准算法，能够在内窥镜图像上匹配更多特征点对，并且取得更好的配准结果。.3）研究出了内窥镜检查中的病灶跟踪算法。基于HPFT配准算法，构建能够表达内窥镜影像之间空间变换关系的图谱结构，从而将内窥镜检查过程中的病灶跟踪问题，转换为待跟踪病灶所在的影像与图谱中影像节点的配准问题，并采用基于图谱结构的路径规划算法实现了病灶跟踪。.4）利用运动恢复结构方法，实现了基于单目可追踪镜头下的胃镜视频图像重建胃内壁的几何结构。通过内窥镜视频图像构建软性器官的三维表面模型，并与真实的检查场景进行配准，建立可导航的内窥镜检查空间，最终实现了内窥镜复查中的病灶定位和导航。.5）实现了基于单目视频的内窥镜视野扩展算法。该算法通过对单目内窥镜影像进行拼接，为医生提供了更全面的检查视野。在本文的全景建模算法中，构建的模型共有两种：一是局部全景模型，该模型以原始内窥镜检查视野为中心，通过影像拼接来扩大医生术中的检查视野；二是全局全景模型，该模型能够反映术中器官的全局纹理信息。如在胃镜检查中，将从原始胃镜影像序列选取的帧样本集合映射到双立方体模型上，然后再通过模型展开实现胃内壁的全局全景呈现。.6）综合以上的研究算法及成果，构建了图像引导胃镜检查系统，并开展了临床实验评估和验证。目前我们已在浙江大学附属邵逸夫医院进行了六十多例临床实验，实验结果表明，该系统能够在解决表达胃腔实时运动形变特征的问题基础上，实现对基于胃镜的微创手术、无创定标活检技术进行书中引导和导航。