面向手功能康复的高保真视触觉增强现实方法研究

In rehabilitation medicine, multi-sensory superimposition is conducive to the recovery and reorganisation of neurological functions. The application of augmented reality (AR) technology in hand function rehabilitation after stroke is beneficial for patients who conduct task-oriented training in scenarios that naturally integrate virtual environments with reality. In this project, high-fidelity visuo-haptic AR methods for hand function rehabilitation are examined on the basis of interdisciplinary research, including information science, medicine and cognition science and other disciplines. Visual-haptic feature models for different rehabilitation stages are analysed by focusing on “feeling re-learning” process in hand function rehabilitation. The tracking and registration algorithms of the hand features of patients and the methods of high-fidelity visual-haptic fusion interaction are employed by emphasising “realistic experiences” in an AR environment. A visuo-haptic AR prototype system that can be applied to hand function rehabilitation is also established on the basis of activities of daily living. The applicability and practicability of the designed mechanism and system are demonstrated and assessed through experiments. The implementation of this project provides a new concept for research on the recovery mechanisms of sensation and perception. New human–computer interaction techniques for rehabilitation therapy are also presented. This study promotes the development of intelligent medical treatments.

康复医学研究已证实多种感觉叠加有助于神经功能的恢复和重组。将增强现实技术应用于脑卒中后手功能康复，有利于患者以自然的方式在虚实融合场景中进行任务导向训练。本项目基于信息、医疗、认知科学等多学科交叉，面向手功能康复，开展高保真视触觉增强现实的方法研究。围绕手功能康复中的“感觉再学习”过程，研究患者在不同康复阶段的视触觉特征模型。围绕增强现实环境的“真实感体验”，研究适应患者手部特征的跟踪注册算法以及高保真视触觉融合交互方法。参照基于日常生活活动的训练场景，搭建用于手功能康复的视触觉增强现实原型系统。通过实验，论证并评价所设计机制和系统的适用性和实用性。本项目的实施将为感觉恢复的机理研究提供新思路，为康复医疗人机交互技术提供新方法，促进智慧医疗的发展。

康复医学研究已证实多种感觉叠加有助于神经功能的恢复和重组。将增强现实技术应用于脑卒中后手功能康复，有利于患者以自然的方式在虚实融合场景中进行任务导向训练。本项目研究了视触觉特征模型，适应患者手部特征的跟踪注册算法，高保真视触觉融合交互方法等内容。参照基于日常生活活动的训练场景，搭建了用于手功能康复的视触觉增强现实原型系统。通过实验，论证并评价了所设计机制和系统的适用性和实用性。项目研究取得了多项成果，已在国内外期刊发表论文27篇，其中SCI论文16篇；发表会议论文1篇；授权专利11项，其中发明专利5项，登记软件著作权10件。已完成项目计划中规定的研究任务。在本项目资助期间，本人作为骨干成员入选全国高校黄大年式教师团队，获批江苏省首席科技传播专家，指导学生获得全国大学生创业大赛第十一届“挑战杯”大学生创业计划竞赛铜奖、第十六届中国研究生电子设计竞赛商业计划书专项赛江苏省赛团队一等奖等奖项。本项目研究成果具有良好的应用前景，将为感觉恢复的机理研究提供新思路，为康复医疗人机交互技术提供新方法。