基于深度学习的场景文本检测和识别算法研究及应用

The goal of this project is to propose novel approaches for scene text detection and recognition in unconstrained natural images, by leveraging the high capability of deep learning models. We analyze main limitations of existing deep models for text-features extraction, and propose new deep models for robustly learning and recognizing the true text-features, in an effort to bridge the performance gap between recognitions of scene texts and traditional printed/scanned documents. On the one hand, motivated by the basic functions of human vision system, we develop a Text-attentional Convolutional Neural Networks (Text-CNN) to accurately extract character features from natural images(patches). We build a multi-tasks learning model to effectively incorporate different levels of highly-supervised text information (including low-level shapes, character labels and text/non-text ) into the learning process subsequently. This enables the Text-CNN with stronger capability for locating the true text-features, and hence reduces the influences of non-text objects or background information substantially. On the other hand, for recognition, we design a novel Word-attentional Convolutional Neural Networks (Word-CNN) to compute a sequential high-level features from a whole word patch, by exploiting a similar mulit-tasks learning pipeline. Then we build the final word recognition system by leveraging the strong 'memory' and discriminative abilities of the recurrent LSTM model for long sequential word-features recognition. Besides, we also develop a novel Dual Text-Salience Detection (DTSD) approach for improving the Recall of the MSERs method for text component detection.

本项目以深度学习为主要工具, 研究和开发非限制条件下自然场景文本检测和识别算法。通过深入分析现有深度模型在提取场景文本特征方面的局限性，我们提出新的深度模型来更有效地提取和识别文本/字符特征，以缩小场景文本与传统打印/扫描文档识别之间的性能差距。首先，我们从人类视觉基理出发，设计一个专注文本特征的卷积神经网络（Text-CNN）来准确提取场景字符特征。通过构建多任务学习模型， 我们把不同层次的文本监督信息（文本轮廓， 类别和是否文本）逐层强化到模型训练过程中，使Text-CNN能够更准确地定位场景图片中的字符信息，从而降低其他物体和背景的干扰。接着，在识别方面，我们采用类似的多任务学习机制，设计一个基于词条的特征提取模型（Word-CNN）。最后利用循环LSTM 对连续特征长时间记忆和辨别能力，构建新的词条识别系统。另外，我们还设计了一个双层文本显著性检测算法，以提高文本成分检测的召回率。

本项目以深度学习为主要工具, 研究和开发非限制条件下自然场景文本检测和识别，以及场景分类算法。通过深入分析现有深度模型在提取场景文本特征方面的局限性，我们提出新的深度模型来更有效地提取和识别文本/字符特征，设计一个专注文本特征的卷积神经网络 - Text-CNN 来准确提取场景字符特征。同时，基于最新的通用物体检测框架，提出CTPN和SSTD两个高性能的文本检测算法。在识别方面，我们提出DTRN文本识别算法。在DTRN算法中，我们首次提出CNN+RNN+CTC的场景文本识别框架，大幅提升识别的准确率。整个项目的工作形成一套完整，准确，高效，可工业应用的场景文字检测和识别的技术解决方案。其中，成果中的CTPN检测和CNN+LSTM+CTC识别的解决方案成为当前图像OCR工业界的标准解决方案。项目的相关技术直接应用于华为技术和商汤科技相关产品中，并直接和间接产生营收和利润。