融合语义信息与社会化信息的图像美感增强模型研究

With the rapid development of photographic device and social network, it has become very popular that people record their daily life by taking and sharing pictures. Furthermore, people tend to generate enhanced effects in terms of visual aesthetics for their photos with more diversified or personalized styles. However, traditional enhancing methods mainly focus on modeling low-level image appearance, while they usually neglect their high-level semantic and socialized information. In this research proposal, we aim at building semantic- and social-aware models for enhancing image aesthetics. We plan to carry our research in the following aspects: (1) We plan to extract the socialized information, and build representing models for users and groups at micro- and meso-scopic level. (2) We plan to analyze the correlation between the visual semantic information and socialized information of images, and unify them into a common representation model. (3) We plan to build an image aesthetic assessment model based on the multi-task deep convolutional network. (4) With the guidance of the obtained semantic and social information, we plan to develop several aesthetics-enhancing models for tasks such as detail enhancement, color enhancement and image gallery enhancement. Our research is expected to provide theoretic and technical support to the field of user-centric image enhancement.

随着拍照设备和社交网络的迅速发展与普及，用图像记录、分享日常生活已成为一种普遍方式。人们进而希望能对所拍摄图像进行灵活多变、富有个性的增强，以提高其视觉美感效果。传统增强模型侧重于对图像底层表观建模，往往忽略了图像语义信息和社会化信息。本课题旨在构建与研究具有语义感知和社交感知的图像美感增强模型，将围绕以下核心内容展开：(1) 抽取互联网图像的社会化信息，从微观和介观层面对用户与群组进行表示建模；(2) 分析社会化信息与图像视觉语义信息之间的关联，形成协同视觉语义与社会化信息的图像表示模型；(3) 构建基于多任务深度卷积网络的图像美学评价模型；(4) 在此基础上，建立具有语义和社会化感知的引导模型，用于细节增强、色彩增强、群图展板增强等具体模型之中。本课题将推动图像增强的理论与应用，为以用户为中心的图像智能显示提供核心算法与技术。

本项目利用语义信息和社会化信息进行图像视觉美感增强。研究内容包括社会化信息和语义信息如何抽取及其在图像增强建模中的应用。项目研究进展包括以下几个方面：首先，提出一系列高效的暗光图像增强方法，解决了当时方法无法有效感知图像中的光照分布和场景结构分布、对暗光图像中的噪声等干扰因素无法有效抑制等问题，有效地提高了图片的视觉感观效果。其次，项目提出了一系列语义信息高效提取方法，并构建了语义感知和用户驱动的个性化图像增强算法。项目按照预期计划顺利执行，顺利完成了各项研究任务。项目研究成果在IEEE TMM、ACM TOIT、ACM KDD国内外期刊和会议上发表/录用期刊论文17篇，会议论文3篇，申请专利5项、软著2项。项目部分研究成果得到了国内外知名学者的正面评价，并在资源受限的移动端平台实现，形成了良好的应用价值。