基于多模态信息深度语义融合的个性化音乐推荐模型研究

As an effective means to solve the problem of music information overload, music recommendation technology has become a hot topic in the academic circles at home and abroad. However, most of the content-based music recommendation models still have the following problems: the semantic gap between the low-level acoustic features and the high-level music understanding of human beings, ignoring the important influence of music background on music perception and cognition, only paying attention to the recommendation accuracy but ignoring the user’s personal preferences and subject experience, as well as not generating the overall preference model for the user and then establishing a complete correlation between it and the songs in the collection, which seriously hinders the practical process of music recommendation technology. Based on the multimodal nature of human perception and cognitive mechanism and the subject-dependent properties of music recommendation, this project will study the deep semantic representation methods of multimodal information (music content, artist characteristics, and user tags) that affects the music perception and cognition. Then, the overall user preference model will be constructed on this basis. Finally, efficient personalized music recommendation will be realized by the effective fusion of the multimodal semantic similarities between the user preference and the music in the music library. The implementation of this project will not only provide new solution for the outstanding problems existing in personalized music recommendation systems and thus promote the practical process of music recommendation technology, but also broaden the research field of music recommendation and then further improve the theoretical framework of music information retrieval.

音乐推荐技术作为解决音乐信息过载问题的有效手段成为国内外学术界的研究热点，然而大多基于内容过滤的音乐推荐模型仍存在以下问题：低级声学特征与人类高层次音乐理解之间的“语义鸿沟”、忽略音乐背景对音乐感知和认知的重要影响、只注重推荐的准确性而忽视用户个人偏好和主观体验、以及不能对用户偏好进行整体建模并建立其与曲库歌曲的完整相关性关系，从而严重阻碍音乐推荐技术的实用化进程。本项目从人的感知和认知机理的多模态本质和音乐推荐的主体依赖特性出发，采用深度学习理论研究面向音乐推荐的多模态信息（音乐内容、艺术家特性、用户标签）的语义表示方法，并在此基础上构建用户偏好整体模型，通过对用户偏好和曲库音乐之间的多模态语义相似度进行有机融合实现高效的个性化推荐。项目的实施不仅为解决个性化音乐推荐的突出问题提供了新方法，推动了音乐推荐技术的实用化进程，而且拓宽了音乐推荐的研究领域，进一步完善了音乐信息检索的理论框架。

如何从海量数字音乐作品中寻找符合特定用户个性化要求的音乐作品是国内外学术界共同关注的热点问题。传统的基于内容过滤的音乐推荐模型难以达到预期效果的主要原因是：低级声学特征与人类高层次音乐理解之间存在语义鸿沟、音乐背景对音乐感知和认知的重要影响被忽略、只注重推荐的准确性而忽视用户个人偏好和主观体验、不能对用户偏好进行整体建模，从而严重阻碍音乐推荐技术的实用化进程。针对这些问题项目组：1）研究并提出基于多分辨率卷积神经网络和长短时记忆网络的高级音乐语义特征提取模型、基于CNN和宽度学习的音乐特征提取模型、以及音乐代表性特征序列抽取模型；2）研究并提出可用于艺术家专辑封面检索的多边形图像形状表示方法和基于艺术家网页信息深度语义挖掘的艺术家特性表示模型；3）研究并提出了基于上下文信息的相似度图融合模型和基于深度多模态相似度融合的音乐专辑流派分类模型；4）研究并提出基于交叉递归图融合和多核整合的音乐相似度模型、基于张量积图融合的音乐相似度模型、以及混合融合模型。本项目的实施在一定程度上为解决个性化音乐推荐的突出问题提供了新思路，同时有益于拓宽音乐推荐的研究领域，并完善音乐信息检索的理论框架。