非可微深度学习理论研究

Nowadays, "deep learning" is always referred to be the "deep neural network" based on parameterized and differential components, which achieves great success in many real applications, followed by the challenges of huge computational cost, manpower, verifiability, etc. Recent years have witnessed the increasing interests on deep learning based on non-differential components, such as deep forest and deep boosting, whereas there is a lack of theoretical support. This project tries to present a theoretical analysis on the generalization, stability, consistency and sample complexity for deep learning, and the main studies include: presenting the margin-distribution generalization bounds for deep learning; investigating the stability of deep learning, and presenting generalization bounds based on stability; presenting finite-sample and distribution-dependent convergence rate for the consistency of deep learning; presenting data-dependent sample complexity, and giving the upper and lower bounds. It is expected to publish 5-8 high quality papers, apply 2-4 patents, develop a prototype system and supervise several graduate students.

“深度学习”目前一般泛指参数化可微分构件的“深度神经网络”，在实际应用中取得了巨大成功，但同时也伴随着巨大的计算开销、人力成本、可验证性等问题。近年来日益兴起了基于非可微构件来构建的深度模型，如深度森林、深度Boosing等，但相关研究缺乏相应的理论支撑。本项目拟围绕非可微深度学习泛化性、稳定性、一致性和样本复杂度四方面展开研究，提供相应的理论支持与算法指导。主要研究内容：给出有效刻画间隔分布的深度学习泛化性理论；探索深度学习稳定性、并给出基于稳定性的深度学习泛化性理论；给出基于有限训练样本、依赖数据分布的深度学习一致性收敛理论；给出数据依赖的样本复杂度理论分析，推导深度模型的样本复杂度上下界。本项目研究可望产生高水平论文5-8篇，申请专利2-4项，研制一个原型系统以及培养多名研究生。

非可微深度学习一般以决策树为基本构件单元的深度模型，近年来受到越来越多的关注。本项目致力于非可微深度学习的理论研究：给出了有效刻画深度学习的间隔分布泛化性理论；给出了针对对抗扰动的非可微深度学习稳定性理论与方法；给出了基于有限训练样本、依赖数据分布的深度学习一致性收敛理论；以及给出了数据依赖的样本复杂度理论。在执行期间，项目成果包括Artificial Intelligence、ICML、NeurIPS等国际一流期刊/会议10篇，其中CCF-A类论文7篇，CCF-B类论文1篇，CCF-C类论文2篇，申请发明专利3项，研制原型系统1项，培养研究生4人，该项目按照原计划顺利执行。