基于神经网络权值随机化的模型选择和学习算法的理论研究

With the explosive growth of data information, how to efficiently obtain the knowledge from massive data has become a hot topic, which brings the neural network as a typical representative of machine learning methods a huge impact. In the traditional learning of neural networks, all the input weights、hidden layer biases and the output weights are need adjusted, which leads to high computational cost. Weight randomization method of neural networks is that both the input weights and the hidden layer biases are randomly assigned, and only the output weights need tuning, which extremely reduced the computational burden. This project bases on the weight randomization strategies of neural networks, aims at sparse neural networks model, and focuses on the theoretical assessment for the weight randomization strategies of neural networks. Specifically, three aspects need to be investigated: firstly, how to estimate the number of hidden neurons after the weights being randomized; Moreover, we discuss the generalization of SCNs in theory; At last, we study the universal consistency of ELM learning algorithm. In conclusion, the desired achievements will not only complet the theretical analysis of weight randomization strategies, but also provide the theoretical support on model selection and learning algorithms of neural networks.

随着数据信息量爆炸式增长，如何从海量数据中高效地获取知识已成为人们关注的热点，这项需求给以神经网络为典型代表的机器学习技术以巨大冲击。传统神经网络学习中，内权、阈值和外权都通过训练得到，因此很难实现神经网络的高效性。神经网络权值随机化是通过一定的机制随机选择内权和阈值，只有外权需要训练，从而大大降低了神经网络的计算负担。本项目基于权值随机化策略，以稀疏神经网络结构为目标，聚焦研究神经网络权值随机化后的模型选择和学习算法的理论性态，以为更加高效的神经网络学习方法的选择提供理论基础。具体内容包括：神经网络权值随机化后神经元个数的选择；SCNs算法的泛化性；ELM型学习算法的万有一致性。所得结果将不仅完善神经网络权值随机化的理论研究，更重要的是为神经网络的模型选择和学习算法提供理论支撑。

人工神经网络是机器学习领域中一种有效而活跃的学习技术，为机器学习提供模型，使得计算机不再只是执行命令的机器，一定程度上具有举一反三的能力。随着数据信息量爆炸式增长，如何从海量数据中高效地获取知识已成为人们关注的热点，这项需求给以神经网络为典型代表的机器学习技术以巨大冲击。传统神经网络学习中，内权、阈值和外权都通过训练得到，因此很难实现神经网络的高效性。神经网络权值随机化是通过一定的机制随机选择内权和阈值，只有外权需要训练，从而大大降低了神经网络的计算负担。本项目回答了基于权值随机化思想的ELM型学习算法的万有一致性，同时给出ELM是强万有一致性时对应激活函数所满足的充分必要条件，并给出神经元的选择策略；最后证明含有三个隐层的深度神经网络不仅可以克服浅层网络的不足，而且具有浅层神经网络的最优阶特征。所得结果将不仅完善神经网络权值随机化的理论研究，更重要的是为神经网络的模型选择和学习算法提供理论支撑。