基于深度强化学习的插电式混合动力汽车智能能量管理方法研究

Aiming at some basic problems in energy management of plug-in hybrid electric vehicle(PHEV): (1) the multi-source miscellaneous information from PHEV’s driving environment should be analyzed, and in order to extract feature from multi-source high-dimension information of driving cycle, a feature extraction method based on deep learning should be proposed, which will provide a foundation for the fusion of multi-source driving cycle information. (2) A redundant information culling algorithm for driving cycle’s characteristic states should be developed, and then an information fusion method for multi-source driving cycle information can be proposed. Based on deep neural network, a nonlinear mapping relationship between characteristic states of driving cycle and value of control actions can be constructed, which can provide support to mine the action mechanism of multi-source high-dimension driving cycle information on energy management effect. (3) An intelligent energy management strategy for PHEV based on deep reinforcement learning architecture can be constructed. A dynamic reward modification algorithm based on model and driving cycle states should be developed. Meanwhile, by constructing stable neural network objective functions, and selecting suitable stable training and learning methods for deep neural network, the self-training and self-learning of the energy management based on deep reinforcement learning can run stably and efficiently. The research results of this project have scientific significance in three aspects: revealing action mechanism of multi-source high-dimension driving cycle information on energy management effect, completing intelligent energy management theories for PHEV, and promoting the intelligent level of power system control.

针对插电式混合动力汽车智能能量管理的基础问题：（1）分析具有多源混杂特点的PHEV行驶环境信息，提出基于深度学习的多源高维行驶工况的特征提取方法，为实现多源工况信息融合奠定基础；（2）构建工况特征状态的冗余信息剔除算法，提出连续型工况状态表征的融合方法，建立基于深层神经网络的工况状态与控制动作价值间的非线性映射关系，为挖掘工况信息对强化学习能量管理效果的作用机理提供支撑。（3）建立基于深度强化学习算法的PHEV智能能量管理方法架构，构建基于模型和工况状态的强化学习动态奖励修正算法，同时，通过构建神经网络稳定性优化目标函数，提出深层神经网络的稳定性训练和学习方法，最终实现基于深度强化学习算法的最优能量管理策略高效稳定的自主训练和学习。项目研究成果对揭示PHEV多源高维工况状态和控制动作对能量管理效果的作用机理，完善PHEV智能能量管理理论，提升动力系统控制的智能化水平具有重要的科学意义。

面向插电式混合动力汽车节能化、绿色化发展，项目以深度学习和强化学习在 PHEV 智能能量管理上的应用为目标，针对新能源车辆的智能能量管理策略开展了系统的创新性研究：（1）构建工况特征状态的冗余信息剔除算法，提出连续型工况状态表征的融合方法，建立基于深层神经网络的工况状态与控制动作价值间的非线性映射关系，为挖掘工况信息对强化学习能量管理效果的作用机理提供支撑。（2）建立基于深度强化学习算法的PHEV智能能量管理方法架构，构建基于模型和工况状态的强化学习动态奖励修正算法，同时，通过构建神经网络稳定性优化目标函数实现控制能量管理去离散化。（3）针对动力系统各部件复杂耦合约束及动力电池长期规划带来的稀疏奖励问题，进一步优化基于深度强化学习的能量管理策略，避免局部最优与训练不安全。（4）开展环境交互复杂信息的能量管理研究，考虑车辆行驶与交通、电网外部环境交互，提出以面向系统能耗优化控制。项目研究成果对揭示PHEV多源高维工况状态和控制动作对能量管理效果的作用机理，完善PHEV智能能量管理理论，提升动力系统控制的智能化水平，对军民新能源汽车发展都具有重要的科学意义。