Deep Reinforcement Learning-Based Energy Management for Hydrogen-Enabled Community Microgrids Under Uncertainty

Abstract

Hydrogen-enabled community microgrids can improve renewable energy utilization and local resilience, but their operation is complicated by intermittent generation, uncertain residential demand, dynamic electricity prices, and the coupled dynamics of battery and hydrogen storage. This paper develops a proximal policy optimization (PPO)-based energy management system (EMS) for a grid-connected community microgrid integrating photovoltaic and wind generation, battery storage, an electrolyzer, hydrogen storage, a fuel cell, and diesel backup. The EMS is formulated as a Markov decision process with an 11-dimensional state and three continuous control actions for battery, diesel, and hydrogen dispatch; grid exchange is determined from the residual power balance. The framework is evaluated using 8,760 hourly observations for a 1,000-household community in Rockhampton, Australia. Under the normal operating scenario, the learned policy produced a net annual operating revenue of A$195,690.67, a renewable fraction of 91.2%, a carbon intensity of 0.085 kg CO2/kWh under the adopted accounting boundary, and 99.77% load satisfaction. With the hourly grid-outage probability increased from 1% to 5%, the policy retained A$169,892.21 in net operating revenue and supplied 98.79% of demand, supported by a 413% increase in battery discharge and a 429% increase in diesel generation. The results demonstrate the potential of PPO for coordinated battery-hydrogen dispatch while also highlighting sensitivity to renewable-profile variability, training stability, and the choice of evaluation boundary.