Optimal planning of hybrid energy storage systems using curtailed renewable energy through deep reinforcement learning

Dongju Kang, Doeun Kang, Sumin Hwangbo, Haider Niaz, Won Bo Lee, J. Jay Liu, Jonggeol Na

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


Energy management systems are becoming increasingly important to utilize the continuously growing curtailed renewable energy. Promising energy storage systems, such as batteries and green hydrogen, should be employed to maximize the efficiency of energy stakeholders. However, optimal decision-making, i.e., planning the leveraging between different strategies, is confronted with the complexity and uncertainties of large-scale problems. A sophisticated deep reinforcement learning methodology with a policy-based algorithm is proposed to achieve real-time optimal energy storage systems planning under the curtailed renewable energy uncertainty. A quantitative performance comparison proved that the deep reinforcement learning agent outperforms the scenario-based stochastic optimization algorithm, even with a wide action and observation space. A robust performance, with maximizing net profit and a stable system, confirmed the uncertainty rejection capability of the deep reinforcement learning under a large uncertainty of the curtailed renewable energy. Action mapping was performed to visually assess the action the deep reinforcement learning agent took according to the state. The corresponding results confirmed that the deep reinforcement learning agent learns how the deterministic solution performs and demonstrates more than 90% profit accuracy compared to the solution.

Original languageEnglish
Article number128623
StatePublished - 1 Dec 2023

Bibliographical note

Publisher Copyright:
© 2023 Elsevier Ltd


  • Curtailed renewable energy
  • Energy management system
  • Machine learning
  • Mathematical programming
  • Process planning
  • Reinforcement learning


Dive into the research topics of 'Optimal planning of hybrid energy storage systems using curtailed renewable energy through deep reinforcement learning'. Together they form a unique fingerprint.

Cite this