Deep-learning- and reinforcement-learning-based profitable strategy of a grid-level energy storage system for the smart grid

A profitable operation strategy of an energy storage system (ESS) could play a pivotal role in the smart grid, balancing electricity supply with demand. Here, we propose an AI-based novel arbitrage strategy to maximize operating profit in the electricity market composed of a grid operator (GO), an ESS, and customers (CUs). This strategy, the buying and selling of electricity to profit from a price imbalance, can also cause a peak load shift from on-peak to off-peak, a win-win approach for both the ESS operator (EO) and the GO. Particularly, to maximize the EO's profit and further reduce the GO's on-peak power, we introduce a stimulus-integrated arbitrage algorithm, providing an additional reward to the EO from the GO with different weights for each peak period. The algorithm consists of two parts: the first is recurrent neural network-based deep learning for overcoming the future uncertainties of electricity prices and load demands. The second is reinforcement learning to derive the optimal charging or discharging policy considering the grid peak states, the EO's profit, and CUs’ load demand. We find it significant that the suggested approach increases operating profit 2.4 times and decreases the on-peak power of the GO by 30%.