Short-term forecasting of electricity imbalances using artificial neural networks

Currently, the problem of improving results of short-term forecasting of electricity imbalances in the modern electricity market of Ukraine is a current problem. In order to solve this problem, two types of neural networks with recurrent layers LSTM and LSTNet were analyzed in this work. A comparison of the results of short-term forecasting of daily schedules of electricity imbalances using LSTM and LSTNet neural networks with vector autoregression model (VARMA) was carried out. Actual data of the balancing market were used for the research. Analysis of the results shows that the smallest forecast error was achieved using the LSTM artificial neural network architecture.


Introduction
Today, one of the important factors affecting the reliability and efficiency of the Integrated Power System (IPS) of Ukraine [1,2] is the instability of electricity imbalances, which has been observed since the implementation of the new model of the wholesale electricity market [3].
Electricity imbalance -calculated in accordance with the rules of the electricity market [4] for each calculation period, the difference between the actual volumes of sale or consumption, import, export of electric energy of the party responsible for the balance, and the volumes of purchased and sold electric energy registered in accordance with the market rules.The overall electricity imbalances includes: the error of the forecast of the consumption of electric energy, the error of the forecast of the volumes of generation of renewable energy sources (RES) [5], the difference between the actual release of balancing service providers and the dispatch team, interruptions in power supply due to emergency situations in the power system.
At present, forecasting the value of overall electricity imbalances is a rather urgent task.In real time, the transmission system operator buys or sells the electricity needed to cover imbalances in the balancing market.According to the results of the balancing market sessions, the prices of imbalances are determined, according to which market participants get invoices for non-compliance with the declared volumes of electricity sales.
Access to prior information on electricity imbalance allows: • the transmission system operator to improve the accuracy of creating the energy system operating mode.It also enables optimizing the balancing market and ancillary services market operations, thus increasing the reliability of the IPS of Ukraine functioning; • market participants, including power companies, to determine their participation strategies in different market segments to reduce the cost of electricity [6]; • reducing the cost of imbalances for electricity suppliers, which has a positive impact on the retail electricity market and the prices of end consumers [7].
Previous developments to solve this problem were related to the study of autoregression models (ARIMA, ARIMAX, SARIMA, VARMA) [8].Considering the widespread use of neural networks in various fields of application, it is appropriate to use this tool to solve the problem under study.Therefore, the purpose of this work is to develop and research models of artificial neural networks of deep learning for short-term forecasting of electrical energy imbalances of the power system and to compare the results with the results obtained using the VARMA vector autoregression model, which according to previous studies had the most accurate results.

Aim and methodology
Two types of neural networks with recurrent LSTM layers [9] and LSTNet [10] were built to predict electricity imbalances.After which two linear layers with hyperbolic tangent and sigmoid activation functions, respectively.Training was performed on 100 epochs and the ADAM optimization algorithm with a training step of 0.001.
The structure of the LSTM network is described using formulas (1-6) [11].At the first stage, the information to be replaced is selected according to the conditions of the sigmoid layer (1).After that, the next layer determines the values to be updated (2) and builds a vector of values C t , which can be added to the state of the cell (3).Then the old state of the cell C t−1 is replaced by the new C t according to equation ( 4).The last step is to calculate the source information using several filters (5,6).
where σ is the activation function; x t -input data; h t−1 , h t -forecast values; C t is the correction vector of the memory cell; C t , C t−1 -state of the memory cell.The memory vectors (C) and the LSTM output of the previous step (h) are initialized to zero values at each prediction step x t .The vector C t represents a new version of the memory values, and the vectors f i and i i regulate the extent to which the memory vector is updated with new values.Due to the fact that the values of W f , W i and b f , b i are estimated during the learning process.The neural network regulates the update dynamics of C optimally from the point of view of minimizing the loss function.In addition, the presence of a memory vector makes it possible to reduce the effect of blurring or explosion of the gradient during the backpropagation of the error, while using a bypass connection, which smoothes the surface of the error of the neural network, significantly speeds up learning and allows to achieve a lower prediction error.
The built LSTNet network has the following components [10]: 1. Convolutional.The goal of an unconnected convolutional network is to extract short-term patterns in the temporal dimension.The output matrix simultaneously enters components 2 and 3. 2. Recurrent.Uses RELU as a hidden activation function.Result: hidden state for each timestamp.
3. Recurrent-skip.A repeating structure with temporal relationships to extend the temporal range of information, thus facilitating the optimization process.The structure of this layer is presented in the form of the following formulas: where p is the number of hidden cells; 4. Temporal Attention Layer.Learns the weighted sum of hidden representations at each window position of the input matrix.As a result, a concatenation of the context vector and the hidden representation of the last window together with a linear projection is obtained.5. Autoregression.Breaks the final forecast into linear and non-linear components.The linear component solves the problem of local scaling, while the nonlinear one contains repeating patterns.
Both models are developed in the PyCharm environment using the Python programming language.

Results
Overall data include hourly volumes of positive and negative imbalances of the IPS of Ukraine for the period from 07/20/2020 to 07/16/2021 (a total of 8,784 values), training/test split is 90%/10%.The input vector includes 4 previous values; the forecasting horizon is 1 hour, the number of training epochs is 5000.The best prediction results were achieved with 20 hidden layer neurons.Data obtained from the official website of the National Energy Company Ukrenergo (balancing market and settlement of imbalances data).The test sample for comparison is 36 days.For comparison, the VARMA model was used.
Table 1 shows the average, maximum, and minimum relative errors for the test day of 07/20/2021 and the test sample as a whole, and table 2 shows the RMSE errors.According to the data indicated in tables 1 and 2 and illustrated in figures 1-4, the most accurate forecasting results of both positive and negative electricity imbalances were obtained using the LSTM model.At the same time, the graphs of daily MAPE and RMSE errors for LSTM results have less pronounced local extrema, which indicates the preservation of the trend throughout the entire range of the test sample, i.e. greater stability of forecasting results.The results obtained with the LSTNet model are much worse: by 50% compared to the other models for positive imbalances and by 47% and 66% compared to VARMA and LSTNet, respectively, for negative imbalances.When forecasting positive electricity imbalances using the LSTM neural network, the errors of forecasting results increase slightly, with the exception of the third test day.This is due to the significant asymmetry of the distribution of positive imbalances.The use of the studied neural network for forecasting negative imbalances made it possible to reduce forecasting errors by 2÷5 times.
For example, figure 5 shows the values of electricity imbalances predicted using the built models, compared with the actual ones for 07/20/2021.
The analysis of the received forecasting results of the considered models shows the need to improve the method to include the impact of external factors on the value of electricity imbalances.Great asymmetry of imbalance data can be adverse in particular by finding other influencing factors and building a more flexible model of the connection between them.The significant asymmetry of the imbalance data suggests that data preparation can strongly influence the accuracy of forecasting results.Additional, some kind of mixture distribution models could be used for improving of forecast accuracy.

Conclusions
The use of artificial neural networks made it possible to reduce the error of forecasting total electricity imbalances for different testing periods.LSTM shows the lowest values of MAPE forecasting error, for positive imbalances the daily forecast error is 20.73%, while for VARMA model the error is 40.86% and LSTM is 46.95%.When predicting negative imbalances, LSTM also has the lowest error value of 9.1%, compared to VARMA of 17.18% and LSTNet of 66.91%.The forecast on the entire data sample has significantly larger errors especially on the data of positive electricity imbalances, LSTM 66.25%, VARMA 140.84%,LSTNet 132.78% and for negative imbalance LSTM 17.98%, VARMA 34.2%, LSTMNet 52.7%.Therefore, the smallest error of forecasting results is achieved using an artificial neural network based on LSTM.In this way, the LSTM is the most effective method for imbalances forcasting in terms of flexibility and ability to model multivariate data, which essentially increases the effectiveness of forecasts.
The results of the conducted research prove the success of the proposed forecasting models and indicate the need for their further development taking into account additional factors.

Figure 1 .
Figure 1.Daily graphs of MAPE values for positive electricity imbalances.

Figure 2 .
Figure 2. Daily graphs of RMSE values for positive electricity imbalances.

Figure 3 .
Figure 3. Daily graphs of MAPE values for negative electricity imbalances.

Figure 4 .
Figure 4. Daily graphs of RMSE values for negative power imbalances.

Table 1 .
Assessment of the accuracy of the studied models using MAPE, %.
Figures 1-4 show daily values of MAPE, RMSE between actual ∆P f and forecast values of electricity imbalances ∆P p .

Table 2 .
Assessment of the accuracy of the studied models using RMSE, MW.