Conceptual principles of forecasting demand on the day-ahead market using changes in hourly bidded demand between previous similar days

On the basis of the Law on the Electricity Market adopted in 2017, the retail electricity market in Ukraine opened on January 1, 2019, and later, on July 1, 2019, the wholesale electricity market was launched. The day-ahead market (DAM) is one of the key segments of the Ukrainian electricity market. In order to implement trading strategy and successfully conduct business, to maximize the economic results in the specified market segment, it is important to understand the market situation and the structure of demand and supply. One of the indicators that must be taken into account when planning sales in the course of one’s activity is the demand for electrical energy. Currently, there are no universal algorithms in Ukraine suitable for short-term (per day) forecasting of the amount of electrical energy that will be traded on DAM. Therefore, to solve such a problem, a specialized forecasting algorithm is proposed. The basis of the developed algorithm is the possibility of considering the formulated problem in a parametric form, where, as indicators, the forecast and real data of the hourly demand on the DAM are used. At the same time, in order to find the forecast hourly demand on the market “a day ahead” – the values of the unknown indicators of the problem – an iterative method of their search is used based on statistical data of the amount of electricity purchases on the DAM, using the principle of multi-iteration analysis of changes in demand for previous similar days. The proposed algorithm is implemented in the MS Excel package, which indicates its versatility and ease of use. The high speed of obtaining a solution to the formulated problem is shown.


Introduction
The single-buyer model of the wholesale electricity market was also replaced by a market model based on bilateral markets, day-ahead and intraday markets, as well as balancing and ancillary services markets, where participants can freely trade electricity and power companies can provide services that ensure the stability of the power system and supply electricity to the final consumer.
One of the main segments for electricity market subjects for its purchase and sale is the DAM.Trading on DAM takes place according to the principle of margin pricing -sellers submit orders at the minimum price at which they are ready to sell, buyers -at the maximum price at which they are ready to buy.According to the results of auctions on the DAM, sellers sell at a price no cheaper than their bid, and buyers sell at a price no higher than their bid.
Hourly demand for DAM is the hourly amount of electric energy that customers declare and want to buy on DAM at the respective hourly prices.This amount directly affects both the marginal price that will be created and the amount of electricity that will be bought and sold.According to the law of supply and demand, the lower the demand for a product, the lower the price will be; the higher the demand, the higher the price will be.
This directly affects the volumes that will be accepted for sale for sellers on the DAM.If sellers incorrectly determine the declared demand volumes and set inappropriate prices in their sales bids -either their volumes will be accepted incompletely (in case of low demand) or accepted at a low price (in case of high demand value).At the same time, in Ukraine on the DAM, volumes for purchase are usually declared at the maximum possible price, so the value of the price is actually affected by the prices in the sellers' bids, and the incorrect determination of such a price affects the economy of the seller's enterprises.
Thus, there is a need to forecast the demand for DAM.But now in Ukraine there are no mechanisms, algorithms or mathematical models that would allow forecasting with a low forecasting error.

Aim and tasks
The purpose of the research is to determine an algorithm that will make it possible to forecast the hourly volumes of demand for DAM for the next day, and which will have a slight deviation of the forecast indicators from the real data (that is, it will be reliable and have a forecasting error within 10%).
The tasks of the research are: 1. Determination of various algorithms for calculating forecasted hourly volumes of demand for DAM; 2. Calculation of the prediction error of each of the identified algorithms; 3. Determination of the most accurate calculation algorithm; 4. Formulaic description of the calculation algorithm.

Approaches to forecasting hourly demand on DAM
Trading on DAM takes place at 12:00 for the following day (we will take studying day as D).Thus, on the D-2 day, it is possible to make a forecast for D. It is clear that such forecasting is short-term, so it is fully justified to use the most current and latest data available.The scope of the current research is 4 weeks.
It is necessary to understand that the demand on DAM depends on a large number of factors [1][2][3]: 1. Volumes of contracted bilateral agreements; 2. Seasonality; 3. Power outage due to the destruction of critical energy infrastructure facilities; 4. Day of the week; 5. Other factors.
To obtain forecast results that are closest to the real data, it is suggested to: 1. Use the trading results for the last available day; 2. Use the change in demand between the day for which the actual results are available and the day for which the forecast is made; 3. Take into account the change in demand, for example, over the previous 4 weeks between similar days of the week.
Given the possible volatility of the DAM, as well as the fact that from day to day the conditions of the DAM vary depending on both buy and sell bids, using as a basis for calculation the data of the most recent actual trading results is reasonable and appropriate.And taking into account that depending on the day, the consumption of electrical energy by consumers changes, there is a corresponding change in the demand for DAM.At the same time, usually, such a change in demand has a predictable nature -for example, on Saturday, under normal conditions, demand decreases compared to Friday.And on Monday, on the contrary, the demand increases significantly compared to Sunday.
Thus, to calculate the forecast hourly demand for DAM on D, it is necessary to calculate a certain value of the change in this demand between the corresponding days for previous periods, then add it to the actual value of the hourly demand for DAM in D-1.But the question ariseswhich algorithm to use to calculate this value of the change in demand.To answer this question, it is necessary to understand, firstly, the nature of the change occurrence, and secondly, how to take into account or balance these values for previous periods.
As already mentioned, the change in demand between days of the week can occur due to the natural change in consumption for the corresponding day of the week.But this is not the only factor.We can add the following indicators: 1. Contracting on the market of bilateral contracts; 2. Change in weather conditions; 3. Change in consumption of the final consumer due to independent conditions; 4. Manipulative behavior of buyers on the DAM; 5. Other factors (for example, an error when entering data on the trading platform).
It is clear that some of these indicators cannot be predicted and they can distort the final result of the calculation, worsening the accuracy of forecasting.But a certain mechanism or algorithm should be provided, which would allow either to reduce the influence of these indicators, or to reject these values.
Another aspect is the question of taking into account the value of the change in hourly demand for the calculation of the forecasting value.Actually, the answer to this question will give an understanding of how the calculation will be carried out.For comparison, 4 calculation options are considered: 1.As the arithmetic mean value of the change in demand; 2. As a weighted average value of the sum of positive and negative changes in demand, weighted by the number of days with a positive change in demand and a negative change in demand, giving preference to those days that are more in number (by increasing their weight) and that are closer to today's day.The essence of the algorithm is as following: (a) the number of days with positive and negative changes in demand is determined; (b) if all days have either a positive change or a negative one, then the arithmetic mean is calculated -there is no need for weighting; (c) if there are more days of a certain type, the weight of that type increases (one plus the ratio of the number of such days to the total number of days), and the weight of another type decreases (one minus the ratio of the number of such days to the total number of days); (d) if the number of days of both types is equal, then the arithmetic mean of the former two days, multiplied by 0.5, plus the arithmetic mean of the latter two days, multiplied by 1.5, is calculated).3.As a weighted average value of the sum of positive and negative changes in demand, weighted by the number of days with a positive change in demand and a negative change in demand, giving preference to those type of days that are more in number (by increasing their weight).The essence of the algorithm is as following: (a) the number of days with positive and negative changes in demand is determined; (b) if all days have either a positive change or a negative one, then the arithmetic mean is calculated -there is no need for weighting; (c) if there are more days of a certain type, the weight of that type increases (one plus the ratio of the number of such days to the total number of days), and the weight of another type decreases (one minus the ratio of the number of such days to the total number of days).4. As a weighted average value for positive and negative changes in demand, taking into account the deviation from the arithmetic mean of the absolute values of changes in demand to determine atypical days (determined by the author by experimental method to remove such atypical days).The calculation algorithm is as following: (a) the sum and number of positive and negative days are calculated, the arithmetic mean of absolute values among changes in demand is determined; (b) an iterative process of discarding non-typical days to reduce the influence of nonrepresentative values on the final value of the forecast.For this, the so-called "deviation weight" is calculated by calculating the inverse of the relative deviation for each value of the change in demand from the calculated arithmetic mean.If the calculated value is less than 0, such an indicator of the change in demand is rejected and a recalculation is made, starting from the first stage, but already without this indicator.After the iterative process, the average value of the "deviation weight" is calculated separately for positive changes in demand, separately for negative changes in demand; (c) adjustment of the weight of positive values of changes in demand (or negative valuesthe value of the adjustment will be reversed for positive values -there is no significant difference).Weight values for positive changes in demand are adjusted by this factor.After that, the weighted average forecast value of the change in demand for the day under study is calculated, where the weight is the adjusted values of the "deviation weights".
It is worth to mention that in practice the most accurate forecasting results come from similar hours from previous day.Such principle is behind reasoning of using previous day (D-1) as a basis for adding certain value of change in demand between the corresponding days for previous periods.
For calculations in the algorithms and comparison of forecast calculated values with real data, real data of hourly demand based on the results of auctions on DAM in the period from July 30, 2022 to August 31, 2022 are used.In order to simplify the calculations, verify the reliability of the proposed algorithms and achieve the set goals, the depth of the analysis of the change in demand is 4 previous weeks, therefore, the actual hourly demand data based on the results of the DAM trades from July 1, 2022 were used for the analysis in the calculations.

Algorithms calculation results
Calculations for the proposed algorithms were performed in the MS Excel software package.This ensures the universality and ease of use of the specified calculations, is clear and shows a high speed of solving the tasks and low or even zero costs for the implementation of the algorithm.
The results of the calculations are presented in the form of a comparative table.Figure 1 shows an example of the results of the forecast calculation and real data based on the results of the DAM auctions.It should be noted that this calculation will be used both to determine the relative error of forecasting (figure 3), and further to determine which of the algorithms turned out to be the most accurate both in terms of the smallest deviation for the entire period and in terms of the number of hours, in which it was more accurate.
Table 1 presents a comparison of the calculation results of all four algorithms.Presented data on the sum of absolute values of absolute deviation and the number of times such algorithms were more accurate than others.There should be noted two things: 1.If more than one type of algorithm was the most accurate in one hour (the same result of the forecast calculation) -all such algorithms are considered as more accurate for the corresponding hour.The larger the value, the greater the number of times the algorithm was more accurate; 2. Regarding the sum of absolute values of absolute deviations -this value is exclusively informative for comparing all algorithms and determining the one that was closer to the real data based on the results of the auctions on the DAM.The smaller the value, the more accurate the algorithm.
As can be seen from the table 1, Algorithm 4 shows both the smallest deviation from the real data and the largest number of times when it was more accurate.At the same time, the application of algorithm 4 allows both the rejection of atypical days (which is impossible when applying algorithm 1) and the monitoring of atypical behavior of market participants when submitting bids on the DAM.
Table 2 illustrates the calculation results and accuracy of algorithm 4 for a typical day.
2. Calculating sum of positive and negative A h w separately: 3. Calculating separate number of positive N h pos for A h w > 0 and negative N h neg for A h w < 0.

Calculating arithmetic mean of absolute values A h w :
A h aver,abs =

Calculating first iteration of absolute value of inverse relative deviation
for all A h w (so-called "weight of deviation"): If value D h w,weight,I < 0 then A h w significantly deviates compared to other indicators, so it must be discarded, zeroed and recalculation should be done (II iteration) and calculate N h pos,iter−last , N h neg,iter−last and D h w,weight,iter−last .Such recalculations (iterations) are performed until there are no significant deviation.Maximum number of iterations in this example will be four (equalizes to the number of analyzed A h w ). 6.After the final iteration average value of the sum D h w,weight,iter−last is calculated separately for positive and negative A h w , if the number of such values is not 0: This concludes formulaic expression of researched algorithm.Provided that there is one of the main goals of EU as being climate neutral by 2050, companies need to achieve such goal and remain economically profitable at the same time.Therefore it is necessary to conduct business with the aim of maximizing its finances at the every possible situation and market -such as DAM [4].

Conclusions
To this day, achieving environmental goal as being climate neutral for most EU countries is still a challenge, taking into account the need to get rid of one of the most largest sources of emissions -coal -by 2030.Although the increase in energy resource prices and their shortage in Europe during the last months of 2022 has created new challenges for achieving environmental goals in the EU and Ukraine.
Forecasting the demand on the DAM is one of the components that must be carried out for successful business and optimization of business processes of trading companies in the electricity market.Given that quite often the income of such companies depends on how they will sell on the DAM, the choice of the correct approach to forecasting is quite acute.
In the article, four algorithms for forecasting the hourly demand for DAM are considered, corresponding calculations are performed, and a comparative analysis of the calculation results is carried out.For each of the algorithms, their prediction errors were calculated and the most accurate algorithm out of four was determined.
A formulaic description of the most accurate algorithm is given -as a weighted average value for positive and negative changes in demand, taking into account the deviation from the arithmetic mean of the absolute values of changes in demand to determine atypical days.Forecasting is carried out on the basis of statistical data on the amount of electricity purchases on the DAM, using the principle of multi-iteration analysis of changes in demand for previous similar days of previous months (with the possibility of increasing the sampling depth to quarters or years).At the same time, it should be noted that all calculations were performed in the MS Excel package, which indicates their ease, availability, speed, as well as the low or zero cost of implementing such an algorithm.
As a result of the application of the specified algorithm, there is also the possibility of tracking atypical behavior of market participants on the DAM, which will require additional attention and research.

Figure 2 Figure 1 .
Figure 1.Calculation results of every algorithm and real data.

Figure 2 .
Figure 2. Calculation results of every algorithm and real data.

Table 1 . 1 .
Algorithms calculation results comparison of aggregated deviation data and number of more accurate calculations.Sum of modules of absolute deviations 224 284.7 249 400.7 249 763.7 219 057.9 Times of being more accurate 322 201 196 360 5. Description of the most accurate algorithm Calculations can be made in D-2 after receiving the actual results of the auctions on the DAM in D-1.It should be noted that all calculations are performed separately for each hour h.Calculating demand change on the DAM for the previous 4 weeks w for the following days d: D-6 and D-7, D-13 and D-14, D-20 and D-21, D-27 and D-28:

Figure 3 .
Figure 3. Calculation results of every algorithm and real data.

Table 2 .
Accuracy of algorithm 4 on the example of one day (04.08.2022).
9 10.Calculating forecasted value of the demand on the DAM: