Identification of the energy crisis in the EU electricity markets

In the second half of 2021 - the first half of 2023, the EU electricity markets were under unprecedented pressure caused by soaring gas prices and aggravated by other fundamental factors. This period corresponds to the energy crisis in the EU electric power sector. However, all EU electricity markets responded to the energy crisis differently. This paper proposes a methodology for revealing the energy crisis in the electricity market, which is based on the tools of descriptive statistics, explanatory data analysis and k-means clustering which allows to identification duration and phases of the energy crisis such as escalation, peak, and recovery phases. Using this methodology, the energy crisis was identified in the EU electricity markets (by separate bidding zones), and as a result, the starting and ending points of the energy crisis, as well as the changing by phases, were determined. Additionally, the use of hierarchical and agglomerative clustering methods made it possible to divide all EU electricity markets into 9 clusters by the evolution of the energy crisis and identify unaffected, resilient vulnerable markets to the energy crisis, and markets with their own dynamics.


Introduction
Electricity is a critically important commodity in modern society.Therefore, their price dynamic depends on many fundamental factors: solvent demand, determined by the macroeconomic situation, the structure of supply, determined by the technical availability and economic efficiency of the certain generator units, and the situation in adjacent energy markets.The continuum of fundamental factors determines a wide range of studies of electricity markets and pricing trends in them [1][2][3][4][5][6][7].The energy crisis in the EU electric power sector, which has been developing since the 3 rd quarter of 2021, is the result of the combined action of the above factors.The introduction presents the main features of the evolution of the energy crisis in the EU electricity markets based on the EU Quarterly Electricity Market Monitoring Reports data from the 3 rd quarter of 2021 to the 4 th quarter of 2022 [8][9][10][11][12][13].
According to the Electricity Market Monitoring Report for the 3 rd of 2021 [8], the energy crisis in the electricity markets began in the second half of 2021.Since then, electricity prices have skyrocketed several times.Initially, the rise in electricity prices was attributed to economic recovery from the Covid-19 pandemic, which led to a recovery in electricity consumption levels to prepandemic levels, as well as tensions in gas supply chains.The surge in gas prices triggered gas-to-coal switching and consequent increase in coal and carbon prices.However, the force of the energy crisis was mitigated by the growing share of non-fossil generation (renewable and nuclear).At that time, it was believed that the electricity market mechanism could self-manage the energy crisis through (i) competitive market structure; (ii) dynamic pricing; and (iii) risk hedging.
However, the energy crisis continued to evolve [9] and the additional increase in intermittent renewable electricity generation required a reduction in nuclear generation.Thus, in the fourth quarter of 2021, electricity prices broke several new records.In early 2022, this pressure softened and then increased with renewed vigour due to Russia's invasion of Ukraine.
In the first half of 2022 [10,11] electricity prices hit new records, driven by high gas, coal and carbon prices, reduced pipeline gas flows and security concerns over gas supply.These trends were accompanied by low nuclear and weak hydropower generation, as well as the growth of intermittent renewable generation.In response to higher electricity prices, businesses have slightly reduced their electricity consumption, despite increased economic activity.This in turn led to a divergence in carbon and gas prices and provoked a further gas-to-coal switching.At this time, the need to intervene in energy markets to protect consumers was realised and formalised in the REPowerEU plan [14].
The third quarter of 2022 [12] was characterised by a reduction in EU energy supply chains from Russia (imports of coal and pipeline gas were halted from August 2022), interruptions in Norwegian gas flows, reduced nuclear generation due to unplanned outages, weak hydropower generation due to droughts, increased renewable and fossil fuel generation.At this time, intervention reducing gas demand [15] and energy bills [16] were decreed.Thus, electricity consumption fell below the pandemic level of 2020, despite an increase in economic activity.
Electricity price dynamics in the fourth quarter of 2022 [13] were determined by slowed down economic activity, amid elevated inflation.Also their reduction was supported by a mild temperature.At the same time, electricity markets continued to be pressurised by reduced nuclear and weak hydropower generation, with increased intermittent renewable generation.However, increased gas storage reserves had already been established and coal imports were diversified from around the world.But the main achievement that determined the possibility of overcoming the energy crisis was the adopted system of interventions: Regulation on enhancing solidarity through better coordination of gas purchases, exchanges of gas across borders and reliable price benchmarks [17], Regulation establishing a market correction mechanism to protect citizens and the economy against excessively high prices [18].
The above allows to define the energy crisis in the EU electricity markets as a phenomenon to be investigated in detail.
The objective of this study is only to determine the duration and phases of the energy crisis in the EU electricity markets.The main hypothesis of the study can be defined as follows: electricity markets in the EU member states differed by the evolution of the energy crisis.
The rest of the paper is organised as follows.Section 2 presents the methodology and data for identifying the duration and phases of the energy crisis in the electricity markets of the EU member states.Section 3 is devoted to the results of the identification of the duration and phases of the energy crisis in the electricity markets of the EU member states.Section 3.1 presents empirical results for identification energy crisis in the DELU bidding zone.Section 3.2 focuses on the differences in the duration of energy crisis phases in the electricity markets of EU member states.Section 3.3 provides a clustering of the electricity markets of EU member states according to the phases of energy crisis in electricity markets.The conclusions are presented in Section 4.

Methodology and Data
The study consecutively gives the answers on the following questions: (i) where are the starting and ending points of the energy crisis in the electricity market; (ii) the phases of the energy crisis and their duration; (iii) the distribution of electricity prices by the phases of the energy crisis; and (iv) the clustering of electricity markets in EU member states by the impact of the energy crisis.
To answer these questions, we used the k-means cluster analysis of electricity prices [19,20].Price clustering is most often used in financial exchange markets [21,22,23], with specificity during the financial crisis [24,25].However, this method has recently become increasingly popular for energy markets [26,27,28], including electricity markets [29,30].The last one became possible since energy commodities have acquired the characteristics of exchange-traded goods.In electricity markets, spatial clustering of price zones [31] and temporal clustering of price spikes and price periods [32] have gained special attention.In this study, electricity price clustering is used to identify abnormal electricity prices and their distribution by periods.
Given the fact that electricity prices are cyclical [33], prior to the cluster analysis, their trend, i.e., the dynamics cleared of seasonality and random fluctuations, was determined using additive explanatory data analysis.Based on the trend component of electricity prices, a cluster analysis was conducted to determine the starting and ending points, as well as the phases of the energy crisis in electricity markets.
The phases of the energy crisis were determined based on technical analysis and the minimummaximum method was used to identify support and resistance lines [34].Crossing of the support and resistance lines means change the phase of the energy crisis.
The use of the elbow method allows us to determine the feasibility of dividing the dynamic series into 2 or 3 clusters.In the case of clustering into 2 clusters, only pre-crisis, crisis and post-crisis phases can be identified [35], while the division into 3 clusters allows to identify phases within the energy crisis, namely crisis escalation, critical peak of crisis, and crisis recovery [36].Thus, the paper identifies the following phases of the energy crisis in the electricity market: escalation (cluster 1) lasts after the 1st to 2nd crossing of the support and resistance lines; peak (cluster 2) lasts after the 2nd to 3rd crossing of the support-resistance lines; recovery (cluster 1) lasts between the 3rd and 2nd crossing of the support and resistance lines; Cluster 0 corresponds to the pre-crisis and post-crisis periods.
For each phase and the energy crisis as a whole, descriptive statistics of electricity prices are determined, including mean, standard deviation, variation, minimum, maximum, first and last prices.
The proposed research algorithm was implemented in Python in Jupeter Notebook.Identification of the energy crisis for each of the electricity markets of the EU member states allows for their clustering, for which the methods of hierarchical (Ward's method and Euclidean distance) and agglomerative clustering (centroid method) are used according to the following parameters: starting point of the energy crisis, duration of the energy crisis, maximum reached and price range (as the ratio between the maximum reached and the price at the starting point of the energy crisis).
To identify the energy crisis in the electricity markets of the EU member states, day-ahead electricity prices from the ENTSO-E Transparency Platform for the period 01.01.2021-01.09.2023 were used [37].Hourly data were aggregated using the MS Power Query add-on package in the format of baseload electricity prices.

Energy crisis in the electric power sector of DE-LU bidding zone
This section presents a detailed application of the proposed methodological approach on the example of the DELU bidding zone, the largest electricity market in the EU, which covers the electric power sector in Germany and Luxembourg.Fig. 1 shows the dynamics of electricity prices in the DELU bidding zone in January 2021 -August 2023, which was used to establish the existence of an energy crisis and identify it duration and phases.The results of the analysis are presented using descriptive statistics tools (Table 1 and Table 2).
In January 2021 -September 2023, the distribution of original electricity prices in the DELU bidding zone was abnormal and heterogeneous: (1) the mean was higher than median by 37 €/MWh, incl. in 2021 by 23   The trend of electricity prices in the DELU bidding zone, identified by the additive explanatory data analysis, allows to confirm their abnormality and heterogeneity in the period under study: (1) mean was higher than median by 35 12.22) was lower than the previous ones, which indicated the recovery from the crisis.The only exception was 13.12.2022,when a new round of energy crisis could have been triggered, but the price trend continued its decline.
Thus, the distribution of electricity prices by trend component allows us to confirm (a) the existence of the energy crisis in the electricity market of the DELU bidding zone; (b) the exit from it in 2023.At the same time, there was a decrease in the lower and upper price extremes, which were significantly influenced by seasonality.Thus, it is only on the basis of the trend of electricity prices energy crisis can and should be determined.
The clustering of electricity prices into 3 clusters made it possible to determine the starting and ending points of the energy crisis and price dynamics by its phases.Based on the minimum-maximum method, it was found that the support and resistance lines for the DELU bidding zone were at the price levels of 29 €/MWh, 137 €/MWh, 281 €/MWh, 590 €/MWh, the crossing of which meant a change in the phase of the energy crisis.Thus, the following can be defined: 1) from the beginning of 2021 until 12.09.21electricity prices moved in a horizontal corridor; 2) on 13.09.21there was a break of the support and resistance line at 138 €/MWh, which can be defined as the starting point of the energy crisis in the DELU bidding zone, after which an upward trend was observed.The next support and resistance line at 281 €/MWh was briefly broken, but prices subsequently returned to the previous price range.On 25/06/22 electricity prices broke the support and resistance line again, rising to 284 €/MWh, briefly tried to return to the previous price range, but continued to rise.Therefore, the period from 13.09.21 to 24.06.22 can be defined as the escalation phase of the energy crisis; 3) the breakdown of support and resistance lines on 25/06/22 can be defined as the beginning of the peak phase of the energy crisis -a period of both rapid price rise and fall.The price maximum was reached on 28/08/22 at 590 €/MWh, while the peak phase ended on 29/09/22 at 282 €/MWh; 4) the break of the support and resistance line on 30/09/22 marked the beginning of the recovery phase, during which another price peak was recorded on 28.11.22-17.12.22, but sooner prices continued their decline and on 24.12.22 they reached the pre-crisis level; 5) 23.12.22 can be defined as the ending point of the energy crisis in the DELU bidding zone.Short-term rises took place, but each of them was lower than the previous ones and further prices returned to the horizontal corridor.
Descriptive statistics of electricity prices in the DE-LU bidding zone by the phases of the energy crisis are highlighted in Table 2.
Table 2 Thus, the energy crisis in the DELU bidding zone lasted for 467 days, during which electricity prices varied by 46%.The range of prices was 502 €/MWh or 670 %, including 61 €/MWh in the first quartile, 43 €/MWh in the second, 96 €/MWh in the third, and 302 €/MWh in the fourth, meaning that electricity in the DELU bidding zone was traded in a high price range most of the time.In terms of percentage, the escalation phase took up 61%, during which prices increased by 101%, but were moderately variable at 30%, with a price range of 442%.The peak phase was 21% long, with a price range of 236% and the lowest variability at 24%.The recovery phase was the shortest and took 18% of the time, during which prices decreased by 45%, but were highly variable at 43%, with a price range of 409%.

Differences in the energy crisis of the EU electricity markets
The electric power sector and the electricity market of each EU member state is unique.Therefore, the energy crisis in each country had its own specificity in terms of starting and ending points, duration by phases, dynamics, and strength of electricity price fluctuations.This subsection presents the results of the identification of the energy crisis for each electricity market (bidding zone) of the EU member states (Table 3, Appendix A), using the developed and validated methodological approach.The study has revealed the following features of the emergence and умщдгешщт of the energy crisis in the electricity markets of the EU member states.
There is no clearly defined starting point for the energy crisis in the electricity markets among EU member states.The EU electricity markets can be divided into 6 groups according to the starting point of the energy crisis: 1) in Central Europe (Austria, Belgium, Croatia, Denmark, Germany, Italy, the Netherlands), the energy crisis began in mid-September 2021 (12.09.21-17.09.21), when electricity prices crossed the threshold of 143 €/MWh on average.Among this group of countries, the energy crisis started the latest in Italy, which had naturally higher electricity prices; 2) in Eastern Europe (Czech Republic, Hungary, Greece, Romania, Slovakia, Slovenia), it began in early October 2021, when electricity prices crossed the threshold of 159 €/MWh on average.Slovenia, which is highly integrated with Central European countries, was the earliest to experience the energy crisis; 3) in the Nordic and Baltic countries (Lithuania, Latvia, Estonia, certain bidding zones of Norway and Sweden, Finland, and Poland), the energy crisis began in November 2021 (09.11.21-25.11.21), when electricity prices exceeded 120 €/MWh on average.The earliest energy crisis began in Poland, although it had naturally higher prices, and it is poorly integrated with the countries in the region.In contrast, Norway was the latest to experience the energy crisis.Although Norway has strong integration ties, its electricity markets are export-oriented and rely on hydropower generation, which is independent of fossil fuel prices.
4) in some Nordic bidding zones (NO1, NO2, SE1, SE2), it was impossible to identify the starting points of the energy crisis, as their electricity markets were self-sufficient and based mainly on nuclear and hydropower facilities; 5) in the Iberian countries (Spain, Portugal), the energy crisis began at the earliest, at the end of August 2021, when electricity prices crossed the threshold of 114 €/MWh, but their electricity markets were strongly dependent on gas generation and consequently gas prices; 6) the energy crisis in Bulgaria's electricity market occurred separately, at the end of June 2021, when electricity prices exceeded 167 €/MWh (most likely due to national circumstances rather than European trends).
Also, the EU electricity markets can be divided into 4 groups according to the date of the price peak: 1) in 20 out of 31 EU bidding zones, the peak of the energy crisis was in August 2022 and can be associated with a surge in demand for gas due to the suspension of pipeline transport of russian gas; 2) in some Nordic bidding zones (NO3, NO4, SE1, SE2, SE3), the price peak occurred in December 2022, and in some of them (FI, SE4), a double price peak was recorded, in August and December 2022, the second one was due to rigorous temperature conditions; 3) in the Iberian electricity markets (Spain, Portugal), the price peak occurred in early March 2022; 4) separately Bulgaria stands out, as it passed 2 price peaks in December 2021 and August 2022.
Even more significant uncertainty is associated with the ending point of the energy crisis, which was linked to the degree of adaptation to new regulations in the electricity and gas markets of each EU member state: 1) in 20 bidding zones, the exit from the energy crisis occurred at the end of December 2022.This period was associated with the adoption of the market correction mechanism by the European Commission; 2) the earliest exit from the energy crisis was in the Iberian electricity markets, which occurred in the first half of December 2022; 3) in some Nordic bidding zones (NO3, NO4, SE1, SE2, SE3), the energy crisis was associated with short-term price spikes in November-December 2022, after which prices returned to the normal range; 4) in 4 electricity markets of Southeastern Europe (Czech Republic, Greece, Italy, and Slovakia), the energy crisis ended in mid-February 2023; 5) only in Poland the energy crisis in the electricity market can be considered incomplete as of 01 September 2023.
Thus, the energy crisis in the electricity markets of the EU member states started and evolved in different ways, depending on the structure of each electric power sector, the degree of integration of a particular bidding zone with neighbouring electricity markets, and the measures taken at the national level to adapt to the energy crisis.However, general trends can be identified that are common to most EU member states: (1) the onset of the energy crisis was in mid-September 2021; (2) the price peak was in late August 2022; and (3) the end of the energy crisis was in late December 2022.

Clustering EU electricity markets by the evolution of the energy crisis
As the occurrence and evolution of the energy crisis were found to be different between EU electricity markets, it is needed to identify homogeneous groups in which the energy crisis had similar trends.For this purpose, hierarchical and agglomerative clustering were performed according to 4 parameters: the starting point of the energy crisis, duration of it, the maximum price reached and price range.The results of clustering are presented in Fig. 2 and Table 4.As can be seen from Fig. 2 and Table 4, all electricity markets in EU countries are divided into 9 clusters: The 1st cluster represents the Polish electricity market, which had a late start date of the energy crisis, a moderately high increase in electricity prices, but the energy crisis is not over yet, which makes this market different from the others; The 2nd cluster consists of the Iberian electricity markets along with the SE4 bidding zone.These markets were characterised by the early onset of the energy crisis, its long duration, moderately high price range, but also early end; The 3rd cluster is formed by the adjacent markets of Finland, Estonia and the SE3 bidding zone, which were characterised by a late start of the energy crisis and therefore a short duration, but high electricity price range; The 4th cluster separately represents the Bulgarian electricity market, where the energy crisis started 2.5-3 months earlier compared to the other ones, and had the longest duration, ending, as in most EU countries, at the end of 2022.The growth of electricity prices was moderately high; The 5th cluster was formed by the selected bidding zones of Norway and the 2 Baltic countries, like the 3rd cluster, they were characterised by a later start of the energy crisis, but with a comparatively longer duration and higher electricity price range; The 6th cluster was formed around the DELU bidding zone as the largest in the EU, and also included electricity markets of Belgium, Denmark, the Netherlands and Greece.In terms of the duration of the energy crisis, these markets were on the 3rd place among the EU countries, characterised by high growth and price hikes; The 7th cluster represents the electricity markets of Southeastern Europe, and were characterised by a lag at the beginning of the crisis and a delayed exit from it.So the duration of the energy crisis was the longest among others, except Bulgaria.Also, the highest prices during the energy crisis were recorded in this cluster; The 8th cluster was unilaterally formed by the NO4 bidding zone, in which the energy crisis lasted only 16 days and had the lowest increase in electricity prices; The 9th cluster was formed by separate bidding zones of Nordic countries (NO3, SE1, SE2), which had the shortest duration of the energy crisis, only 12 days, but during which the electricity prices jumped almost twice.
Thus, the clustering of the EU electricity markets according to the evolution of the energy crisis allowed us to determine that: (i) the energy crisis almost passed over the individual Nordic bidding zones (8th and 9th clusters); (ii) the Nordic and Iberian markets (2nd and 3rd clusters) were the most resilient to the energy crisis; (iii) whereas the Central and Southeastern Europe markets (7th and 8th clusters) were the most vulnerable; (iv) the electricity markets of Bulgaria (4th cluster) and Poland (1st cluster) had individual trends during the energy crisis.

Conclusions
This study proposes a methodology to reveal the duration and phases of energy crisis in electricity markets, which includes the following steps: determination of abnormal electricity prices using descriptive statistics tools; singling out the trend of electricity prices using additive explanatory data analysis; k-means clustering and distribution of electricity prices by phases of the energy crisis; setting support and resistance lines based on the minimum-maximum method.Approval of the methodological approach was carried out on the example of the largest electricity market in the EU -DELU bidding zone.
Studies of the EU electricity markets allowed to determine their differentiation by the starting and ending points, duration, phase change and price range of the energy crisis.

Figure 1 .
Figure 1.Identification of the energy crisis in the DELU bidding zone a Source: constructed based on data from ENTSO-E transparency Platform [37] and developed Python code

Figure 2 .
Figure 2. Dendrogram of the distribution of EU electricity markets according to the evolution of the energy crisis a Source: own calculations

Figure A1 .
Figure A1.Identification of the energy crisis in the EU electricity marketsa Source: constructed based on data from ENTSO-E transparency Platform[37] and developed Python code

Table 1 .
Descriptive statistics of electricity prices in the DELU bidding zone in January 2021 -August 2023 a .

Table 3 .
Discovering the energy crisis in the electricity markets of EU member statesa a,b .

Table 4 .
Results of clustering EU electricity markets by the evolution of the energy crisis a .