A deep learning based classification of atmospheric circulation types over Europe: projection of future changes in a CMIP6 large ensemble

We train our deep learning classifier on historic examples of HB circulation types for the period from 1900 to 1980. The supervised training process is based on two data components. First, the catalog of Grosswetterlagen over Europe by Hess & Brezowsky (Werner and Gerstengarbe 2010) contains a list of daily class affiliations for the 29 HB circulation types since 1900 derived from a manual classification of observed atmospheric pressure constellations. We use the catalog's class affiliations as labels for the training of our deep neural network. Table 1 lists the 29 circulation patterns with their acronyms and full names. The second data component is the ERA-20C reanalysis by the European Centre for Medium-Range Weather Forecasts (Poli et al 2016) covering the period from 1900 to 2010. This data contains the spatial atmospheric pressure patterns that match the labels of the catalog and are interpreted as images according to their pixelwise structure. We use the variables slp and z500 in a 5^∘ spatial resolution over a domain covering Europe and parts of the North Atlantic (30^∘ N–75^∘ N, −65^∘ O–45^∘ O) based on Werner and Gerstengarbe (2010). Due to an implausible sudden discontinuity of the labels of the catalog that starts around the mid-1980s with an artificial increase in circulation type persistence (Kučerová et al 2017), the period from the year 1980 onward is excluded and only the consistent data from 1900 to 1980 is used for training. The training database contains 29 585 training examples of daily, historic HB circulation types. Figure 1 illustrates the typical air pressure constellations for each of the 29 classes for slp and z500.

Zoom In Zoom Out Reset image size

Our classification approach builds upon the image-like structure of the circulation patterns and uses a convolutional neural network. Its architecture is an adaptation of the model provided by Liu et al (2016) in the context of weather pattern detection and consists of two convolutional layers, a dropout layer and two-fully connected layers. In the convolutions, we use two individual channels for the climate parameters (slp and z500). Based on the original definition by Hess and Brezowsky (1952), the circulation types have to last for at least three days. This is why we apply transition smoothing as a post-processing step and smooth out class predictions that last for less than three days (details in the appendix; section 'Transition smoothing'). The model is trained using Adam optimization (Kingma and Ba 2014) with a batch size of 128, for 35 epochs and early stopping with a patience of six epochs. Hyperparameter tuning for learning rate and dropout rate is performed using Bayesian optimization (Snoek et al 2012). The performance of the model is evaluated using the overall accuracy and the macro F1-score (see appendix: equations (A.1)–(A.3)), which takes the average of the class-specific F1-scores and has a value range from 0 to 1 (Opitz and Burst 2021). To obtain reliable and robust performance estimates, we apply nested cross-validation (Cawley and Talbot 2010) with an inner for-loop for model tuning and an outer for-loop to split off independent test sets. We use folds that contain ten years each, i.e. eight outer folds (test sets) and seven inner folds (model tuning). For each inner fold and its best hyperparameter set, we train five networks to account for different random weight initializations. The performance metrics (e.g. overall accuracy, F-scores) quantifying the performance of the deep learning classifier are derived by taking the average over the eight outer test sets and five networks. This results in robust performance metrics derived from balanced and independent test sets. To account for the time series nature of circulation patterns, training examples from the same year are required to be in the same cross-validation fold.

To derive the final weights of a trained deep neural network that can be used for applications on new data (e.g. the SMHI-LENS), all available training examples from 1900 to 1980 are used for training without splitting off test sets (Hastie et al 2009). Model tuning (the inner loop of the nested cross-validation) is applied again to find the best hyperparameter configuration before training with all data.

Due to their complexity, deep neural network training and their predictions are subject to uncertainty. In order to quantify the uncertainty of our deep learning classifier, we use a deep ensemble (Lakshminarayanan et al 2017) by generating 30 networks based on different random weight initializations while all other settings (e.g. hyperparamter configurations) are kept stable. Using this approach, we can quantify the variance of predictions and generate more robust class affiliations by applying all 30 networks to the data and calculating a weighted average prediction (Krogh and Vedelsby 1994). The weighted average considers the trust in each of the 30 networks as quantified by the F1-score. Instead of applying only a single final model, we apply the deep ensemble of 30 networks to new data.

The deep ensemble introduced in section 2.3 is furthermore applied to the climate ensemble SMHI-LENS (Wyser et al 2021). SMHI-LENS is a SMILE of the Swedish Meteorological and Hydrological Institute (SMHI), with the EC-Earth model (version 3.3.1; Döscher et al 2022) and 50 members. The SMHI-LENS follows the protocol of the Coupled Model Intercomparison Project phase 6 (CMIP6). We chose the SMHI-LENS for its high number of members and the high performance of the EC-Earth3 model in reproducing daily sea-level pressure circulations types. Cannon (2020) compared 15 general circulation models with two reanalysis data sets. The EC-Earth3 was found to be one of the best performing CMIP6 models in terms of reproducing frequency and persistence of circulation types under the consideration of internal variability, especially over Europe (Cannon 2020). The SMHI-LENS is available for the period from 1970 to 2100 for four different scenarios with a 0.7^∘ spatial resolution. It uses the macro initialization method for the generation of its ensemble members. We use the high-emission climate scenario SSP37.0 and a daily resolution. The data is clipped to the Europe-North-Atlantic domain (see section 2.1) and regridded to the 5^∘ grid used during training of the deep learning classifier by means of bilinear interpolation. Frequencies of occurrence of circulation patterns are compared for two 30 year periods, a far future horizon from 2071 to 2100, and the reference period from 1991 to 2020. The signal-to-noise ratio (S/N-ratio) and its significance is calculated according to Aalbers et al (2018) using a two-sided t-test. The S/N-ratio states, if the forced response (ensemble averaged frequency change) exceeds the noise (standard deviation of the ensemble). As we simultaneously conduct hypothesis tests for all 29 circulation types, adjustments for multiple testing are needed to reduce the risk of incorrectly rejecting null hypotheses. We apply the method of Benjamini and Hochberg (1995) to control the false discovery rate, i.e. the proportion of incorrectly significant findings among all significant findings, for the chosen alpha level of 0.05.

To evaluate the performance of our method, the daily class affiliations of the original HB circulation type catalog (Werner and Gerstengarbe 2010) are compared to the class predictions of the deep learning classifier. On the outer folds of the nested cross-validation, we obtain a macro F1-score of 39.3 and an overall accuracy of 41.1%. The class-specific F1-scores are given in the second column of table 2.

Table 2 further shows the performance measures of the method by James (2006b). While the class-specific F1-scores for our deep learning classifier are derived from independent test sets during nested cross-validation based on ERA-20C reanalysis data for the period 1900–1980, the class-specific F1-scores for the method by James (2006b) are based on ERA-40 reanalysis data (Uppala et al 2005) for the time period from September 1957 to August 2002. While a direct comparison between the two approaches is thus not exact, contrasting both approaches is still valid under the assumption that both data sets are representative for the underlying distribution and class ratios. Given the length of both observation periods, this seems to be a reasonable assumption. In addition, our nested cross-validation approach can be considered robust without the risk of drawing an overly optimistic comparison in favor of our method. As table 2 shows, the deep learning classifier outperforms the method by James (2006b) in 20 of the 29 classes. The overall accuracy of our deep learning method is 41.1% (macro F1-score: 38.3) and 39.1% (macro F1-score: 36.28) for James (2006b). For the circulation patterns WS, NEA, SEA and SZ, the performance of the deep learning classifier is more than 10% higher, while the approach by James (2006b) works especially well for TRM and HNFA. The confusion matrix showing the average classifications of our deep learning classifier on the test sets during cross-validation is given in table A1 in the appendix. Most of the misclassifications occur between pairs of anticyclonic- and cyclonic circulation.

Our deep learning classifications are compared to the HB circulation type catalog in respect to the frequency distribution of the classes (see appendix, figure A2(a)). The network reproduces the relative order of the classes well, but clearly underestimates the class WZ. Classes HM and BM are also underestimated by the network, while it overestimates class WS. The climate ensemble SMHI-LENS reproduces the circulation types well (see appendix, figure A2(b)). Except for BM, for which the climate model overestimates the frequency, all boxplots cover or intersect with the frequencies in ERA-20C reanalysis data.

Figure 2 evaluates the 'synoptic performance' (Verdecchia et al 1996) of our deep learning classifier for four selected cases. Figure A1 in the appendix shows all 29 circulation types. The signature plots in figures 2 and A1 are derived by taking the average field of a certain class and subtracting the average field of all other classes from it. This shows which synoptic characteristics distinguish a single class from the other classes. Signature plots are given for four different cases: labels (column 1), predictions (column 2), false positives (column 3) and false negatives (column 4). If columns 1 and 2 are very similar, the average signature of the deep learning prediction agrees well with the average signature of the labels as derived from the original HB catalog. The four selected cases in figure 2 show two positive (green) and two negative (red) examples for the performance of our deep learning classifier. Signature plots allow a visual comparison of the spatial patterns of the circulation types. This reveals for example if the intensity of low pressure system is on average weaker in the network predictions compared to the labels (as it is the case for TB) or if the spatial extent of a synoptic feature is overestimated (like for TRM). A further noteworthy insight is given if there is good agreement between column 1 and 3 (as it is the case for HFZ), which can indicate a misclassification in the catalog. In this case, our deep learning method might correctly classify the situation, while the catalog labels disagree. Additionally, the difference between the signature plots is quantified using the root-mean-square error (RMSE) as metric (see equation (A.4) and table A2 in the appendix; Müller et al 2020). In case of a perfect match between two spatial patterns, the RMSE has a value of zero. The four examples in figure 2 are chosen according to the minimum and maximum RMSE values for false positives and false negatives.

Zoom In Zoom Out Reset image size

The RMSE when comparing the signature plot of the predictions (column 2) with the signature plot of the labels (column 1) has on average over all 29 circulation types a value of 0.89 (see table A2 in the appendix). For the false positives, the RMSE is slightly higher with on average 1.09. This matches to the visual comparison of the signature plots: apart from slight differences for some classes, the spatial patterns in column 3 agree generally well with column 1 (see figure A1). Exceptions for which the false positives differ the most from the signature plot of the labels are: TB, for which the extent of the low pressure system is too small (RMSE = 1.83), SA, whose low pressure system is too weak (RMSE = 1.69) and HNA and HNZ, for which in both cases the low pressure system in the South is too strong (RMSE = 1.41 for both). In other cases with low RMSE values and good agreement of the spatial patterns, the signature plots are an indication to question the choice in the subjective catalog. For example for HFZ (RMSE = 0.73), NEA (RMSE = 0.76) and WA (RMSE = 0.78). Although the deep learning classifier only considers the information provided in the data and subjective reasons for the label classification are not available, a certain level of arbitrariness in the catalog has already been recognized before by James (2006b). Column 4 reveals the false negatives of the deep learning classifier. Here, the average RMSE value over all 29 circulation types is 1.28. In some cases (e.g. SZ, WS, HNFA, TB and HFZ), the false negatives clearly differ from the signature patterns of the labels in column 1. In these cases, the catalog labels may be questioned and predictions of our approach seem plausible—at least based on the objective slp and z500 data.

Figure 3 depicts the uncertainty obtained by our deep ensemble (30 members) compared to the internal climate variability (50 members) by plotting stacked barplots of the percentages of the total uncertainty (50 climate model members times 30 deep ensemble members) with the respective attribution to these two sources. The network uncertainty range lies at 11%–33% for the entire year. It is larger in winter and smaller in summer. Note that for the deep learning part this does not take the variability of hyperparameter tuning into account. With regard to typical climate modeling uncertainties (Hawkins and Sutton 2011), the influence of climate model choice and scenario choice is not considered.

Zoom In Zoom Out Reset image size

We apply the weighted deep ensemble to the SMHI-LENS with its 50 members to quantify the spread of internal variability for future frequency changes of the 29 circulation patterns between 2071–2100 and the reference period (1991–2020). Figure 4 shows absolute frequency changes and the spread of internal variability for all circulation types for the entire year as well as the winter and summer half-year illustrated by boxplots. Relative frequency changes are illustrated in figure A3. Significant changes in terms of the S/N-ratio are indicated with bold class names. Table A3 shows the complete list of S/N-ratio values for the absolute frequency changes. For most circulation types, the boxplots intersect with the horizontal line at zero and the members disagree in the sign of the trend. Overall, absolute changes are small and lie within a range of ±5 days for most circulation types. This finding is in line with Huguenin et al (2020), who find small changes of ±4 days per season in a multi-model ensemble for ten groups of circulation patterns that are based on the 29 HB circulation types. Note that for the circulation types TM-TRW (in the presented order), which have small absolute frequencies, changes of ±5 days can still mean high relative changes of around ±50% (see appendix, figure A3). For some circulation types, single members outside the interquartile range show relative changes of $\gt$50%, especially in the winter half-year. Different to the studies by James (2006a) and Ringer et al (2006), who analyzed all 29 circulation types in the climate model HadGEM1 and have not found significant frequency changes, our analysis of a SMILE allows to identify significant frequency changes due to climate change despite the high spread of internal variability.

Zoom In Zoom Out Reset image size

In figure 4(d)–(f), we plot the class-specific F1-scores of our deep learning classifier and their range throughout the deep ensemble. This allows to take into account the quality of predictions for each class. Reliable statements can be made for the circulation patterns WA, WZ, WS, HM, HNA, HB, HFA, SA, and TB throughout the entire year. The clearest absolute climate trend is found for the anticyclonic westerly circulation (WA), which shows an increasing trend for the entire year (and the summer half-year) with a median of 6.6 days per year (summer: 5.4 days per year) and a S/N-ratio of 1.5 (summer: 1.7). For WA, the climate change signal clearly exceeds the noise of internal variability. The increasing winter trends of HFA and TB are also significant, as well as decreasing summer trends of WS, HB and SA and the increasing summer trend for HM.

In general, we find a decreasing trend for south-easterly circulations (SEA and SEZ) in both summer and winter (trends are significant except for SEA in winter), although their reliability based on F1-scores fluctuates seasonally. For winter, this goes along with the findings by Herrera-Lormendez et al (2021), who have detected a decreasing trend for south-easterly circulations from the Jenkinson–Collison classification using four members of EC-Earth3 under SSP58.5. The classification by Jenkinson and Collison (1977) is an automated version of the subjective Lamb catalog developed for the British Isles. Herrera-Lormendez et al (2021) applied this classification to Europe and distinguished 11 circulation types. Our results also support the findings of Herrera-Lormendez et al (2021) for the increasing summer trend of north-easterly circulations (in our case significant for NEA) and the decreasing summer trend for Northerlies (in our case significant for NZ and HB).

Our results make clear that the spread of internal variability is tremendous and it is difficult to derive systematic changes of circulation patterns grouped by their wind directions. Despite the high internal variability, the results of the S/N-ratio are very clear, showing a significant change in 69% of the classes for the total year, 34% for the winter and 69% for the summer half-year.

Our classifications must adhere to the definition that a circulation type lasts for at least three days. A transition-smoothing step ensures that this rule is respected by post-processing the time series of the network classifications. Firstly, circulation types that last for less than three days are identified. Next, these transitions are tested for neighborhood consistency and transition membership. Neighborhood consistency describes the situation if the same circulation type occurs before and after the transition. The transition days are then smoothed by assigning this type to it. In case of transition membership different circulation types occur before and after the transition. Here, the transition days obtain the class affiliation of the circulation type before or after, depending on which class has a higher predicted probability.

The Macro F1-score (F₁) is calculated as the arithmetic mean of the class-specific F1-scores ($F1_{x}$) of all classes (see (A.3); Opitz and Burst 2021). n describes the number of classes (in our case: 29). The F1-Score is based on precision and recall. Precision (P; see (A.1); adjusted based on Lewis et al 1996) is calculated based on items correctly assigned to the specific class (true positives (TP)), and non-class members put into the class (false positives (FP)). Recall (R; see (A.2; adjusted based on Lewis et al 1996) on the other hand considers class members not assigned to this class by the deep learning classifier (false negatives (FN); Lewis et al 1996).

$$\begin{equation} \mathrm P = \frac {\mathrm{TP}}{\mathrm{(TP + FP)}} \end{equation} \tag{ A.1 }$$

$$\begin{equation} \mathrm R = \frac{\mathrm{TP}}{\mathrm{(TP + FN)}} \end{equation} \tag{ A.2 }$$

$$\begin{equation} F_{1} = \frac{1}{n} \sum_{x} F1_{x} = \frac{1}{n} \sum_{x} \frac{2P_{x}R_{x}}{P_{x} + R_{x}}. \end{equation} \tag{ A.3 }$$

Equation for the calculation of the RMSE with I being the predicted image (in our case: signature plot of the deep learning classifier) and K being the reference image (in our case: signature plot of the labels). M are number of rows and N the number of columns of the pictures to compare. The RMSE thus compares the pixel-wise values of two images. A value of zero indicates a perfect match. Equation based on Müller et al (2020).

$$\begin{equation} \mathrm{RMSE} = \sqrt{\frac{1}{M\times N} \sum_{i = 0, j = 0}^{M-1, N-1} [I(i,j) - K(i,j)]^2}. \end{equation} \tag{ A.4 }$$

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Table A1. Confusion matrix of our proposed smoothed approach, averaged over the test sets of nested cross-validation. Correctly classified classes are highlighted in bold.

Labels
		WA	WZ	WS	WW	SWA	SWZ	NWA	NWZ	HM	BM	TM	NA	NZ	HNA	HNZ	HB	TRM	NEA	NEZ	HFA	HFZ	HNFA	HNFZ	SEA	SEZ	SA	SZ	TB	TRW	$\sum$	Precision
Outputs	WA	102	62	1	3	4	2	10	8	22	26	0	1	1	0	0	1	3	1	1	0	0	0	0	0	0	0	0	1	3	253	0,40
	WZ	13	195	12	5	2	6	1	11	4	2	0	1	2	1	0	0	6	1	0	0	0	0	0	0	0	0	0	2	5	272	0,72
	WS	1	51	67	3	0	8	0	4	1	1	4	0	0	1	3	0	6	0	0	0	0	0	1	0	3	0	1	6	5	170	0,40
	WW	4	24	3	42	2	4	1	3	6	6	1	0	0	0	0	0	3	2	3	2	1	0	0	2	2	3	2	2	6	125	0,33
	SWA	14	18	1	4	34	11	0	0	25	7	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	4	0	1	2	125	0,27
	SWZ	4	36	9	5	8	30	0	1	4	1	0	0	0	1	0	0	2	0	0	0	0	0	0	0	0	2	1	4	5	115	0,26
	NWA	14	8	0	1	0	0	58	18	12	20	0	1	4	2	1	12	3	2	2	0	0	0	0	0	0	0	0	0	1	159	0,37
	NWZ	10	56	2	2	0	0	15	67	2	4	2	1	7	1	0	1	21	1	1	0	1	0	0	0	1	0	0	1	3	198	0,34
	HM	8	4	1	1	4	1	5	1	142	23	0	1	0	2	0	4	0	2	1	7	1	1	0	2	0	3	0	0	1	216	0,66
	BM	15	5	0	3	1	0	7	2	25	104	0	0	1	1	0	2	3	3	3	3	0	1	0	1	0	2	0	0	2	187	0,56
	TM	0	8	4	1	0	0	0	3	0	1	34	1	6	0	4	0	10	1	3	1	1	1	5	0	1	0	1	4	6	95	0,36
	NA	2	4	0	0	0	0	3	1	4	1	0	12	7	12	3	3	1	2	2	1	0	1	1	0	0	0	0	0	0	59	0,20
	NZ	2	7	0	0	0	0	6	12	1	2	6	4	52	4	5	3	15	1	2	0	0	0	1	0	0	0	0	0	1	125	0,42
	HNA	1	4	1	0	0	0	2	0	11	3	1	6	3	54	5	10	1	1	1	2	0	3	2	2	0	0	0	1	0	116	0,47
	HNZ	0	5	3	0	0	1	1	0	1	1	7	2	7	10	17	1	3	0	1	0	0	1	7	1	0	0	0	1	2	72	0,23
	HB	1	1	0	0	0	0	19	4	13	11	0	2	2	8	0	65	2	5	2	2	0	1	0	0	0	0	0	0	0	141	0,46
	TRM	3	14	2	1	0	1	1	17	1	4	6	0	9	1	0	0	35	1	2	0	0	0	0	0	1	0	0	1	8	109	0,32
	NEA	1	2	0	2	0	0	4	1	10	10	1	1	1	3	0	5	1	43	13	10	2	2	1	1	0	1	0	0	1	114	0,38
	NEZ	1	2	1	1	0	0	5	1	1	5	4	0	2	1	1	2	5	10	26	2	2	1	2	0	1	0	0	0	1	79	0,33
	HFA	1	1	0	1	1	0	0	0	21	5	0	0	0	3	0	1	1	9	2	63	5	4	1	9	2	4	0	0	1	135	0,47
	HFZ	0	1	1	2	0	0	0	1	1	1	3	0	0	0	0	0	1	3	8	12	12	1	4	2	5	1	0	1	1	61	0,20
	HNFA	0	1	0	0	0	0	0	0	2	3	1	1	2	11	1	2	1	3	3	12	1	21	8	3	1	0	0	1	1	79	0,27
	HNFZ	0	3	1	0	0	0	0	0	1	1	7	0	2	4	7	0	1	0	2	4	2	6	24	3	2	0	0	1	2	74	0,32
	SEA	0	1	1	1	1	0	0	0	7	2	0	0	0	3	0	0	0	0	1	12	2	2	3	32	9	8	1	1	1	87	0,36
	SEZ	0	2	3	4	0	0	0	1	0	1	2	0	0	0	0	0	2	1	2	1	4	0	3	6	24	2	3	2	3	68	0,35
	SA	1	2	0	3	4	3	0	0	16	5	0	0	0	1	0	0	0	0	0	6	0	0	0	7	1	34	4	1	3	94	0,36
	SZ	0	1	5	4	1	2	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	3	4	8	19	7	5	63	0,30
	TB	2	20	7	4	2	5	0	3	3	1	3	0	0	0	1	0	3	0	0	1	0	0	1	1	2	2	2	44	12	120	0,37
	TRW	4	18	3	3	1	3	1	4	2	4	2	0	1	0	0	0	12	0	1	0	1	0	0	1	1	1	1	7	32	104	0,31
	$\sum$	205	557	126	97	68	81	140	164	340	255	88	35	111	123	51	115	140	94	79	143	36	49	67	78	59	76	36	89	116	3616	—
	Recall	0,50	0,35	0,53	0,43	0,50	0,37	0,42	0,41	0,42	0,41	0,39	0,34	0,47	0,44	0,33	0,57	0,25	0,46	0,33	0,44	0,33	0,44	0,36	0,40	0,41	0,45	0,53	0,49	0,28	0,41	—

Table A2. Values of the root-mean-square error (RMSE) when comparing the composite plot of the predictions, the false positives and the false negatives with the composite plot of the labels for the 29 circulation types. RMSE values averaged over all 29 circulation types are printed in bold.

RMSE	WA	WZ	WS	WW	SWA	SWZ	NWA	NWZ	HM	BM	TM	NA	NZ	HNA	HNZ
Predictions	0.93	0.82	1.0	0.88	0.77	0.82	0.73	1.06	0.78	1.04	0.91	0.87	0.39	0.85	1.29
False positives	0.78	1.11	1.21	1.08	1.23	1.0	0.83	1.06	0.80	0.84	0.92	0.92	0.89	1.41	1.41
False negatives	1.25	0.84	2.10	1.42	1.62	1.24	0.83	1.36	0.87	1.12	1.03	1.50	0.80	1.04	1.3
RMSE	HB	TRM	NEA	NEZ	HFA	HFZ	HNFA	HNFZ	SEA	SEZ	SA	SZ	TB	TRW	$\varnothing$
Predictions	0.66	1.23	0.58	0.89	0.89	0.75	0.74	0.66	0.99	0.91	1.11	0.75	1.41	1.14	0.89
False positives	1.11	1.29	0.76	0.96	1.41	0.73	0.84	0.94	0.98	1.22	1.69	1.23	1.83	1.27	1.09
False negatives	1.57	0.58	1.0	0.99	1.25	1.70	1.99	1.0	1.31	1.58	1.32	2.31	1.72	0.61	1.28