Assessment of CMIP5 climate models and projected temperature changes over Northern Eurasia

Observations of monthly SAT over Northern Eurasia are obtained from the Climate Research Unit data (CRU TS 3.1) (Mitchell and Jones 2005, Harris et al 2013) (available at http://badc.nerc.ac.uk/data/cru/). The horizontal resolution of the dataset is 0.5° × 0.5°, and the time period in this research is from 1901 to 2005.

24 GCMs outputs obtained from the CMIP5 data archive (http://cmip-pcmdi.llnl.gov/cmip5/index.html) are listed in table 1. The monthly data includes the historical (1901–2005) and the future (2006–2099) periods. Here we focus on the SAT projection under three scenarios (i.e., RCP 2.6, RCP 4.5 and RCP 8.5). The three RCPs represent 'low' (RCP 2.6), 'medium' (RCP 4.5) and 'high' (RCP 8.5) scenarios featured by the radiative forcings of 2.6, 4.5 and 8.5 W m⁻² by 2100, respectively. The CO₂-equivalent concentrations in the year 2100 for RCP 2.6, RCP 4.5 and RCP 8.5 are 421 ppm, 538 ppm and 936 ppm, respectively (Meinshausen et al 2011). For comparison purpose, all GCM outputs are regridded to the same resolution as that of the observed data (0.5° × 0.5° grid).

Table 1.  List of the global climate models in IPCC-AR5.

GCM	Model	Resolution	Source
1	BCC-CSM 1.1	64 × 128	Beijing Climate Center, China Meteorological Administration, China
2	BCC-CSM1.1(m)	160 × 320
3	BNU-ESM	64 × 128	Beijing Normal ${\rm{University}}$, China
4	CanESM2	64 × 128	Canadian Centre for Climate Modelling and Analysis, Canada
5	CCSM4	192 × 288	National Center for Atmospheric Research (NCAR), USA
6	CNRM-CM5	128 × 256	Centre National de Recherches Meteorologiques, France
7	CSIRO-Mk3.6.0	96 × 192	Australian Commonwealth Scientific and Industrial Research Organisation
8	FGOALS-g2	108 × 128	Institute of Atmospheric Physics, Chinese Academy of Sciences, China
9	FIO-ESM	64 × 128	The First Institute of Oceanography, SOA, China
10	GFDL-CM3	90 × 144	Geophysical Fluid Dynamics ${\rm{Laboratory}}$, USA
11	GFDL-ESM2G	90 × 144
12	GISS-E2-H	90 × 144	Goddard Institute for Space Studies (NASA), USA
13	GISS-E2-R	90 × 144
14	HadGEM2-ES	145 × 192	Met Office Hadley ${\rm{Centre}}$, UK
15	IPSL-CM5A-LR	96 × 96	Institut Pierre-Simon Laplace, France
16	IPSL-CM5A-MR	143 × 144
17	MIROC5	128 × 256	Atmosphere and Ocean Research Institute, University of Tokyo, Japan
18	MIROC-ESM	64 × 128	Japan Agency for Marine-Earth Science and Technology, Atmosphere and Ocean Research Institute (The University of Tokyo), Japan
19	MIROC-ESM-CHEM	64 × 128
20	MPI-ESM-LR	96 × 192	Max Planck Institute for Meteorology (MPI-M), Germany
21	MPI-ESM-MR	96 × 192
22	MRI-CGCM3	160 × 320	Meteorological Research Institute, Japan
23	NorESM1-M	96 × 144	Norwegian Climate Centre, Norway
24	NorESM-ME	96 × 144

Because single models are overconfident (Weigel et al 2008) and multi-model ensembles contain information from all participating models (Pincus et al 2008), it is generally believed that multi-model ensembles are superior to single models (IPCC 2001, Duan and Phillips 2010, Miao et al 2013). In this study, three types of popular ensemble methods are used. They are simple model averaging (SMA), reliability ensemble averaging (REA) and Bayesian model averaging (BMA) techniques.

SMA is the simplest multi-model ensemble technique. Each model has the same weight (w_k  = 1/K, where K is the number of models) in the multi-model forecast. When using the SMA, any knowledge about the performance of the model is neglected (Casanova and Ahrens 2009).

The REA is a weighted average of ensemble members method based on the 'reliability' of each model (Giorgi and Mearns 2002). The reliability factor of the kth model (R_k ) takes into account of the ability of each ensemble member to simulate the observed climate (R_B ) and its degree of convergence in the projected climate change with respect to the other models in the ensemble (R_D ).

$$\begin{eqnarray}&&\begin{array}{*{35}{l}} {{R}_{k}}={{\left[ {{\left( {{R}_{B,k}} \right)}^{m}}\times {{\left( {{R}_{D,k}} \right)}^{n}} \right]}^{[1/(m\times n)]}} \\ \quad ={{\left[ {{\left( \frac{\epsilon }{\left| {{B}_{k}} \right|} \right)}^{m}}\times {{\left( \frac{\epsilon }{{{D}_{k}}} \right)}^{n}} \right]}^{[1/(m\times n)]}} \\ \end{array}\end{eqnarray} \tag{ 1 }$$

$$\begin{eqnarray}&&{{w}_{k}}=\frac{{{R}_{k}}}{\mathop{\sum }\nolimits R}\end{eqnarray} \tag{ 2 }$$

where R_B is a factor that is inverse proportional to the absolute bias (B) in simulating the present-day variable and R_D is a factor that measures the model reliability in terms of the distance (D) of the change calculated by a given model from the REA average change. The parameters m and n are the weights of the model performance criterion (R_B ) and the model convergence criterion (R_D ), respectively, which are typically equal to 1. The parameter in equation (1) is the natural variability of the climatic variable. More details of the REA process are provided in Giorgi and Mearns (2002) and Mote and Salathé (2010).

The BMA generates a probability density function (PDF), which is a weighted average of the PDFs centered on the forecasts. The BMA weights reflect the relative contributions of the component models to the predictive skill over a training sample. The combined forecast PDF of a variable y is:

$$\begin{eqnarray}&&p\left( y|{{y}^{T}} \right)=\underset{k=1}{\overset{K}{\mathop \sum }}\,p\left( y|{{M}_{k}},{{y}^{T}} \right)p\left( {{M}_{k}}|{{y}^{T}} \right)\end{eqnarray} \tag{ 3 }$$

where $p\left( y\left| {{M}_{k}},{{y}^{T}} \right. \right)$ is the forecast pdf based on the model M_k alone, estimated from the training period of observations y^T; and K is the number of models combined. $p\left( {{M}_{k}}\left| {{y}^{T}} \right. \right)$ is the posterior probability of the model M_k corrected using the training data. This term is computed based on Bayes' theory:

$$\begin{eqnarray}&&p\left( {{M}_{k}}|{{y}^{T}} \right)=\frac{p\left( {{y}^{T}}|{{M}_{k}} \right)p\left( {{M}_{k}} \right)}{\underset{l=1}{\overset{k}{\mathop \sum }}\,p\left( {{y}^{T}}|{{M}_{l}} \right)p\left( {{M}_{l}} \right)}\end{eqnarray} \tag{ 4 }$$

The BMA weights are estimated using the maximum likelihood (Raftery et al 2005). To estimate given parameters, the likelihood function is the probability of the training data and is viewed as a function of the parameters (Yang et al 2011). The weights are chosen to maximize this function (i.e., the parameter values for which the observation data are most likely to have been observed). The algorithm used to calculate the BMA weights and variance is called the expectation maximization (EM) algorithm (Dempster et al 1977). More details of the BMA process are provided in Raftery (2005) and Duan and Phillips (2010).

For GCM performance assessment, the bias between observation and model simulations is compared. The SAT change during the historical period (1901–2005) and the projected future scenarios (2006–2099) are analyzed.

In order to evaluate the ensemble performance, the Taylor diagram technique is used. The Taylor diagram is quantified in terms of the correlation (R), the centered root-mean-square-error (RMSE) and the amplitude of the standard deviations (Std). The diagram provides a way of graphically summarizing how closely a pattern matches observations (Taylor 2001). Moreover, the uncertainty of the different multi-model ensembles is also compared. Here, we calculate the standard deviation of the changes, δ, defined by

$$\begin{eqnarray}&&\delta ={{\left[ \frac{\underset{k=1}{\overset{K}{\mathop \sum }}\,{{w}_{k}}{{({{T}_{k}}-En)}^{2}}}{\underset{k=1}{\overset{K}{\mathop \sum }}\,{{w}_{k}}} \right]}^{1/2}}\end{eqnarray} \tag{ 5 }$$

where (w_k ) is the weight of kth model generated by different multi-model ensemble techniques, T_k is the kth model output and En is the ensemble. The upper and lower uncertainty limits are thus defined as:

$$\begin{eqnarray}&&Upper=En+\delta \end{eqnarray} \tag{ 6 }$$

$$\begin{eqnarray}&&Upper=En-\delta \end{eqnarray} \tag{ 7 }$$

If the changes are distributed as a Gaussian PDF, the ±δ range implies a 68.3% confidence interval.

It is generally accepted that the agreement between models and observations currently is the only way to assign confidence into the quality of a model (Errasti 2011), and the better a model performance in reproducing present-day climate, the higher the reliability of the climate change simulation (Giorgi and Mearns 2002, Coquard et al 2004). Each GCM's weight can be obtained from the ensemble process during the period of 1901–2005. Applying with these weights, multi-model ensemble projections in temperature over the 21st century scenarios can be generated.

Figure 1 shows the bias in 24 CMIP5 climate models by comparing the observed and simulated data of the 105 year annual and seasonal mean temperatures. Most of the GCMs give reasonably accurate predictions of the mean temperature. Among the 24 GCMs, nine models underestimate the annual mean temperature, while the others overestimate. The maximum bias for SAT simulation comes from the FGOALS-g2 model, with a value of −4.31 °C. The BCC-CSM 1.1 and GISS-E2-R models perform the best, with a minimum bias of 0.10 °C. It should be noted that the 105 year mean of observed temperature is about −4.50 °C. It is also found that models with higher resolution do not always perform better than those with lower resolutions (such as the FGOALS-g2 model). For the seasonal SAT simulation, model biases in March–April–May (MAM) and June–July–August (JJA) are relatively small, while the bias in December–January–February (DJF) is high. Compared with the BCC-CSM 1.1 model, the GISS-E2-R model has the smaller bias in the seasonal SAT simulation.

Zoom In Zoom Out Reset image size

The SAT observation has increased at a rate of about 1.1 °C/100 yrs during 1901–2005 (figure 2). Among the 24 GCMs, 12 models overestimate the warming trend, and the others overestimate. However, the warming trend differences between the observation and the overestimating models are generally larger than that from the underestimating model. The simulated warming trends by the IPSL-CM5A-LR and MRI-CGCM3 models are closest to the observations, and the maximum warming trend (approximately 2.76 °C/100 yrs) is simulated by the BNU-ESM model. It is widely recognized that if the worst impacts of climate change are to be avoided then the average rise in the surface temperature of the Earth needs to be less than 2 °C. Hence, we project the time when the global land mean SAT (without Antarctica) reaches a 2 °C increase relative to 2000 under different RCPs, and compare with the corresponding SAT increase over Northern Eurasia at that time (table 2). Under the RCP 2.6 scenario, six GCMs (CanESM2, GFDL-CM3, HadGEM2-ES, MIROC5, MIROC-ESM and MIROC-ESM-CHEM) show the global SAT will rise by 2 °C in this century. And only one GCM model (GISS-E2-R) estimates the global SAT will not increase by 2 °C within this century under RCP 4.5. The projected time by the remaining GCMs is dispersed from 2034 to 2081. All GCMs affirm that the global SAT will rise by 2 °C in this century under the RCP 8.5, and most of the models forecast that the projected time will occur from the 2030s–2050s. Under different scenarios, most of GCMs (except BCC-CSM1.1 (m), CSIRO-Mk3.6.0, GISS-E2-R and MPI-ESM-LR) believe the warming rate over the Northern Eurasia is higher than the global warming average in this century. And some models (such as GFDL-CM3 in RCP 4.5 and MRI-CGCM3 in RCP 8.5) even show the warming in Northern Eurasia is more than twice faster than the global average.

Zoom In Zoom Out Reset image size

Figure 3 shows the performance of the annual SAT simulation over the Northern Eurasia. The annual mean SAT observation increases during 1901–2005, and the warming has accelerated since the mid-20th century. Compared with the CMIP3 model, the CMIP5 model improves slightly in the annual SAT simulation, showing as closer to the observation point in the Taylor diagram (figure 3(a)). It is also indicated that most of the GCMs can approximate the trend of SAT, but the accuracy of annual SAT simulation is relatively low. The correlation coefficient R between each GCM simulation and the annual observation ranges from 0.20 to 0.56 (figure 3(a)). The ensemble results show that the ensemble technique can improve the temporal SAT simulation when compared to a single GCM (figure 3(b)). For multi-model ensemble simulation, the performances of the SMA, REA and BMA are similar since their outputs nearly overlap in the Taylor diagram. Comparing the weights generated by BMA and REA techniques, it is found that the model weights are similar but not identical. This is an interesting phenomenon that SMA, REA and BMA generate very similar results, even though the ensemble members receive different weights through the three ensemble techniques.

Zoom In Zoom Out Reset image size

Because a GCM cannot accurately reflect the actual annual SAT change (figure 3(b)), here we focus on the decadal SAT simulation (figure 4). It shows that GCM can catch the trend of 10-year moving average SAT over Northern Eurasia. Compared with the annual scale, the correlation coefficients between the decadal SAT simulations and observation are increased, being primarily between 0.6 and 0.9; and the ensemble technique can improve the decal SAT simulation further (figure 4(a)). Similar to figure 3(b), the decadal SAT changes simulated by three kinds of ensemble mean methods are almost the same (figure 4(b)). Besides ensemble average, uncertainty is another important skill score. Hence, the uncertainty of the multi-model ensembles during the period of 1901–2005 is also compared. Figure 5 shows the ensemble uncertainty of the simulated results with a 10 year moving average. It is indicated that the uncertainty generated by the BMA is the smallest among the three multi-model ensemble results for simulating the annual and seasonal SAT. In addition, the results show that the uncertainty in DJF is the largest over Northern Eurasia.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Considering the smallest uncertainty, the BMA method is applied to project the SAT change in the 21st century under the three future emission scenarios (RCP 2.6, RCP 4.5, RCP 8.5) (figure 6). Only the decadal SAT is projected, due to the poor performance on the annual scale. The BMA simulations show that the SAT of Northern Eurasia will increase remarkably over the 21st century. On average, the SAT over Northern Eurasia will rise by 1.03 °C/100 yr, 3.11 °C/100 yr and 7.14 °C/100 yr for the RCP 2.6, RCP 4.5 and RCP 8.5 scenarios, respectively. Under the RCP 2.6 and RCP 4.5 scenarios, the greatest contribution to the decadal warming is from MAM, while DJF is the largest contributor under the RCP 8.5 scenario. The warming trend slows down or even declines after 2050 under RCP 2.6. For the uncertainty of the SAT projections, it is found that the uncertainty of SAT projections increases with time in the 21st century, and the uncertainty under RCP 8.5 is larger than that under RCP 2.6 and RCP 4.5.

Zoom In Zoom Out Reset image size

Figure 7 shows the projected annual mean SAT change for 2080–2099. The annual mean SAT in the last two decades of 21st century relative to 1986–2005 over Northern Eurasia will increase 1.92 °C, 3.25 °C and 6.40 °C under the RCP 2.6, RCP 4.5 and RCP 8.5 scenarios, respectively. It is found that the warming climate accelerates with increasing latitude. Grids with maximum SAT changes are concentrated in the Svalbard region of Norway under three scenarios, the corresponding changes are 3.61 °C, 5.57 °C and 10.24 °C under the RCP 2.6, RCP 4.5 and RCP 8.5, respectively.

Zoom In Zoom Out Reset image size