Climate sensitivity indices and their relation with projected temperature change in CMIP6 models

The values of ECS listed in table S1 are taken from Meehl et al (2020) and represent approximations made from simulations of abrupt quadrupling of the atmospheric CO₂ concentration. The method, initially suggested by Gregory et al (2004), is based on the radiative budget of the climate system, described as:

$$\begin{equation} N = F - \alpha \Delta T, \end{equation} \tag{ 1 }$$

where N is the TOA radiative imbalance, F is the imposed constant forcing, α is the climate feedback parameter and ΔT is the change in global mean surface air temperature (GMSAT) caused by the imposed forcing (compared to some reference temperature in a system with no imposed forcing). In this framework, the feedback parameter is assumed to be time-invariant, and a linear regression of N(t) against T(t) from model output then gives the values for α and F, and the ECS is given by 0.5F/α, where the factor 0.5 is applied because the abrupt-4xCO2 experiment is used rather than an experiment with doubled CO₂ amounts. Dividing the response to quadrupled CO₂ forcing by two introduces an error, as it assumes a linearity that has been found not to hold, especially for high sensitivities (Jonko et al 2013, Bloch-Johnson et al 2015, Tsutsui 2017, Rugenstein et al 2020). More important to note, however, is the assumption of a constant feedback parameter, which is invalidated by pattern effects, i.e. that the time evolving spatial patterns of surface warming trigger different combinations of regional feedbacks at different times, causing discrepancies between regression-based estimates of ECS depending on the time period used. Recent temperature evolution has for instance induced more damping cloud feedback than is expected as warming continues, and hence underestimates ECS (Armour et al 2013, Stevens et al 2016, Tsutsui 2017, Andrews et al 2018, Silvers et al 2018, Dong et al 2020, Gregory et al 2020). Following common practice (e.g. Grose et al 2018, Flynn and Mauritsen 2020, Meehl et al 2020, Smith et al 2020, Wang et al 2021), however, we take the value derived from regression of 150 years of an abrupt-4xCO2 simulation, divided by two, to represent ECS.

The TCR values are taken from Meehl et al (2020) and are diagnosed from experiments in which CO₂ concentrations are increased by 1% per year (1pctCO2). The TCR is calculated as the temperature difference between pre-industrial conditions and the mean of a 20-year period centred on the 70th year of the experiment; the time at which the CO₂ concentration has doubled (Flato et al 2013). Similarly, the transient temperature response at the time of quadrupling of CO₂ in 1pctCO2 simulations is used e.g. by Grose et al (2018) and Sanderson (2020), and included here for comparison, referred to as T140.

The method for investigating the relationship between model simulated temperature change and different sensitivity indices in different forcing scenarios follows that of Grose et al (2018).

A running mean was calculated for the temperature change from 1861 to 2100 for each model and each scenario, for global mean and for specified regions respectively. Each individual year was represented by the mean of the 20 years centred on the year in question. The data were extrapolated 10 years past the year 2100 to allow for averages to be calculated in the last 9 years of the simulations, using coefficients obtained through a linear regression of the last 20 years of the data set. A similar extrapolation was performed at the beginning of the time series. The running mean minimizes the impact of interannual variability of the climate system on the warming (Grose et al 2018). For each year a linear regression was then performed between the GMSAT change since 1861 and the ECS, TCR, and T140, respectively, using the data from all available models.

In addition to the analysis of the different sensitivity indices, the same method was used to examine if historical forcing in models is related to their simulated past and future temperature evolution. The forcings used were the 1850–2014 global mean effective radiative forcing (ERF) from aerosols ($F_{\textrm{aer}}$) and well-mixed greenhouse gases ($F_{\textrm{GHG}}$), and the total anthropogenic ERF ($F_{\textrm{ant}}$). These were calculated from 30-year time slice experiments with sea surface temperatures (SSTs) fixed to pre-industrial conditions, with present-day aerosols, greenhouse gases and total anthropogenic forcing, respectively, compared to a pre-industrial control simulation, as in Smith et al (2020). Hence, like the sensitivity metrics, these forcing quantifications are not time dependent. Table S1 lists the forcings for the CMIP6 models providing the necessary output (a subset of which are given in Smith et al 2020). As for the sensitivity metrics, correlations between the forcing measures and the GMSAT change are used to quantify and compare the skill of the different measures in explaining model spread in simulated past and future temperature change.

Globally gridded correlations between sensitivity indices and 1850–1869 to 2080–2099 temperature change under shared socioeconomic pathways (SSP5-8.5) were also calculated, and for this model data were interpolated to a common $1.5^{\circ} \times 1.5^{\circ}$ grid.

Grose et al (2018) studied the future temperature projections following the representative concentration pathways (RCPs) of CMIP5. The RCPs exemplify different scenarios of future warming, characterized by different pathways for the climate forcing. For example, RCP4.5 has a climate forcing of approximately 4.5 W m⁻² by year 2100 (van Vuuren et al 2011). In CMIP6 the RCPs are combined with so called shared SSPs to create more complex scenarios, taking societal difficulties in mitigation of and adaption to climate change into account (O'Neill et al 2014). The scenarios that are available from most modelling centres are SSP1-2.6 (denoting the combination of SSP1 with RCP2.6), SSP2-4.5 and SSP5-8.5. The availability of SSP runs in the CMIP6 models considered is indicated in table S1.

The investigation of the relation between forcing and sensitivity follows Kiehl (2007) and Forster et al (2013), who surveyed that relationship in previous generation models. Consistent with these studies, the forcing is here estimated directly from the historical simulations rather than from single-forcing times slice experiments as in section 2.2. Historical forcing estimates are based on equation (1), from which Kiehl (2007) argued that for a given energy imbalance and temperature change, such as those observed for the 20th century, there is an inverse relationship between forcing and ECS. The ECS is the temperature change after equilibrium has been reached following a doubling in CO₂, so at equilibrium, equation (1) becomes: $F_{2\times\textrm{CO}_{2}} = \alpha \Delta T_{2\times\textrm{CO}_{2}} = \alpha {\textrm{ECS}}),$ meaning that equation (1) can be generally rewritten as:

$$\begin{equation} F = N + \frac{F_{2\times\textrm{CO}_{2}} \Delta T}{{\textrm{ECS}}}. \end{equation} \tag{ 2 }$$

Using the method described by Forster et al (2013), the forcing in the historical simulation of each model was calculated through a two-step procedure based on equation (1). In the first step, the abrupt-4xCO2 experiment was used in the same way as in the calculation of the ECS, as described in section 2.1. N was regressed against T to obtain α, where N and T are both defined as differences from pre-industrial conditions. In the second step, it was assumed that α is constant in time and independent of forcing, and equation (1) was applied on the temperature change and difference in TOA radiative imbalance since 1850 from the historical simulation to obtain the total forcing $F_{\textrm{total}}$ at year 2003 (as the 2001–2005 average). Following Forster, the temperature and TOA radiation in the abrupt-4xCO2 and historical experiments were first corrected for TOA radiative imbalance in the pre-industrial control simulation (piControl, see figure S1) and for any drift, by fitting and subtracting a linear trend for each model in the abrupt-4xCO2 and historical experiments. Table S1 lists the adjusted ERF ($F_{\textrm{total}}$) for 2003 in the models where abrupt-4xCO2 simulations were made available for the calculation. Again, the assumption that α is constant is in fact not valid, so although the method is consistent with previous studies, and across models, it introduces errors in the forcing estimate. For a smaller number of models in RFMIP (Radiative Forcing Model Intercomparison Project), it is possible to calculate the transient total forcing F from pre-industrial control simulations with all forcing agents, as described by Pincus et al (2016) and also used by Gregory et al (2020). We refer to this more correctly estimated total forcing as $F_{\textrm{RFMIP}}$, and list it in table S1, noting that it agrees within 10% with the $F_{\textrm{total}}$ estimates in four out of five models, while for IPSL-CM6A-LR, $F_{\textrm{RFMIP}}$ is 40% greater than $F_{\textrm{total}}$.

Figure 1 shows the development of the GMSAT anomalies in the models listed in table S1, in historical simulations and under three different future scenarios. In SSP1-2.6, the temperature stabilizes during the 21st century and the 50- and 100-year trends reach their maximum in the early 2000 s, while in SSP2-4.5 trends are reduced later and GMSAT is still increasing in 2075, and in SSP5-8.5 the 50- and 100-year trends in temperature continue to increase and the temperature anomalies grow increasingly towards the end of the 21st century.

Zoom In Zoom Out Reset image size

Figure 2 shows snapshots of the relation between the sensitivity metrics (ECS, TCR and T140) and 1850–2014 forcings ($F_{\textrm{aer}}$, $F_{\textrm{GHG}}$ and $F_{\textrm{ant}}$), respectively, and temperature change since 1861 for 20-year averages centred around the years 1900, 1975, and 2050 (under SSP5-8.5). These three years were selected to illustrate the contrast between the historical period, when $F_{\textrm{ant}}$ dominates the correlation to temperature change with varying contribution from $F_{\textrm{GHG}}$ and $F_{\textrm{aero}}$, and the future period when sensitivity dominates (see further figure 3). In each case the coefficient of determination, R², and statistical significance at the 95% level is indicated in figure 2.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

The 1861–1900 GMSAT anomaly is small, and sometimes even negative (indicating that the trend is negligible compared to the internal variability in temperature), and correlations with sensitivity metrics are negligible. For the period 1861–1975 when the temperature anomaly is still small, this remains the case, but for the future projection period 1861–2050 (SSP5-8.5) the sensitivity metrics display a positive correlation with the GMSAT anomaly, and TCR more so than ECS. The spread in sensitivity among models is also smaller for TCR. T140 has a similar correlation as ECS with GMSAT anomaly at this time.

The correlation between total anthropogenic forcing and GMSAT anomaly, on the other hand, is only significant for the historical snapshots (1900 and 1975) and near zero for the future projection snapshot (figure 2). The correlation with individual forcing components $(F_{\textrm{aer}}, F_{\textrm{GHG}}$) is for all cases small, and statistically significant only for $F_{\textrm{GHG}}$ at 1900.

Figure 3 shows how R² for the correlation between GMSAT, sensitivity metrics and forcing measures changes over time for the three SSPs, from the year 1850 to 2100. The relationship between warming and climate sensitivity is negligible in the historical period, increases strongly in the early 2000s and then remains high in all three future scenarios. The total anthropogenic forcing instead correlates with the modelled temperature anomaly during most of the 20th century, but does not determine model spread in the future projections.

This confirms the picture of warming during the historical period being determined by model forcing strength while feedback strength, and hence sensitivity metrics, play a more prominent role in predicting temperature change in the future, when CO₂ forcing dominates and total forcing is less uncertain (Crook and Forster 2011, Forster et al 2013, Grose et al 2018, Lutsko and Popp 2019). The individual components of anthropogenic forcing, compared to pre-industrial conditions, remain largely uncorrelated with temperature anomaly throughout the period.

For comparison of the different sensitivity indices, it is useful to look first at the SSP5-8.5 scenario (figure 3(a)), which is most similar to the 1pctCO2 simulations that TCR and T140 are derived from. Here, R² for TCR levels out at ca. 0.75 by the end of the 21st century, meaning that the inter-model spread in TCR explains 75% of the variance in the inter-model spread in projected change in GMSAT. For T140 and ECS, the degree of explanation becomes even larger, with an R² of 0.9 by 2100. Considering the effects of warming patterns and time-dependent feedbacks (section 2.1), this is consistent with TCR representing a sensitivity that includes only the feedbacks that dominate in early stages of the warming, whereas delayed feedbacks are to a greater extent captured by T140, taken 70 years later, and of course also by ECS that includes the adjustment of ocean heat uptake to equilibrium, that neither of the transient measures can capture. In the mitigation scenarios SSP5-4.5 and SSP1-2.6 (figures 3(b) and (c)), where the temperature is more stabilized, ECS does not surpass the transient measures in predictive power within the 21st century. In the CMIP6 abrupt-4xCO2 simulations, from which the ECS is estimated, there is a shift in the feedback parameter after around 3–5 K warming (on average 4 K, see figure 3 of Meehl et al 2020). This warming is reached in SSP5-8.5 around the same time as R² for ECS surpasses those for the transient measures (figures 1 and 3), while in the mitigation scenarios an average warming of 4 K is not reached within the 21st century. Hence, the weaker correlation between ECS and GMSAT change in the mitigation scenarios could relate to the weaker warming in those scenarios compared to the stronger forcing scenario. These results differ slightly from those of Grose et al (2018), who in their figure 2 showed the ECS R² in the CMIP5 models to meet or surpass the TCR R² earlier for all scenarios.

For correlations with temperature trends (figure S2), the R² value for ECS becomes larger than that for TCR by the year 2000 in all scenarios. This confirms the greater relevance of ECS on longer time scales, beyond 2100, seen in figure 3, as the later and larger GMSAT anomalies increasingly contribute to the temperature trend.

The climate sensitivity is greater, and the global mean temperature trend is overall larger in CMIP6 than in CMIP5 in all three scenarios reviewed (comparing figure 1 with a and b of figure 3 of Grose et al 2018). This may partly explain the differences in predictive power of ECS and TCR between CMIP5 and CMIP6, for both temperature anomalies (figure 3) and temperature trends (figure S2). A stronger positive temperature trend and higher sensitivity indicates a climate with a longer response time, which is farther from equilibrium, and for which ECS is a less suitable measure of the temperature evolution. A relation between response time and sensitivity follows from equation (1) with assumed time-invariant α (see Hansen et al 1985). It is also manifest as a positive correlation between ECS and response time scale for the abrupt-4xCO2 simulations in the studied model ensemble (figure S3). Dividing the CMIP6 models into two subsets based on their sensitivity indeed suggests that ECS performs better for the lower sensitivity models, and less well for the higher sensitivity models, but the robustness of this analysis is limited by the small sample size (figure S4).

Further insight can be gained from separating the GMSAT into land and ocean, which yields higher correlation with ECS earlier for the ocean only case (see figure S5). This is in line with ECS better describing slow or delayed warming, and shifts in feedback dominance from evolving temperature patterns occurring over the ocean.

An even greater difference, however, is seen from separating the correlations with temperature anomaly into northern (NH) and southern hemisphere (SH) averages. In the SH, ECS clearly dominates the degree of explanation of projected temperature anomaly (see figure S6), which can only partly be explained by the ocean dominance of the SH. A more detailed separation of the temperature anomaly into latitude bands, shown in figure 4, points specifically at the tropics in SH and NH, and SH mid-latitudes as regions where the correlation between temperature anomaly and ECS becomes quite high. In the NH mid- and high latitudes, where the early transient warming is large, TCR explains more variation in model spread in temperature.

Zoom In Zoom Out Reset image size

The relation between regional temperature change and global mean sensitivity metrics is further illustrated in figure 5 which shows R² for the correlation between 1850–1869 and 2080–2099 warming and the three sensitivity indices, respectively, for each grid point on the map. This figure can be compared with figure 4 in Grose et al (2018), except that it uses a different reference period for the warming.

Zoom In Zoom Out Reset image size

The correlation overall increases with stronger forcing scenario (not shown), and figure 5 shows SSP5-8.5, where the temperature change is most prominent at the end of the century. T140 has the overall highest correlation values, i.e. this sensitivity metric best describes the pattern of regional temperature anomaly at the end of the century. Compared to CMIP5 (figure 4 of Grose et al 2018), the pattern for correlation with ECS is similar, but both TCR and T140 are in CMIP6 better correlated with regional temperature anomalies, particularly in the NH. In agreement with Grose et al (2018), however, correlations are weak in the Southern Ocean and the North Atlantic for all three sensitivity indices. These are areas where the relative standard deviation in temperature is large (not shown) and the low correlation indicates that the regional temperature anomaly in those areas is not directly coupled to the global mean temperature change, neither transient (TCR, T140) nor approximately equilibrated (ECS).

Even though the gridded correlations in figures 5(b)–(d) are almost exclusively statistically significant, we cannot assign statistical significance to the geographical distribution of their differences, with the small sample size.

Using model spread in sensitivity and historical forcing to explain variability in past and future simulated temperature also leads to the question whether forcing strength and sensitivity in the models are related, so that high-sensitivity models are also low-forcing models, and vice versa.

Figure 6(a) shows adjusted ERF and ECS for each of the CMIP6 models where total forcing could be calculated (see section 2.4) with the method of Forster et al (2013), and in addition those from the RFMIP pre-industrial SST simulations. The theoretical line from Kiehl (2007), valid for the 20th century temperature change and change in ocean heat content, is included in the figure. A majority of the models fall within the given uncertainty range of ±0.2 W m⁻², but there is significant spread around the line. The value for R² is low (0.28 when calculated with respect to the theoretical line and 0.31 when calculated with respect to the least-squares best fit), comparable to what Forster et al (2013) found for CMIP5 models, and not large enough to suggest a direct compensation between forcing and sensitivity. Figure 6(b) in turn shows the ERF from aerosol forcing only ($F_{\textrm{aer}}$) in relation to ECS, as also shown by Smith et al (2020) and Meehl et al (2020) for two different and partly overlapping subsets of CMIP6 models. Our figure includes those models, and an additional six models, as described in section 2.4, and no significant correlation is found (R² = 0.00).

Zoom In Zoom Out Reset image size

In figure 6(a) the forcing is taken as an average for five years (2001–2005 for 2003), but interannual variability, not least in the natural component of the total forcing, makes the analysis sensitive to the choice of year, and for example 2008 (2006–2010) and 2012 (2010–2015) have greater spread, and R² values of 0.19 and 0.17 respectively, calculated for the best-fit lines in each case (not shown).