The effects of climate extremes on global agricultural yields

We used a global, high-resolution crop yield dataset which includes data of the top four global crops (maize, wheat, rice, soybeans) across ∼13 500 spatial units worldwide, spanning the years 1961–2008 (Ray et al 2012). The dataset was developed by retrieving agricultural information from public source, statistical bureaus and agricultural agencies and contains time series of production, yields and area harvested for each crop type and administrative unit. Data availability was variable across regions and time periods, with data missing especially in the early years of the available data. Where data were missing, the 5-year average per spatial unit was used to interpolate the time series, constrained by available data of the geographic unit at the next higher level (e.g. the national scale; see Ray et al (2012) for further information on the data collection and processing). We separated the wheat yield data into winter and spring wheat based on the distinction provided in the crop calendar by Sacks et al (2010, SI section 1.1.1 is available online at stacks.iop.org/ERL/14/054010/mmedia).

To analyse the impacts of mean and extreme climate conditions on crop yields, we used data from the Climatic Research Unit (CRU) TS 3.23 dataset (Harris et al 2014) and the HadEX2 extremes indicator dataset (Donat et al 2013b) as predictor variables, spanning the same time period as the yield data. These two datasets were used as they are based on observational data, provide meteorological and climate extremes data at monthly resolution, which is required for focusing on the growing season months (the extremes indicators were calculated from daily-scale station data and aggregated to the monthly scale). Both datasets have nearly global coverage and cover several decades back to at least the mid-20th century. This means that relating climate data to the crop dataset is possible, taking advantage of the entire length of record. While the CRU TS 3.23 dataset is available on a 0.5° × 0.5° grid, the HadEX2 data was available on a comparatively coarse 2.5° × 3.75° grid. Other extreme indicator datasets at higher resolution are available, but these either do not provide quasi-global coverage and were not quality controlled to the same extent as the HadEX2 data (e.g. GHCNDEX, Donat et al 2013a; see supplementary figure 1 for a comparison of global coverage of HadEX2 and GHCNDEX), or are based on reanalyses data that do not cover the complete time period for which agricultural data is available (e.g. indices based on ERA-Interim, Dee et al 2011, starting in 1979; see SI section 1.1.2 for a discussion of the dataset choice).

Climate variables included in our analysis are: mean monthly precipitation, mean monthly near-surface air temperature, mean diurnal temperature range, frost day frequency, extreme indicators from HadEX2 capturing high and low temperature extremes as well as heavy precipitation events (see table 1 for an overview of predictor variables). The high and low temperature predictors measuring the frequency of warm days (TX90p) and cold nights (TN10p) are percentile-based relative to the local temperature distribution, hence thresholds are based on local climate conditions (see Donat et al 2013b). We further calculated the SPI-6 index from CRU TS 3.23 precipitation data as a proxy for meteorological drought conditions (table 1).

The harmonised crop calendar (v1.0) of the Agricultural Model Intercomparison and Improvement Project (AgMIP) (Rosenzweig et al 2013) was used to define planting and harvesting dates. We masked this calendar to only include grid cells where published growing season dates from either Sacks et al (2010) or the MIRCA2000 dataset (Portmann et al 2010) were available. The crop calendar provides different growing season dates for irrigated and rainfed systems for each crop and grid cell. We selected the growing season dates with the dominant irrigation status based on the MIRCA2000 land use dataset (Portmann et al 2010). Planting and harvest dates were converted into months; statistical parameters (minimum, mean and/or maximum) were calculated over all months of the growing season, including planting and harvesting months.

All spatial data were re-gridded to a common 1.5° × 1.5° grid by bilinear interpolation to balance the coarse resolution of HadEX2 (2.5° × 3.75°) with the finer 0.5° × 0.5° resolution of all other datasets (see SI section 1.1.4). The regridded time series were detrended using a parameter-free trend estimation method, singular spectrum analysis (Vautard et al 1992), implemented in the 'ssa' R package (Zhao 2016), yielding a multi-annual smoothing (see supplementary figure 2 for a comparison of detrending methods). Yield time series were detrended to remove temporal trends due to technological progress and management changes. Climate time series were detrended assuming that agricultural managers are continuously adapting to changing climate conditions (Butler and Huybers 2013), and climate shocks deviating from the expected trend have the most detrimental effect on yields. Detrended climate predictors and yields were standardised per grid cell by dividing by their standard deviation to account for the fact that grid cells with smaller mean yields tend to have smaller anomalies and assuming that local agriculture is adapted to local mean climate variability (SI section 1.1.5).

All data are publicly available or available on request (see supplementary table 2 for data access).

We applied the machine learning algorithm 'Random Forests' (Breiman 2001) to detrended and standardised time series of crop yields and climate predictors to predict crop yield anomalies. The Random Forest algorithm is a non-parametric statistical method, which uses an ensemble of decision trees and can be applied to regression and classification problems (Breiman 2001). Random Forests have previously been applied to the analysis of the influence of climate factors on yields in specific countries or regions (Jeong et al 2016, Hoffman et al 2017), but an analysis of extreme event impacts at the global scale, using sub-national yield data, has not yet been done.

To estimate the variance of yield anomalies explained by climate predictors, we calculated R² values from cross-validated out-of-sample predictions (SI section 1.2.3 for further information). We obtained R² values from time series at the continental and global scale by calculating aggregated yield anomalies (actual and predicted) as weighted arithmetic mean with harvested area as weighting factors. This calculation ensures that grid cells with large harvesting areas, where yield losses have greater impacts on regional or global production anomalies, contribute more to overall R² values—unlike calculating a simple mean R² over all grid cells (SI section 1.2.3).

To assess the relative impact of the climate extremes, we subsequently trained a Random Forest on a subset of predictor variables representing only mean climate conditions during the growing season and calculated the differences in explained variance (out-of-sample R²) as a proxy for the relative influence of extreme event indicators.

We tested the robustness of our results to the choice of statistical learning method by comparing the Random Forest model against four other widely used methods (Support Vector Machine, Generalised additive models, K-Nearest neighbours, and multiple linear regression). While the Random Forest method yielded overall slightly higher explained variances (R²), we found the results of this study with respect to the yield effects of climate variations and extremes on yields, including the relative importance of climate extremes, to be robust to the choice of statistical method (see SI section 1.4).

The Random Forest algorithm allows for an investigation of relationships between climate variables and yield anomalies by providing the relative variable importance and the functional relationship between each predictor and the target variable (Jeong et al 2016). Variable importance values were calculated for each crop type, based on a metric that captures the increase in mean squared error (MSE), calculated from out-of-sample predictions, after randomly permuting the values of the respective predictor. Variable importance values were obtained using the R 'randomForest' library (Liaw and Wiener 2002).

Partial dependence plots visualise the functional relationship between a predictor variable and the response variable after all other predictors are accounted for, similar to a sensitivity test for each predictor. Partial dependence plots were created by randomly sampling 500 observations from the complete dataset of climate observations (pooling all grid cells and time steps) and varying the one predictor variable of interest at 100 equidistant points between its minimum and maximum. After applying the statistical model to the resulting 5000 data points (500 × 100), we calculated the mean yield response across the whole range of the predictor variable as well as the uncertainty bands of the predictions (see SI section 1.2.7 for further information).

To put our findings into the context of global food production, we estimate the contribution of climate factors, and particularly extreme events, to fluctuations in regional crop production. Crop production varies with crop yields (t/ha) and harvested areas (ha). Thus, to estimate the relevance of climate variations to global and regional variances in crop production, we considered: (i) the share of a region's crop production relative to global production, (ii) the mean variability of regional production, (iii) the extent to which production anomalies are associated with yield anomalies (as opposed to variations in harvest area), and (iv) the explained variance of yield anomalies, predicted by fluctuations in climate conditions and specifically extreme events. The four indices are combined into composite indices by calculating the geometric mean of factors i–iv.

A detailed description of the methodological approach and a discussion of limitations can be found in the supplementary information ('Extended methodology' and 'Limitations'). The data processing and analysis scripts are available online (at https://github.com/elisabethvogel/gs_clim_data for the preparation of the growing season climate data, and https://github.com/elisabethvogel/climate_extremes_agriculture, for the statistical analysis).

The Random Forest models—including both growing season climate means and extremes as predictors—were able to explain nearly half of the variance of global yield anomalies for maize and spring wheat (49% and 46%, respectively), about one quarter (28%) of yield anomalies for rice and one fifth (20%) for soybeans (figure 1(a); all numerical values shown in supplementary table 3). Explained variances of maize yields were highest among all crops, both globally and for most regions (explained variances between 45% and 55% of variance in all continents, except South America, with 25%, and Oceania, with 10%, see figure 1(a)). For soybeans, explained variances are highest in South America and Africa (28%–30%). For rice, approximately 28% of the variance in yields in Asia is explained by the considered climate variables. Explained variances for spring wheat were highest in Oceania, specifically in Australia with an R² of 67%, i.e. about two-third of the variance of observed crop yields can be explained by climate conditions. In contrast, the considered climate indicators can only explain a minor component of observed variations in winter wheat yields (not shown) likely due to the comparatively long growing season spanning several seasons. Therefore, winter wheat was excluded from this analysis.

Zoom In Zoom Out Reset image size

By comparing the explained variances of the full statistical models to variances explained by reduced models which only account for mean climate conditions, we quantify the fraction of the explained variance that can be attributed to extreme climate conditions. While growing season mean climate conditions captured nearly 30% of the global variability in spring wheat yields, they do not capture any significant part of the variance of maize, soybeans and rice yields (6%, 0% and 1%, respectively; figure 1(a); supplementary table 3). When excluding temperature and precipitation extremes as predictors in the Random Forest models, the explained global yield variance dropped by 43% for maize, 20% for soybeans, 27% for rice, and 18% for spring wheat (figure 1(b); supplementary table 3). For maize, soybeans and rice, the additional variance explained by the extreme indicators is larger than the variance that can be explained by mean climate indicators alone.

Climate extremes have a particularly strong association with maize yields at the continental scale; the explained variance in North America and Asia reduces by nearly 40% when extreme event indicators are excluded (figure 1(b)). Both regions together were responsible for more than 70% of global maize production in 1990–2008 (production shares shown in figure 4). Compared to mean growing season temperature and precipitation, extreme climate conditions also have a large influence on rice production in Asia (the explained variance decreased by 26% when excluding extreme event indicators, figures 1(a) and (b)), the most important rice producing continent with a 90% contribution to global production. Accounting for extreme conditions also more than doubles the explained variances of soy yields in North and South America (figures 1(a) and (b)), together responsible for more than 80% of global soy production.

Overall, our analysis suggests that temperature-related predictors are more strongly correlated with crop yield anomalies than precipitation-related predictors across all four crops (see supplementary figure 9 for variable importance plots). Here, the relevance or predictive capacity of a predictor is quantified by the relative increase in MSE—calculated as out-of-sample error—between observed and predicted yield anomalies after randomly permuting the values of the predictor of interest (Methods).

For each crop, the top three predictor variables were among the following temperature-related variables: (i) warm day frequency (TX90p), (ii) cold night frequency (TN10p), (iii) the growing season average temperature, or (iv) the mean diurnal temperature range (supplementary figure 9). A greater diurnal temperature range indicates a wider temperature distribution at the hourly scale, characterised by high day-time temperatures and low night-time temperatures, both of which are shown to have negative yield effects (see next section).

While the number of frost days has low predictive capacity, we find that the indicator for cold night frequency has high variable importance for all crops except spring wheat. Frost events are defined by a fixed, absolute temperature threshold (New et al 2000) and may never occur in warm climate zones, or may generally not occur during the growing season in temperate regions, whereas the indicator for cold nights is percentile-based and depends on the temperature distribution of the given day of year at a particular location (Donat et al 2013b). Our results suggest that colder than usual temperatures reduce crop yields even above the frost-event threshold, because plant growth depends on accumulated temperature exposure (McMaster and Wilhelm 1997, Schlenker and Roberts 2009). Mean precipitation and SPI-6 play a less significant role for determining crop yield anomalies (with the exception of spring wheat in Oceania, supplementary figure 10 for variable importance plots by region). Similarly, the maximum 5-day precipitation intensity, which captures extremely intense precipitation events, has low variable importance across all crops (supplementary figure 9). Additionally, we tested the 1-day maximum rainfall intensity (Rx1day instead of Rx5day, with all other predictors being the same), resulting in equally low variable importance values for extreme precipitation at shorter time scales (not shown here).

Random Forests do not rely on a prescribed functional relationship between predictors and the dependent variable, but represent a data-driven form that can be visualised by partial dependence plots (Methods). In the following and plots are provided for warm day frequency (TX90p) and cold night frequency (TN10p), i.e. two temperature predictors with highest predictive capacity.

We find negative impacts from increased warm day frequencies on all crops, i.e. an increase in numbers of unusually warm days is associated with decreases in yields (figure 2). Similar to the warm day frequency, higher frequencies of unusually cold daily minimum temperatures also have negative effects on yields of all crop types, except of spring wheat (figure 2). However, overall, the cold night frequency has low variable importance for spring wheat (see supplementary figure 9 for variable importance), hence the uncertainty of the functional shape is higher. At the regional scale, the effect of increased warm day frequency is particularly negative for maize in Europe and soybeans, spring wheat and rice in North America; increased occurrences of unusually cold days are negatively correlated with maize and soybean yields in Europe and rice yields in Oceania (supplementary figure 10 for regional variable importance plots).

Zoom In Zoom Out Reset image size

Guided by previous studies (Lobell et al 2013, Carter et al 2016)—which indicate that observed yield reductions associated with high temperatures may be induced by heat-related water scarcity—we tested whether the decline in yields with increasing warm day frequency is reduced by irrigation. We assessed the effect of irrigation on extreme event impacts by calculating the Random Forest for irrigated and rainfed grid cells separately (defined as ≥80% irrigated or rainfed, respectively) and visualising their partial dependences (Methods). We find that irrigation can mitigate negative yield impacts of high temperature extremes for maize, soybeans and spring wheat, with the greatest effects found for soybeans and spring wheat (figure 3). For soybeans, irrigation shifts the threshold above which temperatures have a negative impact to higher temperatures, while for maize and spring wheat, the response curve is flatter and the negative effect of high temperatures reduced. For rice, the fraction of rainfed cropping area is less than 10% globally, therefore, we omitted rice from this analysis.

Zoom In Zoom Out Reset image size

Agricultural production is the product of yield (t/ha) and area harvested (ha), i.e. fluctuations in both variables have effects on fluctuations in crop production. We tested to which extent yield fluctuations due to variations in climate conditions and extreme events translate into production anomalies at the regional and global scale. By developing an aggregate indicator that takes into account a region's contribution to global production, the mean variability of production and the correlation between production anomalies and yield anomalies, we identify hot spot regions that are critical for food production and at the same time particularly susceptible to fluctuations in climate conditions, including climate extremes (Methods).

At the global scale, crop production anomalies are strongly correlated with fluctuations in yields. Correlation coefficients range between 0.78 and 0.94 (except for soybean production, which is dominated by fluctuations in harvested area) and inter-annual changes in yields explain ∼60%–90% of the variance in production anomalies of spring wheat, maize and rice (figure 4; see table 2 for the numeric values). In combination with our previous results on the explained fraction of variance in yield anomalies (figure 1), we find that at the global scale, maize and spring wheat are the crops with the highest sensitivity of production to climate variations and extremes, as they are characterised by a large variance of production anomalies, high correlation of production anomalies with yield fluctuations and a strong relationship between yields and climate conditions during the growing season (table 2).

Zoom In Zoom Out Reset image size

Analysed separately by continent, the regions with highest contribution to global food production and at the same time high estimated sensitivity to climate conditions are: spring wheat production in Oceania, Europe, North America and Asia as well as maize production in North America and Europe (table 2, column for indicator A). Hotspot regions that show strong impacts of climate extremes include: North America for maize, spring wheat and soybeans production, Asia for rice and maize production and Europe for production of spring wheat (table 2, column for indicator B).

Overall, we find that the climate factors considered in this study—capturing climate variability and climate extremes during the growing season—explain almost half of the variability in maize and spring wheat yield anomalies globally, and up to two third of the regional variability, e.g. for spring wheat in Oceania. We show that variables describing climate extremes contribute more than half of the explained variance of yield anomalies of maize, rice and soybeans and nearly half of spring wheat at the global scale, and hence are fundamental for understanding crop yield variations. The explained variances (R²) presented here are of similar magnitude as previously reported in other studies: Lobell and Field (2007) and Ray et al (2015) estimate that growing season mean temperature and precipitation explain about one third of crop yield variations globally; Frieler et al (2017) determined the influence of weather variations on crop yields of main producers to be in the range of ∼5% (rice in India) to ∼80% (wheat in Australia). The reported values and findings are not directly comparable due to very different study designs and methods of calculating the explained variance. However, these studies agree that climate variations only explain a part of the yield variability and other factors, such as soil properties, management decisions (e.g. irrigation rate, fertiliser use) and market factors (e.g. fertiliser and energy prices) likely contribute to the remaining yield variations. To our knowledge, this study is the first to provide estimates of the fraction of variance of yield anomalies explained specifically by climate extremes. Unlike other previous studies, we report a conservative estimate of the explained variances by using a cross-validation approach and reporting R² values from out-of-sample predictions, and by applying one statistical model in one configuration for all crop types and regions.

We identified the most influential predictors using a variable importance metric based on an increase in mean squared error, and found temperature-related predictors to be more relevant for yield predictions than precipitation-related climate factors (such as extreme precipitation or drought). The close link between temperature indicators, particularly high temperature indices, with yields is consistent with previous research at the national scale, which found heat waves to be significantly associated with grain yields (Lesk et al 2016, Zampieri et al 2017). In contrast, SPI-6—low values of which are indicative of meteorological drought—was found to be of low to medium importance for yield anomaly predictions, unlike previous studies that found a statistical relationship between reported droughts and yields at the national scale (Lesk et al 2016, Zampieri et al 2017). These differences in results suggest that (a) either droughts only influence yields if they occur at larger spatial and temporal scales than investigated here, or (b) meteorological droughts, defined by below-average precipitation over a given period, may not be sufficient to capture yield anomalies, and indicators measuring hydrological or agricultural drought (Keyantash and Dracup 2002, Passioura 2007, Quiring 2009) may be more suitable for predicting crop yield losses. Related to this, the relative contributions of temperature and precipitation anomalies to drought are difficult to disentangle—low rainfall and high temperatures both increase drought severity and are often significantly correlated (Trenberth and Shea 2005, Zscheischler and Seneviratne 2017). Previous research (Frieler et al 2017) as well as our results show that the negative yield effects of high temperatures are intertwined with water stress and can be mitigated by irrigation. We discuss the collinearity of predictors and our approach to address these in SI section 2.1. Here, we chose not to use a composite drought index that takes into account both temperature and precipitation to reduce collinearities in the predictor variables. However, future research could investigate the use of agricultural drought indicators that capture both precipitation and temperature effects, including the Standardised Precipitation-Evapotranspiration Index (SPEI) (Beguería et al 2014), or global gridded soil moisture datasets (e.g. Dorigo et al 2017) for predicting yield losses.

Adaptation to long-term climate change requires concerted efforts at local, regional and international level to ensure future food security. By using a composite indicator, this study identifies crop producing regions that exhibit a strong association with climate fluctuations and climate extremes and that are particularly critical for global production. These are primarily located in industrialised, high-input crop producing regions, such as North America and Europe. However, we emphasise that adaptation to climate extremes is critical not only in major crop producing countries, but also in regions with a population that highly depends on local agricultural production for their livelihoods and nutrition. As example, maize production in Africa has the second highest explained variance (R² of 54%, 15.8% contribution of extreme event indicators) of any of the crop/continent combinations and hence a high dependence on climate conditions. While the share of global production is small (∼7%), maize production is critical for the food security in the region—in Sub-Saharan Africa, the proportion of maize grain used directly for human consumption is approximately 70% compared to 3% in North America (Fischer et al 2014); hence any variations in crop production could have very severe impacts on the region's food supply. Such nuances could be further investigated by applying different weights to the individual indicators or by introducing additional indicators, such as the share of subsistence farming or the percentage of production for human consumption per region. Here, we aimed to capture influences of climate extremes on global production anomalies, and therefore calculated a composite index using equal weights, but a range of other combinations of indicators are possible and may be explored in future studies.

Climate change is expected to exacerbate the pressure on food production, as the frequency and severity of certain types of extremes, particularly high temperature extremes, are predicted to further increase in the future (Coumou and Robinson 2013, IPCC 2013, Sillmann et al 2013). Heat and frost extremes already pose a significant challenge to agricultural producers (Zheng et al 2012, Barlow et al 2015). Our results demonstrate that irrigation can partly mitigate the detrimental effects of extreme heat. However, freshwater availability is a concern under future climate conditions and previous research suggests that considerable areas of irrigated cropland worldwide may have to be converted to rainfed agriculture (Elliott et al 2014), increasing the vulnerability to high temperature extremes. Seasonal climate forecasts are an important adaptation tool that can assist farmers in assessing and potentially mitigating climate-related risks to crop production in the upcoming seasons (Hansen 2005, Ash et al 2007, Meza et al 2008, Hansen et al 2011). This study helps prioritising research efforts underpinning the development of such systems, by identifying the most relevant climate factors for yield predictions and estimating their contribution to yield fluctuations, for each region and crop individually. In future studies, the results may be extended to assess predictive skill several months prior to harvest and could be used to improve existing agricultural forecasts or support efforts to introduce such forecast systems in regions where they have not yet been implemented.