Optimal spatial scales for seasonal forecasts over Africa

Our overarching aim is to investigate the representativity and skill of regional seasonal forecast information for anticipating local rainfall conditions in Africa. We analyse seasonal total rainfall during the four calendar seasons of December-January-February (DJF), March-April-May (MAM), June-July-August (JJA), September-October-November (SON), assessing the utility of regional information of the interannual variability of seasonal rainfall, with emphasis on measuring variability of the seasonal rainfall distribution using tercile categories. Whilst it could be argued that finer quantile categories (e.g. quintiles or deciles) might provide more detailed and 'useful' information on the seasonal rainfall distribution, terciles are analysed here since they are the primary format used to communicate seasonal forecast information to users. It should further be noted that the agricultural sector in Africa is highly sensitive to rainfall variability and even variability that is not statistically extreme (i.e. lower tercile events) is likely to have widespread and severe impacts.

2.1.1. Evaluating representativity

We use high horizontal resolution (0.05°) rainfall observations (section 2.2.1) to quantify representativity of regional rainfall variability of local conditions. We measure regional representativity by linearly correlating regional-mean with local inter-annual seasonal rainfall observations using Pearson's correlation coefficient. Correlation magnitudes can range between -1 (negative linear correlation) to 0 (no linear correlation) to 1 (positive linear correlation). Positive correlations close to 1 indicate strong co-variability of regional and local rainfall and thus high representativity. Conversely, positive (or negative) correlations close to 0 indicate weak or no co-variability and thus little to no-representativity. Correlations are only presented if they are statistically significantly different from a correlation of zero at the 95% confidence level (p-value < 0.05) computed using a two-tailed t-test.

We also quantify regional representativity of the inter-annual variability of the seasonal rainfall distribution as summarised by tercile categories. We measure this using the probability of detecting (POD) the observed local tercile, using the tercile observed at the regional scale, defined as follows:

$$\begin{equation} POD = \frac{H}{H+M} \end{equation} \tag{ 1 }$$

Where, H is the total number of hits, i.e. the number of times that the local tercile is correctly detected by the regional tercile, and M is the total number of misses, i.e the number of times that the local tercile is not detected by the regional tercile, for a given tercile category. POD values therefore range from 0 to 1, with 1 representing perfect local tercile detection and 0 indicating that all local tercile events were missed. Computing the POD in observations therefore provides an indication of the maximum possible representativity that could be obtained if a regional tercile forecast was perfect.

2.1.2. Evaluating skill

2.1.3. Combining representativity and skill

2.2.1. CHIRPS rainfall observations

2.2.2. Dynamical hindcasts of seasonal rainfall

Comparisons between representativity and spatial scale for wet seasons at five locations in Africa are illustrated in figure 2. As spatial scale is increased, both the correlation between regional and local inter-annual rainfall variability and the POD for the lower tercile category decrease at a rate highly dependant on location (figures 2 (c) and (d)). This location-dependence of representativity can be seen by examining both metrics at 5°; a typical spatial scale at which a regional forecast might be issued. At three locations (Zambia, Somalia, DRC), regional-local rainfall is strongly positively correlated (> 0.9) and the POD is relatively high (> 0.8) suggesting that regional variability is highly homogeneous and thus representative of local variability there. In contrast, lower correlations at Fort Portal, Uganda and the Jos Plateau, Nigeria where the topography is complex, reveal that the regional scale is less-representative due to highly localised inter-annual variability due to orographic effects and the characteristics of convective systems.

Zoom In Zoom Out Reset image size

The representativity of the typical spatial scale of a regional forecast varies seasonally and spatially across Africa, as shown by mapping correlations between regional (5°) and local (0.05°) interannual variability at every 0.05° grid-box in the CHIRPS data (figures 2(i)–(l)). High correlations indicate regionally homogeneous interannual variability, highlighting regions where seasonal rainfall is modulated by large-scale drivers. Such regions include southern Africa (DJF wet-season), East Africa (MAM; long rains and SON; short rains) and the Guinea Coast of southern West Africa (JJA; West African monsoon) where rainfall variability is known to be strongly linked to large-scale drivers (Black et al 2003, Cook et al 2004, Hastenrath 2007, Nicholson 2013, Cretat et al 2019, e.g.). Strong spatial homogeneity in inter-annual variability over East Africa during the short rains corroborates with previous studies (Moron et al 2007, Camberlin et al 2009). Conversely, lower correlations indicate increasingly localised variability and exhibit more complex spatial patterns over central Africa and highland areas, including the Ethiopian, Cameroon and Guinea Highlands, and the East African Rift system (c.f. figure 2). Such regions experience high interannual variability in deep convection (Hart et al 2019), are where convective systems preferentially form (Rowell and Milford 1993, Laing and Fritsch 1993, Hodges and Thorncroft 1997, Laing et al 2008, Jackson et al 2009), and rainfall is locally enhanced by orography (e.g. Kamara 1986, Nicholson 1996). Crucially, these correlation maps suggest that regional-scale forecasts of seasonal rainfall, such as those produced by RCOFs, are more likely to be representative and therefore locally useful over southern Africa and East Africa where there is strong co-variability between regional and local seasonal rainfall. Note that these representativity results may be less certain over some regions with low gauge density in CHIRPS (e.g. Central Africa). However, the spatial precipitation patterns estimated by CHIRPS can be treated with some confidence since they are highly dependant on satellite observations of deep convective systems, underpinned by a high-resolution climatology (Funk et al 2015a,b).

The local representativity of regional forecasts also depends on whether their information content (i.e. tercile probabilities) is a representative indicator of local conditions. This is evaluated by mapping the local tercile POD when using the terciles observed at the regional spatial scale of 5° in CHIRPS (figures 3(a)–(h)). Unsurprisingly, POD values for the below normal tercile are high over regions of homogeneous inter-annual variability and low over regions of heterogeneous variability, introduced previously in figure 2. Similar spatial patterns and magnitudes of POD are found for the above normal tercile category as shown in figure S1 in Supplementary Material. However, for the normal category the POD is substantially lower (< 0.6), revealing frequent disagreement between the regional tercile and normal tercile observed at the local scale. This disagreement might be expected since by definition the normal category is more constrained, requiring rainfall totals to fall within both the lower and upper boundaries of the category. Conversely, single boundaries for the lower and upper terciles are more likely to be crossed in the presence of a large-scale anomalous atmospheric signal driven by external forcing. However, the low POD for the normal category strongly emphasises that the information content of any regional tercile forecast of the normal category, is frequently unrepresentative and therefore has little value for decision-making at local scales. This is of particular concern for regional scale tercile forecasts issued by RCOFs of the normal category, which tend to be over-forecast and for which the probabilities are generally not reliable (Mason and Chidzambwa 2008, Hansen et al 2011, Bliefernicht et al 2019, Walker et al 2019). In contrast, re-calculating the POD using a regional scale of 1° instead of 5° causes a widespread increase (POD > 0.7) in representativity even over heterogeneous rainfall regions. This marked increase can be explained by the fact that the smaller spatial scale of 1° is much closer to the spatial correlation scales of rainfall variability from convective systems at seasonal timescales (Lebel and Le Barbe 1997, Ali et al 2003, Maidment et al 2013). Therefore a skillful tercile forecast at a spatial scale of 1° could be highly relevant for local monitoring. Skillful forecasts at even smaller spatial scales may yield higher representativity over some regions (e.g. figure 2(d) observed POD ≥ 0.8 at spatial scales ≤ 1°). However, the relatively high POD at 1° is encouraging since 1° is approximately the typical spatial scale at which gridded dynamical forecasts are available for users.

Zoom In Zoom Out Reset image size

An assessment of the prediction skill of 1° SEAS5 hindcasts verified against the 0.05° CHIRPS observations is shown in figure 4. Regions of ensemble mean skill (ACC > 0) and lower tercile probability skill (ROC Area > 0.5) are very similar, with the skill patterns consistent with a spatially coarser evaluation of SEAS5 at 2.5° by Johnson et al (2019). Similar skill patterns are also found for the above normal tercile category, and in hindcasts from three other dynamical models (see figures S2-S5 in Supplementary Material). Statistically significant high skill regions include Southern Africa (DJF), East Africa (DJF, SON) and southern West Africa (JJA), where seasonal rainfall has known predictability (Weisheimer and Palmer 2014, Mwangi et al 2014, Ogutu et al 2017, Walker et al 2019) due to strong teleconnections with SST variability particularly in the Pacific (via the El-Ni$\tilde{n}$o Southern Oscillation; ENSO) and Indian Oceans (Mason et al 1996, Nicholson and Jeeyoung 1997, Indeje et al 2000). Over high skill regions the SEAS5 hindcasts can therefore be expected to provide skillful predictions of the expected sign of the seasonal rainfall anomalies and the lower and upper terciles of the rainfall distribution. In contrast, no skill (ACC ≤ 0, ROC Area = 0.5) is present over areas in Central Africa, Northern West Africa (JJA, SON), Southern Africa and orographic regions, where deep convective activity is frequent (Bennartz and Schroeder 2012, Hart et al 2019) and precipitation processes are complicated by localised orographic effects (Kamara 1986, Oettli and Camberlin 2005). Note that the lack of statistically significant skill regions are a likely consequence of the relatively short hindcast period analysed. The overall skill patterns are comparable with the representativity results described in section 3.1. Regions of statistically significant high skill are strongly concomitant with the spatially homogeneous and thus representative regional rainfall regions presented in figures 2 and 3. Higher forecast skill over regions experiencing larger spatial scales of rainfall variability has also been shown at sub-seasonal timescales by Moron and Robertson (2020). This is expected since regions of highly spatially homogeneous rainfall variability are more likely to represent areas where the rainfall variability is driven by more predictable large-scale anomalous conditions (Haylock and McBride 2001, Moron et al 2006, Moron et al 2007).

Zoom In Zoom Out Reset image size

Figure 5 shows the dependence of SEAS5 skill on spatial aggregation when evaluated against CHIRPS at the same spatial aggregation scales at the five locations introduced previously in figure 2. The skill dependence with spatial aggregation is highly variable between the locations which have been chosen specifically to illustrate the 'regimes' of skill and representativity introduced in figure 1. Livingstone and Southern Somalia (SON) are locations with skill within homogeneous regions, whereas Northern DRC and Southern Somalia (MAM) are locations with no-skill within homogeneous regions (c.f. figure 5 and figures 2(c) and (d)). In contrast, the Jos Plateau is a location with skill within a heterogeneous region, while Fort Portal is a location with no-skill within a heterogeneous region. While spatial aggregation increases skill at Southern Somalia (MAM), Northern DRC and Fort Portal, it has no impact on skill in Southern Somalia (SON) and even degrades skill at Jos Plateau. Moreover, the skill response to spatial aggregation can be different for the ensemble mean compared to the lower tercile probabilities (e.g. Livingstone). Skill increases with aggregation may indicate locations where aggregation helps to reduce the less-predictable noise of localised rainfall variability, isolating the large-scale predictable rainfall signal (Gong et al 2003). However, skill increases with aggregation may also be due to the conflation of gridpoints from neighbouring regions with higher skill. In contrast, skill degradation with aggregation may indicate locations where aggregation conflates neighbouring regions with different climatic rainfall regimes and/or low skill. These results also reveal that the skill response to aggregation can be opposite over locations with similar representativity. For example, skill increases with aggregation at Fort Portal and decreases with aggregation at Jos Plateau even though both are located within relatively spatially heterogeneous regions. Moreover, even if spatial aggregation improves skill, representativity may be substantially lost (e.g. Fort Portal, c.f. figure 1). These combined impacts of aggregation emphasise the importance of using representativity alongside skill to decide whether spatial aggregation is useful.

Zoom In Zoom Out Reset image size

Finally, we combine skill and representativity to map out the optimal spatial scales for presenting seasonal forecast information across Africa, using SEAS5 as an example (figure 6). The spatial aggregation scales at which the SEAS5 tercile probabilities of below normal rainfall exhibit skill are shown in figures 6(a)–(d) (see figures S6 and S7 in Supplementary Material for equivalent results for the above normal and near-normal tercile categories). Aggregation is not required over spatially homogeneous rainfall regions where SEAS5 is already skillful at 1° (e.g. Southern and East Africa, c.f. figures 2 and 4). However, increasing aggregation is typically required towards the edges of such regions. In these transition regions, seasonal rainfall variability progressively becomes less predictable at smaller scales because of the stronger influence of more localised rainfall processes and weaker influence of large-scale drivers. The skill threshold is not exceeded over areas of Central Africa, East Africa, West Africa, Madagascar and the outer regions of the domain, highlighting deficiencies in skillful seasonal prediction there which cannot be sufficiently resolved by spatial aggregation.

Zoom In Zoom Out Reset image size

We next present the 'local' representativity of the skillful aggregation scales using the CHIRPS lower tercile POD at the skillful spatial aggregation scales of SEAS5 at each grid-box (figure 6(e)–(h)). Over many regions where hindcasts meet the skill criteria without aggregation, such as East Africa (SON), the POD is near 1 and thus the native 1° scale of the model is both skillful and representative of local conditions. Conversely, locations where spatial aggregation both satisfies the skill criteria and exhibits relatively high POD values (e.g. > 0.7) are extensive but more localised. Many areas where substantial spatial aggregation is needed to obtain skill (> 5°) are associated with POD < 0.5, highlighting regions where skill gains from aggregation reduce the utility of the forecast because of the large representativity lost.

To help delineate such regions, we finally apply a representativity threshold of 0.75 to the POD values in figure 6(e)–(h) to combine skill and representativity, showing where the skillful aggregation scales of the hindcasts are locally useful (figure 6(i)–(l)). Skillful and representative locations are spatially variable and not only limited to the more predictable homogeneous rainfall regions (e.g. central Africa). Although such results depend on the skill and representativity thresholds used, they provide an example of the type of approach which could be used to diagnose the utility of seasonal forecast information alongside spatial scale, while allowing a user the flexibility to choose thresholds relevant to their risk aversion and requirements of the forecast information.