Tropical rainfall monitoring with commercial microwave links in Sri Lanka

Accurate and timely surface precipitation measurements with high spatial and temporal resolution are of paramount importance for many applications such as water resources management, agriculture (irrigation scheduling and crop insurance) and weather prediction. However, the majority of the Earth's land surface lacks rainfall data of sufficient quality. The total number of rain gauges from operational meteorological networks is estimated to be at most ∼0.25 million, while the number of those giving useful near-real time observations is estimated to be 8000–12 000 (Kidd et al 2017). Networks are especially sparse in developing countries (Lorenz and Kunstmann 2012). Moreover, in these areas the coverage of ground-based weather radars is also limited (Saltikoff et al 2019). Hence, many regions over the globe (particularly in Africa, South and Southeast Asia and Latin America) largely depend on satellite-based rainfall observations. Satellite estimates, however, are still associated with large uncertainty and often have a low spatial or temporal resolution (Kidd and Huffman 2011, Li et al 2021). Since rainfall displays large variability in space, the current coverage by radars and rain gauges and accuracy of satellites are often not sufficient for many applications such as short-term weather forecasting, flood prediction and water balance monitoring. This calls for alternative sources of near-surface rainfall information to complement the dedicated sensors to improve the quality, spatio-temporal resolution and coverage of quantitative precipitation products. Examples of these alternative sources, also referred to as 'opportunistic sensors', are windshield wipers (Rabiei et al 2013), crowdsourced internet of things personal weather stations (De Vos et al 2019a), and satellite broadcast receivers (Mercier et al 2015). These sensors—often not even designed to measure rain—fit the rise in demand for environmental opportunistic sensing (Muller et al 2015, Zheng et al 2018).

Here, we present one of the most promising sources of opportunistic sensing of rain: commercial microwave links (CMLs), also known as point-to-point or backhaul links. CMLs, called links from now on, are close to the ground radio connections used globally in cellular telecommunication networks. Along such links, radio signals propagate from a transmitting antenna at one base station (radio site, telephone tower or node) to a receiving antenna at another base station (figure 1(c)). A link often consists of multiple sub-links, mainly to allow for communication in both directions. Links operate at frequencies where raindrops substantially attenuate radio waves (Hogg 1968, Atlas and Ulbrich 1977, Olsen et al 1978). In rainy circumstances, signals lose their strength as they propagate from a directional antenna on one base station to a directional antenna on another base station within each other's line of sight. This is caused by absorption and scattering of microwaves by raindrops, which have sizes of a few millimeters. Hence, these raindrops are only slightly smaller than the typical wavelength of approximately 1 cm employed by links. The harder it rains, the more raindrops will be present on the path between transmitter and receiver and the larger these drops will be on average. As a consequence, the attenuation of the microwave signal increases with higher rainfall rates.

Zoom In Zoom Out Reset image size

Rain-induced attenuation and, subsequently, path-averaged rainfall intensity can be retrieved from the signal's attenuation between transmitter and receiver (Upton et al 2005, Messer et al 2006, Leijnse et al 2007a). If transmitted signal levels are constant, the rain-induced attenuation over the link path can be computed by subtracting the received signal level from a reference received signal level, the latter being representative for dry periods. Alternatively, in case of variable transmitted signal levels, the total attenuation over the link path must be used instead of received signal levels. This attenuation is computed by subtracting received signal levels from transmitted signal levels. A reference total attenuation during dry conditions must be computed similarly. Sometimes transmitted signal levels are unknown, but known to be varying; obviously, this method cannot be used in that case. The core of link rainfall retrieval is physics-based and computes the path-averaged rainfall intensity from the rain-induced microwave specific attenuation (dB km⁻¹), i.e. the average attenuation per km along a link path due to absorption and scattering of microwaves by rain drops. The relation between the two takes the form of a power-law (Atlas and Ulbrich 1977, Olsen et al 1978) of which the coefficients depend mainly on signal frequency and polarization, and are derived from measured rain drop-size distributions and simulations of electromagnetic scattering by rain drops (Leijnse et al 2008), preferably for the local climate. Alternatively, the values recommended by Int. Telecommunication Union (2005) can be used as a first guess. The acquired path-averaged rainfall intensities over a link network can subsequently be interpolated to rainfall maps (Overeem et al 2013). Comprehensive overviews on the history and physics of link rainfall retrieval have been published (Messer and Sendik 2015, Uijlenhoet et al 2018, Chwala and Kunstmann 2019).

We highlight two important sources of error in link rainfall estimation. First, signal levels often decrease during dry periods, e.g. caused by reflection of the beam, dew formation on the antennas, and dust. Wet antennas and strong humidity and temperature gradients in the atmosphere may lead to an altered antenna pattern or to refraction of the beam along the link path (Upton et al 2005, Overeem et al 2011, Chwala et al 2012, Doumounia et al 2014, Messer and Sendik 2015). Hence, a reliable classification of wet and dry periods is needed to prevent erroneous rainfall estimates during dry periods (Overeem et al 2011, 2016b, Polz et al 2020). Moreover, this is also needed to determine a reliable reference signal level, which is representative for dry periods. Here, we apply a method which considers the mutual decrease in received signal levels from surrounding links to determine whether it is raining or not. Second, antennas may also get wet during rain. Hence, a wet antenna attenuation correction is commonly applied, to avoid overestimation due to a water layer on the antenna(s) (Kharadly and Ross 2001, Leijnse et al 2007a, 2007b, Schleiss et al 2013). Systematic descriptions of encountered sources of error and methods to account for these using quality control and rainfall retrieval algorithms are provided by Overeem et al (2016a, 2016b) and Graf et al (2020).

Since approximately 2005, a growing community of researchers (Gosset et al 2016) has, in close collaboration with mobile network operators, developed retrieval algorithms to convert the raw link signals to rainfall estimates. So far, this has resulted in tens of peer-reviewed scientific publications, which demonstrate the potential of link rainfall monitoring in the Czech Republic (Fencl et al 2015), France (Schleiss et al 2013), Germany (Chwala et al 2012, Graf et al 2020), Israel (Messer et al 2006, Rayitsfeld et al 2012), Italy (Roversi et al 2020), the Netherlands (Leijnse et al 2007a, Overeem et al 2013, 2016b, De Vos et al 2019b), and Switzerland (Bianchi et al 2013a). A key advantage is that the required infrastructure is already in place, in the form of the ∼5 million links in global use (Ericsson 2018) by mobile network operators, who typically store the required signal level data in their network management systems to monitor network stability. Typically, minimum and maximum, average, or instantaneous received signal level data are stored every 15 min.

Currently, link rainfall estimation has been primarily demonstrated for temperate and Mediterranean climates, with generally reasonable coverage by dedicated sensors. Datasets with a few thousand links were analyzed over a 2.5 year (Overeem et al 2016b) and a 1 year period (Graf et al 2020) over the Netherlands and Germany, respectively. However, the greatest potential of link rainfall estimation is in low- and middle-income countries with subtropical or tropical climates, with limited real-time rainfall observations to capture the typical localized and intense rain showers. Rain gauge data will not always be automatically disseminated. For instance, charts from the self-recording pluviographs in Sri Lanka, used in this study, need to be manually read and digitized. Only few studies address link rainfall estimation in these regions. Good performance is found by Doumounia et al (2014) for one sub-link in Burkina Faso over a 2 month period and by Sohail Afzal et al (2018) for 35 sub-links in Pakistan over a 55 day period. Hoedjes et al (2014) show a rainfall map based on data from six sub-links in Kenya and compare it to a map of satellite brightness temperatures. Rios Gaona et al (2018) provide rainfall estimates for Brazil, based on 145 sub-links (95 link paths) over 81 days. City-average rainfall dynamics for São Paulo were captured, but mixed results were obtained for individual link estimates. Finally, pilot studies have been performed in Nigeria and Sri Lanka (GSM Association 2019).

To assess the potential of links for rainfall monitoring in tropical regions a thorough evaluation on much larger datasets is required. We contribute to this by employing the open-source R package RAINLINK (2020) to retrieve rainfall in Sri Lanka, a middle-income country. Rainfall in Sri Lanka's central mountains is important for hydropower and the tea industry. The quality of tea is influenced by rainfall amount, intensity and duration (Herath and Ratnayake 2004). Furthermore, strong seasonal and spatial rainfall variability creates periodic water shortages for irrigated agriculture and households. Both rain-fed and irrigated agriculture is important for exports for Sri Lanka. Understanding spatio-temporal rainfall variability and improving rainfall forecasts may help in, for example, planning crop cultivation and drainage channels for flood mitigation, or in designing water storages (Jayawardene et al 2005). Here, we employ link data from, on average, 1140 link paths over a 3.5 month period. Link rainfall maps are compared to hourly and daily rainfall depths from 11 and 16 official meteorological stations, respectively. Given the absence of weather radars and the limited quality of many satellite precipitation products, for the first time link rainfall maps are compared to those from the Dual-frequency Precipitation Radar on board the global precipitation measurement (GPM) core observatory satellite (Hou et al 2014, Skofronick-Jackson et al 2017, 2018) for 12 events. This active satellite sensor will generally provide much better rainfall estimates than those from passive satellite-based sensors, although its temporal coverage is limited. Note that GPM is the successor of the tropical rainfall measurement mission building upon its legacy of precipitation estimation from space from 1997 to 2015 (Liu et al 2012, NASA 2021a, 2021b).

2.1. Study area

2.2. Commercial microwave link data

Minimum and maximum received signal levels over 15 min intervals were obtained for the period from 12 September to 31 December 2019, from part of the cellular telecommunication network operated by mobile network operator Dialog Sri Lanka, covering the majority of Sri Lanka. Figure 1(a) shows the locations of the 1326 link paths for which rainfall estimates are obtained. Information on link path length L, employed microwave frequency f and link orientation is presented in figures 2(a)–(c). The majority of links is shorter than 5 km, the average link path length being 4.7 km. Most links operate at frequencies between 17 and 19 GHz. The orientation of links is quite uniformly distributed over all directions. The employed carrier frequency decreases for increasing link path lengths (figure 2(d)), because link networks have been designed to prevent signal loss and lower frequencies are less affected by attenuation. Higher carrier frequencies are generally used in (urban) areas where more bandwidth is needed. Frequencies are generally lower compared to those in areas with a temperate climate, in order to prevent signal loss due to the more intense tropical rainfall. Figure 2(e) shows that the mean 15 min data availability varies in time, but is generally high. Rainfall estimates have been provided for, on average, 2134 sub-links and 1140 link paths (this is less than the 1326 link paths for which data are available due to filtering we apply on these data) over the 3.5 month period.

Zoom In Zoom Out Reset image size

Link metadata are coupled with signal level data and an initial quality control is performed. Overeem et al (2016a) give a detailed description of the employed rainfall retrieval algorithm, the interpolation methodology and the implementation thereof in the RAINLINK software package, written in scripting language R. Version 1.2 is used to retrieve 15 min path-averaged rainfall intensities, which are subsequently interpolated to 15 min rainfall maps on a 0.02^∘ (∼4 km²) grid. These are aggregated to hourly rainfall maps and to daily rainfall maps from 08:30 to 08:30 local time (03:00 UTC) to match the measurement interval of rain gauges. RAINLINK's default parameter settings (Overeem et al 2016a) are employed. Although these have been optimized for the Netherlands, which has a temperate climate, we expect that these settings will yield acceptable rainfall estimates. The entire processing chain is described in the supporting information (text S1) (available online at stacks.iop.org/ERL/16/074058/mmedia).

2.3. Rain gauge data

2.4. Satellite product

Figure 3 shows the cumulative daily rainfall depths from links and gauges from 12 September until 31 December 2019, the period being shorter in case of missing rain gauge data. Note that cumulative rainfall can easily exceed 800 mm in 3.5 months, and can even exceed 1500 mm. The variation with time of the rainfall depths from links and gauges agree reasonably well. The magnitude is in good agreement for six locations, reveals overestimation for one location, underestimation for four locations, and severe underestimation for five locations.

Zoom In Zoom Out Reset image size

A more quantitative evaluation is provided by the scatter plots of link hourly and daily rainfall depths against the gauge-based ones (figure 4). A slight overestimation by links is found for hourly rainfall, whereas for daily rainfall the average underestimation is 17.6%. This is probably caused by differences in availability and collection of hourly and daily gauge data. The coefficient of determination (r²; the fraction of explained variance) is fairly large (0.57), although quite some scatter is found, as is confirmed by the coefficient of variation (CV) of the residuals of 4.33. Here, the CV is defined as the ratio of the standard deviation of the residuals and the mean of the reference. This scatter and the apparent large link rainfall depths for (near) zero gauge rainfall (false alarms) can be a result of errors in link rainfall retrievals, but may also be related to the interplay between strong spatial rainfall variability and differences in link and gauge locations. Furthermore, gauge rainfall estimates themselves are influenced by sources of error. Except for the underestimation, link rainfall estimates vastly improve for daily rainfall (r² = 0.79 and $\mathrm{CV} = 0.87$), where spatial rainfall variability is generally much smaller, such that differences in locations between links and gauges are expected to play much less of a role. It is remarkable that links can even accurately capture daily rainfall depths over 150 mm.

Zoom In Zoom Out Reset image size

Finally, 15 min link rainfall maps are compared to the satellite product for 12 events (figure 5). In most cases the locations of rain areas agree reasonably well. One of the best matches is found for 17 October 2019 (figure 5(e)): The entire event of link rainfall maps is provided as supporting information (movie S1) and shows realistic patterns of showers moving across Sri Lanka during this Second Intermonsoon. Link and satellite rainfall maps may differ because the satellite only takes a snapshot during its overpass, and hence does not provide an estimate for the entire 15 min interval. Moreover, low link network density in some areas may lead to lower-quality link rainfall estimates.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

It is rather speculative what causes differences in performance among gauge stations (figure 3). Only for one location the large distance (indicated by the red dots in figure 3(k)) between the gauge and the nearest link may have contributed to link rainfall underestimation. Although differences in distance to the nearest link do not seem to provide an explanation, link network density could differ between rain gauge locations. Higher densities could capture local extremes more accurately. Possibly some locations have shorter link paths, which may negatively affect results (De Vos et al 2019b). Differences between link and gauge rainfall estimates in figures 3 and 4 may also be caused by sources of error in gauge rainfall observations, which may vary between gauge locations. For instance, daily rainfall depths are based on eight manual observations and can hence be affected by reading errors. Rain gauges can malfunction due to dirt (Steiner et al 1999) and turbulence can lead to undercatch (Pollock et al 2018). A more extensive reference network would be needed in order to draw more definite conclusions regarding link-gauge differences.

When only gauge estimates above 1 mm are considered (i.e. a subset of figure 4), an underestimation of 15.6%, a r² of 0.58 and a much better CV of 0.86 is found for hourly rainfall (1422 values). For daily values over 1 mm we find an underestimation of 18.6%, a worse r² of 0.75 and a better CV of 0.63 (929 values). This performance for daily rainfall above 1 mm is better than what was found for the Netherlands (∼35 000 km²), except for the stronger underestimation: Overeem et al (2016b) report an underestimation of 7.8%, a r² of 0.59 and a CV of 0.67. This is based on rainfall maps from a dense link network spanning about 7 summer months, with a gauge-adjusted radar rainfall dataset as reference. The current results are quite remarkable since RAINLINK's parameters have not been optimized for the Sri Lankan climate. We think this can be partly explained by the generally higher rainfall intensities in Sri Lanka. Then the influence of wet antenna attenuation and errors in wet-dry classification will be relatively small. Still, it is recommended to customize the rainfall retrieval algorithm settings for local climate and network conditions instead of using the parameter values found for the Netherlands, which has a temperate climate. This customization concerns the core of the algorithm: the coefficients of the power-law between path-average rainfall intensity and specific attenuation, and the mixing of minimum and maximum received signals to derive average rainfall intensities. The power-law coefficient and exponent for a given microwave frequency are derived from measured rain drop-size distributions and simulations of electromagnetic scattering by rain drops (Leijnse et al 2008). This requires drop-size distribution measurements from the considered region or a similar climate. The relative contribution of the minimum and maximum received signals depend on the probability density function of path-averaged rainfall intensities typical for the region. This parameter can hence be derived from such probability density information, but also by means of a statistical calibration by comparing to a reference dataset (Overeem et al 2016a, De Vos et al 2019b). Such a calibration can also be used to optimize other parameters for the region, preferably by using a stochastic optimization that optimizes RAINLINK's most important parameters for wet-dry classification and rainfall retrieval separately (Wolff et al 2021).

Messer and Sendik (2015), Uijlenhoet et al (2018) and Chwala and Kunstmann (2019) provide detailed discussions on (the sources of error of) different rainfall retrieval and mapping algorithms, where Overeem et al (2016a, 2016b) specifically focus on RAINLINK and its applicability. To gain a better understanding of differences between link and reference data, notably the underestimations in cumulative rainfall against gauges for several locations, would require a detailed study of sources of error. This could be accomplished by employing local weather station data and a dedicated research experiment with rain gauges and disdrometers (instruments that measure raindrop size distributions) along a link path combined with time lapse cameras to monitor the link antennas and the link path (Van Leth et al 2018). Such an experiment would also provide opportunities to improve link rainfall retrieval algorithms (Wang et al 2012).

These are the main conclusions from this study:

• An unprecedented CML dataset in terms of spatial and temporal coverage was used for rainfall mapping in the tropical country Sri Lanka.
• A spatial comparison with a high-quality satellite product (Grecu et al 2016) and an extensive local comparison with rain gauge data confirms the potential of microwave links for detailed tropical rainfall monitoring over land.
• The density of the link network is generally much higher than that of gauge networks providing sub-daily data. For example, the Sri Lanka Department of Meteorology operates 23 official meteorological stations providing hourly rainfall for the entire country, whereas data from, on average, 1140 link paths were available in this study. This is only part of the network of one of the mobile network operators active in Sri Lanka.

Only satellites can provide precipitation information on a global basis, albeit with issues concerning accuracy (Kidd and Huffman 2011). For instance, the IMERG product of the GPM mission is a gridded rainfall dataset covering 60^∘ N–60^∘ S with a spatial resolution of 0.1^∘ and an interpolated temporal resolution of 30 min (Hou et al 2014). This coverage cannot be achieved with CML, rain gauge or radar networks, also because they are virtually absent above the 71% Earth's surface consisting of oceans. The advantage of cellular telecommunication networks is that they are ubiquitous in populated areas, covering more than 90% of the world's population (International Telecommunication Union 2020). In 2007, already 20% of the Earth's land surface was covered by cellular telecommunication networks (GSM Association 2012). Hence, CMLs in these networks could provide accurate and much more detailed rainfall information in populated areas, where the impact of severe weather is expected to be largest.

Links are a complementary source of rainfall information and are not meant to replace existing observational networks. Once the quality of link rainfall estimates has been established for a certain region, these are ultimately merged with rainfall products from dedicated sensor networks (Grum et al 2005, Overeem et al 2012, Bianchi et al 2013b, Haese et al 2017). For developing countries in subtropical and tropical areas, link and satellite rainfall products could be merged. In these areas, the high spatio-temporal resolution and accuracy of links is unmatched, complementing the large coverage of satellites. Rios Gaona et al (2017) evaluate the IMERG product of the GPM mission, two geostationary rainfall products and link-based rainfall estimates against a gauge-adjusted radar rainfall dataset over the Netherlands. The 7 month evaluation shows that links outperform satellite rainfall products, highlighting the potential for improving satellite products through merging with link rainfall estimates.

A number of hurdles have to be overcome before (merged) link rainfall products can be used operationally. First, gaining access to link data is usually difficult and is typically achieved by contacting mobile network operators on a country-by-country basis. Here, we obtained link data from Sri Lanka by the active involvement of the Mobile for Development AgriTech program of the GSM Association. The GSMA represents the interests of mobile network operators worldwide. Ideally, link data should be openly available to enable the development of rainfall products. Support from the World Meteorological Organization, the International Telecommunication Union, space agencies, meteorological satellite agencies and policy makers could foster this.

Second, several technical issues need to be addressed. Metadata (comprising link locations, lengths, frequencies, polarizations, etc) are often not present in the link data files, while such data are crucial in converting link data to rain estimates. Coupling of available metadata to the data files is challenging and time-consuming because of large variation of storing metadata between mobile network operators, and the sometimes ambiguous or missing information therein. This calls for standardization of link (meta)data at the link vendor level, for which the preliminary HDF5 data format standard 'cmlh5' could be used as a starting point (cmlh5 2020). Additionally, the data management of mobile network operators is not geared toward (continuous real-time) delivery of link data to other parties. Ultimately, an application programming interface (API) should be developed which can handle link data acquisition, the coupling of data and metadata, the rainfall retrieval and the dissemination of rainfall information. Implementing an open source data acquisition system may help to gain link data more easily and would increase control of the involved sampling up to 1 s (Chwala et al 2016).

Third, in order to perform a thorough scientific evaluation of their performance, rainfall retrievals from the typically hundreds to thousands links from preferably one or more years need to be evaluated for each network and region. Also, initial quality control of (meta)data may be needed. This requires good reference data, which are especially scarce in subtropical and tropical areas. A satellite combined radar-radiometer product provides a means to perform a spatial comparison, but typically only once a day. Also customizing retrieval and mapping parameters to the local climate and network is recommended, although we have shown here that using standard parameters derived for a very different region (the Netherlands) yields remarkably good rainfall estimates.

Fourth, business models need to be developed. Link data will feed into a range of applications, from improving and localizing weather forecasting services, early warnings (e.g. landslides), crop production monitoring, precision agriculture services, agricultural weather insurance and water management. For example, this data will allow national meteorological services or private weather companies to accurately nowcast rainfall up to a few hours in advance, which is currently not feasible given the absence of weather radars. Imhoff et al (2020) use link rainfall maps as input for the open-source algorithm pySTEPS and demonstrate that link rainfall nowcasts compare well to radar rainfall nowcasts over the Netherlands. In addition, engineering companies and water authorities could use these data to improve (flash) flood forecasts (Brauer et al 2016). Weather micro-insurance products, specifically rainfall excess insurance, can be developed to cater to the predominantly smallholder farming systems found in low- and middle-income countries. These applications present opportunities for mobile network operators to valorize their link data, either as a 'data as a service' product, or by co-developing the above-mentioned services with relevant partner organizations, where other mobile network operator assets, such as delivery channels, location data, and mobile money channels facilitate the creation of such services (GSM Association 2019, 2021). This is also confirmed by the demonstrator project Microwave-based Environmental Monitoring (MEMO 2020) from the Swedish Meteorological and Hydrological Institute and Ericsson, demonstrating live link-based rainfall maps for Gothenburg and Stockholm (Sweden).

To conclude, we hope that this study contributes to enable real-time link rainfall monitoring, especially in those subtropical and tropical regions around the world with few surface rainfall observations.

The satellite product, the global precipitation measurement (GPM) combined (CMB) precipitation product (based on the GPM Combined Radar-Radiometer Precipitation Algorithm, version V06A), can be freely obtained via https://gpm.nasa.gov/data/directory. The gridded rainfall maps retrieved from CML data from Sri Lanka over the 3.5 month period can be freely obtained (Overeem 2021). The data generated and/or analysed during the current study are not publicly available for legal/ethical reasons but are available from the corresponding author on reasonable request.

We thank Jenny Prosser (formerly GSMA) for establishing contacts with mobile network operator Dialog Sri Lanka. We are indebted to mobile network operator Dialog Sri Lanka for providing the commercial microwave link (meta)data and for their assistance in interpreting the (meta)data. We thank Matthew Strickland (GSMA) for obtaining the rain gauge data from the Sri Lanka Department of Meteorology. We acknowledge the Sri Lanka Department of Meteorology for their help in answering questions regarding the rain gauge data and for providing a photograph of a rain gauge. We thank two anonymous reviewers for their constructive comments.

This project was partially funded through the Enhancing Digital and Innovations for Agri-food Systems and Livelihoods (eDIAL) Programme of UK aid. It was granted to the GSMA Foundation (IATI Activity ID: GB-GOV-1-300644-104), with Wageningen University and Research as subcontractor.