Assessing uncertainties in sea ice extent climate indicators

The approach we use is to simply look at an ensemble of estimates from different products. We assume that no passive microwave product is perfect and no estimate will be 'correct'. However, by comparing extent estimates of several different products, we can use the variation between the estimates to provide a range of extent values into which the true extent is likely to fall. Of course, this is not necessarily the case: all of the estimates may be biased high or low. But the ensemble of extent values at least provides a reasonable range of possible extent values.

As noted above, there are several sea ice extent products published, either as graphical time series and/or provided as text data. The different products used here are shown in table 2.

We note that this exercise is not an algorithm inter-comparison, which has been done by others (e.g. Ivanova et al 2015). The purpose here is to inter-compare the published extent estimates as the data providers have processed them. So, we use the products as-is and do not control for differences in processing or input data. Thus, some differences in the extent values are likely due to non-algorithmic choices (e.g. land mask, projection, etc.) made by the providers. However, such differences are relatively small compared to the extent and should be consistent throughout the time series.

In assessing the differences, we use the NSIDC SII estimates as the baseline. This is for consistency with the next section on relative uncertainty. It does not suggest that these are more or less correct than others, but simply to provide a consistent measure for comparison of the different products.

The products are compared over a three-year period, from 2015–2017. This period was chosen to coincide with the next section on relative uncertainty. Though relatively short, this is long enough to clearly show the differences between the products at different times of the year. It shows a general consistency in the differences through the years (for a given season). Comparisons have been conducted on products over a longer period to compare long-term trends (e.g. Comiso et al 2017), but that is not the goal of this study.

The extent estimates from the product all clearly track the seasonal variability of the ice cover, from a maximum in winter through the summer minimum (figure 1). However, there are clear differences in the estimates from the products and these differences vary seasonally (figure 2). These relate to two primary aspects of the products. First is the sensitivity of the algorithms to emission by different ice conditions, most notably thin ice and surface melt. Surface melt generally reduces concentration (the algorithm interprets the liquid water as open water) and thin ice is underestimated because of emission from the water beneath the ice (Steffen et al 1992). Second, as noted above, the spatial resolution is a key factor in the estimate of extent. Higher resolution products, such as those from AMSR2 will more precisely detect the edge, other factors being equal.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

One outlier in the extents is the Ocean and Sea Ice Satellite Application Facility (OSISAF) values in the Arctic, which are as much as 1.5 × 10⁶ km² above the SII estimates during summer, while other estimates vary within ∼500 000 km². The reason for this is that the OSISAF version 1 concentration fields do not employ GR weather filters, but instead use atmospheric corrections based on numerical weather model output (Tonboe et al 2016). These corrections remove some weather effects, but some remain, evident as low-concentration (slightly >15%) ice in water regions off the ice edge. These concentrations contribute to the extent, biasing the OSISAF values higher than other products. We note that a soon-to-be-released version 2 of the OSISAF product (Lavergne et al 2019) will include a concentration field with GR filters applied.

Of particular interest are the annual maximum and minimum extent values, especially the Arctic minimum. The focus on the Arctic minimum is due to the significant long-term downward trend observed over the record (since 1979) and the record low minimum extents in 2007 and 2012. It also defines the extent of the multi-year (perennial) sea ice cover. Reduced summer sea ice cover is also of great interest to stakeholders in the region. The minimum and maximum extent show fairly consistent behavior between the products (tables 3 and 4). The Japan Aerospace Exploration Agency (JAXA) is consistently lowest, while OSISAF is highest or second highest (due in part to the weather effects). The SII values are generally close to the middle range of the products. The spread of the extents is nearly one million square kilometers. Roughly half of the spread in the minimum is due solely to the OSISAF product. In winter, both OSISAF and MASIE are higher than the others. The low JAXA values are likely due to the higher spatial resolution, resulting in less 'smearing' of the ice edge; however, this doesn't necessarily mean that JAXA is more accurate—it may miss thin or melting ice and underestimate the ice edge location.

In the Antarctic (table 4), the range in extents is smaller, perhaps due to the fact that OSISAF and MASIE values are not available for the Antarctic. The minimum extent spread is smaller, ∼250 000 km², than in winter (∼450 000 km²) perhaps simply due to less total ice overall. The larger extent range in winter may be due to the same phenomena—larger extent allows for more variability. There is also a less clear-cut difference between the products with the highest and lowest values varying between maximum and minimum and even the years. The time series of the difference with SII shows interesting features as well (figure 2(b)). The Bootstrap is consistently higher than SII. Bremen and JAXA differences with SII are closer to zero in general, but Bremen has a noticeable drop in the November to February period. This is during austral spring and summer when ice extent is decreasing rapidly. The Bremen product appears to be affected by surface properties (melt) or atmospheric emission (due to the use of the 85–90 GHz channels), which reduce detection of ice relative to the other products.

While the spread of estimates seems large, it reflects a surprisingly small change in ice edge position due to the large perimeter of the ice cover. As demonstrated conceptually in the supplementary materials, a 1 × 10⁶ km² extent difference reflects a change in ice edge position of only ∼25–75 km (one to three grid cells) depending on the latitude of the ice edge. Also recall that some of the differences, especially those with different gridded resolutions, can be attributed to differences in land masks.

The approach here is to investigate the sensitivity of the extent to variations in the processing: (1) NRT versus final processing, (2) different T_b sources, and (3) the GR open water threshold. We use the SII processing scheme as the basis for this assessment. All extents are based on 5-day running averages of daily values, as is done for the published SII values. This smooths out day-to-day variability and yields a more consistent time series. The different inputs are described below and summarized in table 5.

• 1.  
  NRT versus final: NSIDC archives two concentration products. The first is an NRT product (Maslanik and Stroeve 1999) processed at NSIDC that uses NRT T_bs from NOAA CLASS (http://class.noaa.gov/). The second is a final product processed at NASA Goddard (Cavalieri et al 1996) that uses T_bs from Remote Sensing Systems (RSS), Inc. (version 7, Wentz et al 2013; http://remss.com/missions/ssmi/). The different T_b calibration by CLASS and RSS contributes to differences in concentration. In addition, while the concentration algorithm is the same for both the NRT and Goddard product, there are differences in ancillary processing. Goddard fills data gaps with spatial and temporal interpolation and it uses a different ocean mask to filter out false ice retrievals in regions where sea ice never occurs. Finally, it does manual quality control editing to remove clearly spurious ice. The SII extents are derived from the NRT concentrations for recent data, which are replaced with Goddard final data when they become available.
• 2.  
  T_b source: Currently there are three operating SSMIS sensors, on the Defense Meteorological Satellite Program (DMSP) F16, F17, and F18 satellites, all of which have had overlapping operation for the past several years. Though the instruments are the same design, there will be small differences in T_b values due to sensor instrumentation and calibration, as well as observation time. NSIDC has begun processing concentrations from all three sensors internally, though only one sensor (currently F18) is distributed publicly. Also, as noted above, while NSIDC uses CLASS T_bs for its NRT processing, RSS also provides T_bs (albeit not in NRT).
• 3.  
  Weather filter: as noted above, two weather filters are used to automatically remove erroneous ice over open water due to weather effects and these filters potentially affect extent by removing low-concentration ice (potentially including concentrations biased low by melt). The GR2219 was found to only have a small effect on the ice edge (Meier and Ivanoff 2017), so here we investigate only the sensitivity of extent to GR3719. Extent changes due to GR3719 represent sensitivity to both random noise in the T_b values and the ice conditions near the ice edge. The algorithm tiepoints for pure surface type could also be adjusted, which will affect concentration and thus extent, but the weather filter threshold is a more direct and more easily controlled way to test the sensitivity of extent to algorithm parameters.

With the exception of the F17 Goddard final product, all of the test cases above were processed at NSIDC using the same NSIDC processing system that is used for the operational NRT products. We note that the same tiepoints are used for all NSIDC-processed cases, based on the F17 tiepoint values calculated for the NRT CLASS T_b data (Meier et al 2011). Goddard uses different F17 tiepoints, derived based on the RSS T_b data (Cavalieri et al 2012). The Goddard product also made a slight adjustment to the GR3719 weather filter threshold for the Antarctic, from 0.05–0.053.

On 5 April 2016, the F17 SSMIS data started showing spurious behavior due to satellite issues that affected the 37H channel. The channel began operating nominally again on 10 May 2016. This period has been removed from the analysis, as well as a 5-day period in December 2016. One- or two-day periods with missing or bad data occasionally occurred in at least one source; extent values for these days were interpolated from the extents from surrounding days. Details on the missing values are provided in the supplement. The seasonal cycle of all cases is shown for 2015–2017 in figure 3.

Zoom In Zoom Out Reset image size

The relative uncertainty is estimated by comparing extents from the variant cases with the standard baseline case, which is used as the source for the NRT SII extent estimates. While NSIDC now uses F18 CLASS data for NRT, here we use the F17 CLASS with a 0.05 GR threshold as the baseline. This was selected because (1) it is still (through 2018) used for the Goddard product, (2) it was the near-time-standard for several years until the satellite issues in April 2016, (3) RSS has not yet provided F18 SSMIS data, and (4) NSIDC has not distributed F16 data publicly. So, it is the one configuration that can be compared consistently across all other cases.

The first uncertainty is simply in how much extent changes when the NRT concentration source is replaced by the final Goddard concentration fields. A notable feature in the Goddard differences (figure 4(a)) are small sudden jumps that occur at month transitions (e.g. April to May 2015 and August to September 2015). This occurs due to the difference in ocean masks used in the Arctic to filter residual weather effects far from the ice edge where sea ice is not possible. Goddard uses a sea surface temperature-based monthly climatology (Cavalieri et al 1996). NSIDC found that this climatology was too conservative and allowed too much false ice in the NRT product, particularly during summer with the much lower than average extents in recent years. Goddard does manual corrections to remove such false ice, so the conservative climatology is not a problem for clearing out weather effects—they are removed manually. However, much of the false ice occurs along the coast due to mixed ocean–land sensor footprints. As noted in the supplement, a filter removes some of this effect, but not all of it. The ocean masks are monthly fields and a change in month marks a transition to a new ocean mask. So, the differences in the ocean masks result in a different amount of coastal ice being removed in a given month, which causes the sudden, small jumps in the Goddard extent difference. This is shown in figure 5, where coastal ice in the Sea of Okhotsk and weather effects off southern Greenland are evident in the NRT extent field on 31 August, but not on 1 September.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

While the bias between the Goddard and the NRT daily (5-day running average) values is overall fairly small (table 6), it varies by month and during September in the Arctic it is ∼41 000 km². This is largely due to coastal ice from land spillover, as seen in figure 5. There is also variability (standard deviation) in the daily extent values (table 7), which adds further to the uncertainty. Taken together, the uncertainty in NRT extents relative to the Goddard final values is ∼100 000 km² using a two standard deviation range for uncertainty. Note that this uncertainty only exists for the time when NRT values are used. Once the NRT values are replaced by final values from Goddard (6–12 months later), this uncertainty disappears.

In looking at the sensitivity to T_b, the first noticeable feature is a clear bias in the extents using RSS T_bs (figure 4). This is not surprising because, as noted above, the NSIDC processing uses the same tiepoints, derived for the F17 CLASS data. These tiepoints are not optimal for the RSS data, which results in the bias. Biases are much smaller for the extents from the F16 and F18 SSMIS T_bs. Spatially, the differences in extent are small, mostly scattered points along the coast and the ice edge (figure 6). The F17 RSS case (figure 6(b)) shows the most noticeable difference in the Beaufort Sea near Banks Island, where the ice edge is retracted poleward by a few grid cells. This is due to the inappropriate tiepoints for the RSS T_b input.

Zoom In Zoom Out Reset image size

The extent differences for the GR threshold changes are much smaller in magnitude than the T_b source comparison (figure 7). The bias clearly changes with GR threshold because the threshold allows more or less ice (i.e. it shifts the effective minimum concentration threshold). There is greater sensitivity at lower GR values than at the higher GR values. This makes sense because the lower GR values correspond to a more stringent filter that effectively raises the minimum concentration that is allowed. As it raises that minimum concentration above 15%, the total extent will be directly affected. Raising the GR threshold will filter less weather and will lower the minimum detectable concentration, but there are usually relatively few weather effects and if the threshold corresponds to a concentration <15%, it will not affect the total extent. This is seen in spatial maps of extent where the lower threshold values very clearly cut off ice compared to the baseline case, but higher thresholds have little noticeable effect on the ice edge (figure 8). Antarctic extents are more sensitive to GR, particularly during the major ice growth period of September–December. The higher sensitivity is partly due to the large perimeter of ice as it expands to encircle the entire continent; it also likely reflects a sensitivity to thin ice growth at the edge as well as potentially strong atmospheric effects over the Southern Ocean.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

We note that the chosen range of GR threshold adjustment (±0.02, ±0.04) is somewhat arbitrary. Adjusting it more would obviously increase the magnitude of the extent differences. The rationale for our selection is that the open water points generally cluster within a 0.02 range of GR values (Steffen et al 1992) and when the GR threshold has been adjusted to optimize consistency between sensors, the adjustments have been within ±0.04 (Cavalieri et al 2012, Meier and Ivanoff 2017).

In the previous section, we estimated the sensitivity of the extent to changes in parameters for three different configurations. While the NRT versus Goddard difference is largest (both bias and standard deviation), this uncertainty only applies when comparing NRT extent values with Goddard values from previous years. Thus, we consider this a separate uncertainty from the 'internal' uncertainty that affects any extent estimate.

The T_b source uncertainty is indicated by the variation (standard deviation) due to different sources: F17 RSS, F16 CLASS, and F18 CLASS. The differences are due to the different T_b calibration (RSS versus CLASS) or different orbit/sensor properties (different observation times and CLASS calibrations for F16, F17, and F18).

In general, biases between sensors, even remaining biases after intercalibration, do increase uncertainty when comparing extents between different sensors. But for uncertainty of extents from a given product/source (same sensor, same algorithm), the biases between sensors or processing are not relevant, so we use only the standard deviation as the basis for uncertainty. In other words, using the standard deviation of differences from varying sources and algorithm parameters simulates the uncertainty from a single source and set of algorithm parameters.

In terms of estimating total uncertainty, the most straightforward approach is to assume the uncertainties from the T_b sources and the weather filters are independent. In practice, the GR threshold will be affected by the T_b source, but for simplicity we ignore this. To calculate a total uncertainty, we first average the T_b uncertainties (since all configurations are simply different ways of estimating the sensitivity to T_b). Next, we average the GR threshold uncertainty from the ±0.04 GR range. Then we estimate the total uncertainty via a sum of squares of the T_b source and GR average standard deviations (equation (1)):

$$\begin{eqnarray}&&{\sigma }_{Total}=\,\sqrt{{\left(\overline{{\sigma }_{F17RSS},{\sigma }_{F16},{\sigma }_{F18}}\right)}^{2}+{\left(\overline{{\sigma }_{GR0.046},{\sigma }_{GR0.054}}\right)}^{2}}.\end{eqnarray} \tag{ 1 }$$

The total uncertainty is calculated as an overall average for the entire three-year time period and for the months of September and February (Antarctic) or March (Arctic). These pairs of months are of particular interest because it is when the minimum and maximum extents normally occur (table 8).

A common standard for an uncertainty is the 2σ range, the rightmost column in table 8. As noted above, this an uncertainty for a constant sensor and constant processing method. For the NSIDC SII, these uncertainties would be most valid for comparing extents with the same processing method (Goddard or NSIDC NRT) and the same sensor (F17). Currently, the SII values from 2008–2017 are based on Goddard values processed using F17.

Of particular interest to scientists and the general public is the September minimum extent in the Arctic. One application of these uncertainties is to assess these September minimums in light of the calculated uncertainties. Table 9 shows the ten lowest minimum extents (through 2017) with the rankings considering the uncertainty range. Our assessment is that years that have extents within 2σ of each other should be considered to be tied.

As discussed above, these uncertainties assume a consistent sensor and processing. So, they are most valid for comparing extents from a single source. Transitions between different sensors introduce further uncertainty, which depends on the quality of the intercalibration. Meier et al (2011) found that these inconsistencies were in part due to limited overlap periods. In a later study, Eisenman et al (2014) noted inconsistencies in the time series attributable to intercalibration, particularly in the early part of the record. Unlike early transitions, the F13 SSMI to F17 SSMIS intercalibration had a full year of overlap to derive adjustments. This led to a high-quality intercalibration and only very small errors (Meier et al 2011, Cavalieri et al 2012). Another concern is 'sensor drift'. Over time, the orbits of satellites will change due to atmospheric drag. This will change observation times and altitude. These could change the performance within a sensor over time. Fortunately, F13 and F17 had only minimal orbital drift, so this error should be small. Thus, there should be good confidence in the above-derived uncertainty estimates since the beginning of the F13 SSMI in 1995.