DeepCropNet: a deep spatial-temporal learning framework for county-level corn yield estimation

The DCN model proposed in this study is a deep learning framework that uses multiple layers to hierarchically learn the spatial and temporal features from the meteorological and yield data. The DCN consists of two parts: temporal learning module and spatial learning module (figure 1). The input of DCN is a time-series of meteorological factors, including county-level weekly growing degree days (GDD), killing degree days (KDD) and precipitation (PRCP) from corn planted (week 1) to mature (week 20). The output is the corn yield anomaly at the county-level. In this study, an AT-LSTM network is trained with parameters shared for all regions to learn and visualize the general temporal pattern captured from the time-series of weekly meteorological factors. To learn the spatial features in each region and output the corn yield anomaly, region-specific output layers are developed based on MTL as the spatial learning module.

Zoom In Zoom Out Reset image size

The general temporal features are learned by an AT-LSTM network, which consists of an LSTM Net and an Attention Net (figure 1). The LSTM Net has five layers: input layer (one layer), hidden layers (three layers), and output layer (one layer). The input is a time sequence {x₁, x₂,...,x_t} and x_t is a vector which includes meteorological factors [GDD_t, KDD_t, PRCP_t] in the period t during the corn growing season. The hidden layers are three LSTM layers composed of LSTM cells to transport and store information selectively. The dimension of features in each LSTM layer is set as 32 after selection range from 16 to 256. The multiple LSTM layers enable the LSTM Net to generate hierarchical features from layer by layer. Temporally, the feature vector and the cell state vector in each LSTM layer are transferred from the current period (t) to the next period (t + 1). After processing the whole-time sequence data, the LSTM Net calculates the output, h_t, the hidden feature vector of the LSTM in period t. In this study, we define the vector of hidden features, h_t, as the reference of the effect of meteorological factors on corn yield. The calculation process of h_t is as follows:

$$\begin{eqnarray}&&{g}_{t}^{f}=\sigma \left({W}_{f}\cdot \left[{h}_{t-1},{x}_{t}\right]+{b}_{f}\right)\end{eqnarray} \tag{ 1 }$$

$$\begin{eqnarray}&&{g}_{t}^{i}=\sigma \left({W}_{i}\cdot \left[{h}_{t-1},{x}_{t}\right]+{b}_{i}\right)\end{eqnarray} \tag{ 2 }$$

$$\begin{eqnarray}&&{S}_{t}={g}_{t}^{f}\,\ast \,{S}_{t-1}+{g}_{t}^{i}\,\ast \,\tanh \left({W}_{c}\cdot \left[{h}_{t-1},{x}_{t}\right]+{b}_{c}\right)\end{eqnarray} \tag{ 3 }$$

$$\begin{eqnarray}&&{g}_{t}^{o}=\sigma \left({W}_{o}\cdot \left[{h}_{t-1},{x}_{t}\right]+{b}_{o}\right)\end{eqnarray} \tag{ 4 }$$

$$\begin{eqnarray}&&{h}_{t}={g}_{t}^{o}\,\ast \,\tanh \left({S}_{t}\right),\end{eqnarray} \tag{ 5 }$$

where t represents the period t during the corn growth season. ${W}_{f},$ ${W}_{i},$ ${W}_{o},$ ${W}_{c},$ ${b}_{f},$ ${b}_{i},$ ${b}_{o}$ and ${b}_{c}$ are weight matrices and bias vector parameters of forget gate, input gate, output gate and cell state update, respectively. ${S}_{t}$ is the updated cell state vector at period t. ${g}_{t}^{f},$ ${g}_{t}^{i},$ and ${g}_{t}^{o}$ are the vectors generated by these gates. The three vectors consist of numbers range from zero to one, for the operation of filtering information. Zero represents totally removing, and one represents totally reserving. S_t is the LSTM cell state vector to store information across the whole-time sequence. h_t is the hidden feature vector of the LSTM. $\sigma$ and tanh are activation functions.

The Attention Net can generate different attention value ${\alpha }_{t}$ based on the h_t to reflect the importance of hidden feature vector to corn yield in the corresponding period. The structure of the Attention Net is designed as one single-layer full-connected neural network in this study. The final feature vector (H) captured by the AT-LSTM is the combination of attention values of Attention Net and hidden feature vectors of LSTM Net calculated as follows:

$$\begin{eqnarray}&&{\alpha }_{t}=softmax\left({W}_{AT}\cdot {h}_{t}+{b}_{AT}\right)\end{eqnarray} \tag{ 6 }$$

$$\begin{eqnarray}&&H={\alpha }_{1}{h}_{1}+{\alpha }_{2}{h}_{2}+\ldots +{\alpha }_{t}{h}_{t},\end{eqnarray} \tag{ 7 }$$

where ${\alpha }_{t}$ is the attention value, softmax is the activation function to limit the sum of attention value to one, W_AT is the weight matrix of the Attention Net, and b_AT is the bias vector.

As the sum of attention values during the corn growth period equals to one, the distribution of attention values can reflect the importance of hidden features at each period. The higher attention value represents that the hidden features in the corresponding period have a higher impact on the estimated yield. The attention distribution and its association with the corn growth process can provide insights to help us understand how the temporal module work for estimating yield.

MTL is applied to learn region-specific features in the relationships between corn yield and meteorological factors. Considering the meteorological conditions (extreme heat in particular) vary across latitudes (figure S1 is available online at stacks.iop.org/ERL/15/034016/mmedia), the nine states are divided into three regions and the yield estimation in each region is defined as an individual task to adjust the regional meteorological conditions. In the three regions, the LSTM layers and the Attention Net layer of the temporal learning module are shared while the final output layers of the spatial learning module are task-specific (or region-specific). The shared layers extract the common features across different regions, and the task-specific output layers generate weights based on local characteristics for final yield estimation in each region. The yield anomaly (y) is calculated as equation (8).

$$\begin{eqnarray}&&y={W}_{r}\,\ast \,H+{b}_{r},\end{eqnarray} \tag{ 8 }$$

where r is the index of the region, W_r and b_r are the weight matrix and bias vector of the responding regional specific output layer for region r. H are the attention weighted features captured by the AT-LSTM network.

This study focuses on county-level rainfed corn yield in the nine states of the US Corn Belt region from 1981 to 2016 (figure 2). These 743 counties accounted for 61% of the total corn production of the US in 2016 (USDA-NASS 2017). During the 36 years between 1981 and 2016, the annual county-level average yield in this area is 7.9 Mg ha⁻¹. The high-yielding regions are concentrated in southern Minnesota, Iowa, and northern Illinois. All counties in this study have an increasing trend of corn yield, with an average trend of 0.12 Mg ha⁻¹ yr⁻¹.

Zoom In Zoom Out Reset image size

The county-level rainfed corn yield data is obtained from USDA's National Agriculture Statistics Service from 1981 to 2016 (USDA-NASS 2017). To capture the impact of meteorological factors on corn yield, the yield anomaly (yield after detrending) is calculated by linear regression (Lobell et al 2011). To ensure the reliability of the data in our analysis, we design three criteria for data cleaning and processing:

• (1)  
  The county should have more than 12 year yield statistics, which is one-third of the whole research period.
• (2)  
  The county should have an average corn harvest area higher than 476 ha, which is 10 percentile value of the county-level average corn harvest area in the whole sample set.
• (3)  
  The county should have a yield trend that passes the t-test, with a two-side p-value less than 0.05.

There are 848 counties with 28,374 samples in the raw yield data. After the data cleaning and processing, 743 counties (87.6%) with 26,015 samples (91.7%) remain in our analysis. The state-level corn phenology data is based on USDA-NASS (2017). The growing season is defined as the average historical average length, 20 weeks, from the specific planting week recorded in each state to keep the periods consistent.

The daily county-level meteorological data is from the Applied Climate Information System Web Service (ACIS) (RCC-ACIS 2017). The meteorological data used in this study include maximum daily temperature, minimum daily temperature, and daily precipitation. Based on these meteorological data, cumulative meteorological factors are calculated including cumulative PRCP as an index to represent the water supply, GDD as an index to represent effective heat accumulation, and KDD as an index to represent extreme heat accumulation. The calculation of GDD and KDD is consistent with a previous study (Butler and Huybers 2015). During the corn growth period, the weekly meteorological factors are calculated and fed into the model as sequential vector inputs. To improve the model convergence speed, the input is standardized into standard normal distribution.

In this study, two approaches are chosen as baselines: LASSO regression and RF. For the LASSO regression model, the $\lambda$ value is set to 0.0005 by the 10-fold cross-validations based on the training set. For RF, the tree number is set to 2000 and 20 of 60 input variables are randomly taken as candidates at each split. The maximum number of nodes for each tree is limited to 200. The glmnet and randomForest R package are used to train LASSO and RF, respectively. The DCN is implemented on the Python platform using Pytorch library and performed on a Linux workstation (Ubuntu 14.04 LTS) with 128 GB of RAM, and an Nvidia Geforce GTX1080Ti graphics card with 11 GB of RAM.

To evaluate the models' performance in yield estimation, all models are trained using the dataset during the period from 1981 to 2014 and tested using the 2015 and 2016 data. The estimated yield anomaly is added with the yield trend to get the final estimated yield. The RMSE and R² between actual yield and estimated yield in the test set are used as indicators to evaluate the estimation accuracy of these models. To analyze the benefit from the spatial learning module, the temporal learning module is applied individually to estimate yield as a comparison with DCN.

To evaluate the model performance in predicting the interannual variability of corn yield, 36 years are randomly partitioned into six subsets (table S1) to conduct 6-fold cross-validation. The DCN model is trained by the five subsets and tested in the remaining subset. The RMSE and R² of each subset are used to evaluate the model stability.

The DCN model shows its estimation performance at the county-level, with RMSE equals 0.83 Mg ha⁻¹ in 2015 and 0.81 Mg ha⁻¹ in 2016 (figure 3). The error maps indicate that DCN performs well in most counties, with an average absolute error lower than 0.8 Mg ha⁻¹ (figure 3). While in Missouri, DCN estimates less accurately, with the average absolute error equals 0.88 Mg ha⁻¹. These results can be explained by the high data missing rate (35%) of this state in the test set. A similar pattern that the number of county samples affects model's performance has been reported in previous studies (Li et al 2019). Although there exist few extreme estimation errors, the overall performance of DCN is satisfying which indicates that DCN is able to well capture the yield variability at the county-level.

Zoom In Zoom Out Reset image size

DCN provides the best estimation results with RMSE equals 0.82 Mg ha⁻¹ as compared with LASSO and RF (figure 4). LASSO provides the lowest estimation accuracy with RMSE equals 1.14 Mg ha⁻¹, agreeing with the discovery in previous study that linear methods are limited in simulating the relationship between crop yield and meteorological factors relative to nonlinear approaches (Cai et al 2018). RF provides a better result than LASSO with RMSE equals 1.05 Mg ha⁻¹. Compared with LASSO and RF, DCN reduces the estimation RMSE by 0.23–0.32 Mg ha⁻¹, which demonstrates the effectiveness of the spatial and temporal learning process by neural network based structure.

Zoom In Zoom Out Reset image size

The results show a spatial pattern that all models perform better in northern states than in southern states (figure 4). The estimation RMSEs are correlated with the variances of yield anomaly at state-level, with Pearson's correlation coefficient of 0.79 (figure S2). Similar results have been reported in a previous study (Li et al 2019). The higher variabilities of the yield anomaly in southern states indicate that corn yield is more affected by meteorological stress in the south. For example, the historical meteorological data shows that the average cumulative KDD during the corn growth period in southern states (165 degree day) is 2.2 times higher than that in northern states (76 degree day) in past three decades. Therefore, it appears that the spatial distribution of meteorological stress is one of the driving factors for this spatial pattern.

The DCN model shows its higher estimation accuracy in most states compared with LASSO and RF, particularly in the southern states. In the states in which corn yield is more stable, such as Michigan and Iowa, different models perform similarly and DCN may not be the best approach. While in the southern states in which the corn yield has more variations due to the higher heat stress, DCN performs the best and shows large improvements relative to LASSO and RF. DCN lowers the RMSE in these states by 13.1%–47.1% compared with LASSO and RF. The stable high performance of DCN indicates that it is a practical approach for yield estimation in the regions with more complex and stressful corn growth conditions.

The AT-LSTM captures a general temporal pattern across all states that DCN provides higher attention weights on the features in reproductive stage than vegetative stage (figure 5). The attention values represent the relative importance of the cumulative information captured by the LSTM Net from the previous input data when estimating corn yields. The consistent pattern of attention values implies that the model has the ability to capture the general temporal information during corn growth across the geographical areas, which is essential for the model's generalization in different regions. The weekly average attention value keeps to be 0.02 from week 1 to week 11 (vegetative stage), starts increasing at week 12 (silking stage), and achieves the highest value of 0.16 at week 20 (maturity stage). This result can be explained by the cumulative effect across time. From planted to mature, the effective information accumulates and drives the model to generate increased weights at a later stage. The temporal distribution of attention values shows that the DCN discovers the silking stage as the starting point for increased attention. Cumulative information becomes increasingly important to the yield estimation from silking stage. This indicates that DCN can recognize the key growth phases during the corn growing season through the temporal learning module. Previous studies have demonstrated that corn yield is highly correlated to the rainfall before silking and effective accumulated temperature after silking (Lu et al 2017). The variability of attention values starts increasing from week 15. The increasing variability during the later stage of corn growth possibly indicates that accumulated meteological information have an increased impact on corn yield but may vary among different counties in different years even in the same state. The results illustrate that temporal learning module provides a promising approach to capture the temporal pattern that relates to crop physiological process.

Zoom In Zoom Out Reset image size

The temporal learning module (AT-LSTM) has already achieved a better estimation accuracy than LASSO and RF, with RMSE equals 0.89 Mg ha⁻¹ and R² equals 0.72 (figure S3). This demonstrates the function of the temporal feature learning in crop yield estimation. The results show relatively high estimation errors in Illinois, Indiana, and Ohio (figure 6). These overestimated results may result from the spatial variances in these states that are not captured by the temporal learning module.

Zoom In Zoom Out Reset image size

The spatial learning module improves the estimation performance relative to the temporal learning module, where overall RMSE in the test set is reduced by 0.07 Mg ha⁻¹ (7.9%). All regions in 2015 and 2016 reach a better estimation accuracy with the improvements range from 1.7% to 11.2%, as the regional RMSEs are reduced by 0.02 to 0.10 Mg ha⁻¹ (figure 7). The most significant improvement is in the central region, with the RMSE reduced by 9.1% in 2015 and 11.2% in 2016. These improvements demonstrate that incorporating the spatial learning module can improve the model performance in regions with various meteorological conditions.

Zoom In Zoom Out Reset image size

The spatial learning module brings improvements by reducing the relatively high estimation errors produced by the temporal learning module. The absolute estimation errors can be reduced by up to 1.15 Mg ha⁻¹ with a mean equaling 0.05 Mg ha⁻¹ at the county-level in 2015 and 2016 (figure 7). The largest improvements concentrate in counties of northern Indiana and northwest Ohio in 2015, with absolute estimation errors reduced by up to 1.15 Mg ha⁻¹ (figure 7). The individual temporal learning module overestimates the corn yield in these counties with estimation errors range from 0.75 to 2.92 Mg ha⁻¹. These overestimations can be explained by a localized pattern that these counties suffer excessive rainfall during the corn growing season in 2015, with their precipitation (669 mm) is 1.5 times the historical average (460 mm). These localized improvements indicate that the spatial learning module could possibly capture the local specific pattern (e.g. excessive rainfall; drought; extreme high temperature), to improve yield estimation.

The DCN model outperforms RF and LASSO at estimating interannual variability in corn yields, with the lowest RMSE and highest R² in every subset of the 6-fold cross-validation (table 1). Among these subsets, the RMSE of DCN ranges from 0.84 to 1.18 Mg ha⁻¹ and the R² ranges from 0.71 to 0.86. The DCN model shows the highest performance stability, with the lowest standard deviation of RMSE equals 0.13 Mg ha⁻¹, compared with RF (0.23 Mg ha⁻¹) and LASSO (0.16 Mg ha⁻¹). The DCN model shows the largest performance improvement compared with RF and LASSO in subset 5, which includes the year 2012. The US Corn Belt suffered a heavy drought in 2012 with an average yield at 6.8 Mg ha⁻¹. These results indicate that DCN produces robust corn yield estimation across a wide range of environmental conditions.