Very Short-Term Solar Irradiance Forecasting at a Sub-Minute Scale Based on WT-Cnns

with the increasing of penetration of photovoltaic power, the solar irradiance forecasting has become a concern for power management. Due to the non-stationary and stochastic feature of the solar irradiance in a very time-term scale, the solar irradiance is difficult to accurately forecast. In this paper, the second-level solar irradiance is predicted using a deep-learning-based convolutional neural network (CNN) with the help of wavelet transformation (WT) and bootstrap sampling methods in time-frequency domain. The proposed method extracts the useful information from the reconstructed solar irradiance ‘image’ matrix in the frequency domain to form the next second level solar irradiance. The results show that the proposed model can learn the past features comprehensively, indicating significant potential for further study.


Introduction
Due to the rapid advancement of the renewable energy technologies, photovoltaic (PV) power has become a promising alternative for solving the current energy and environmental crises. The very short-term volatility and variability of solar energy impose a high level of uncertainty into PV power generation, which leads to the challenges with respect to power balancing, maintaining the system frequency, power qualities issues and power scheduling complexities [1].
In time, obtaining sub-minute or minute temporal solar irradiance variability in advance is an open problem [2]. The existing global horizontal irradiance (GHI) prediction methods in time sequences has the obvious distinction. Hourly or longer estimation usually adopts the meteorological data-based methodologies [3] by digging the statistically significant relationships utilizing antecedent solar intensity and meteorological variables in time sequences [4]. Sub-minute or minute scale evaluation tends to rely on the ground-based sky images [5]. High resolution is generally favorable in order to refine the ramp-rates [6,7]. The very short-term forecasting based on the ground-based sky images has been attracted widespread attention in the existing literature. The basic and common routine of this method consist of two steps: sky image classification and solar irradiance prediction [8]. In details, the sky image classification is usually either based on binary clear/cloudy image region or a complicated identification of cloud types by color and texture features. The cloud motivation vector prediction based on the extraction of sky image features is the core of the solar irradiance prediction and then, the statistical method combines the computed image data features and existing irradiance data features to form the prediction sets. Without hesitation, this method improves the ability to detect mutations in the second or minute scale with no extra meteorological data [9]. But the still high price and high storage and computing requirements of the measurement systems blocks the most weather measurement systems to equip [10].
This paper is devoted to addressing the complicated sub-minute solar irradiance forecasting problems based on the convolutional neural network (CNN)-based ensemble approach. The whole prediction method is a hybrid of bootstrap, wavelet transform (WT), CNN, and mean value construction. The job deals with the series of sub-minute irradiance values in the time domain and the frequency domain to explore the useful implications of data. The complicated nonlinear features of solar irradiance in sub-minute time scale can be represented more effectively. Meanwhile, the work introduces the thought of 'image' into solar irradiance forecasting, which makes the two-dimensional convolution operation available in one-dimensional time-series irradiance values. Furthermore, the method digs the maximum similarity in adjacent historical solar irradiance datasets without the need of other types of datasets like meteorological or imager datasets. Anyhow, this work offset a feasible method in the sub-minute irradiance forecasting.

Theoretical overviews
Due to the fluctuant nature of the solar irradiance, the PV prediction possess high variability. Therefore, a CNN-based comprehensive approach is introduced in this paper to dig the deeper level feature information in the solar irradiance sequences to reduce the side-effects on solar irradiance prediction. The sub-minute solar irradiance datasets are first normalized and bootstrapped. The new datasets are decomposed into couple of frequency by WT. CNNs are designed for each frequency and trained to predict the behavior of each frequency. consequently, the wavelet reconstruction and the anti-normalization are used to synthesize the prediction frequencies to obtain a deterministic point forecast of the solar irradiance. Lastly, the PIs are formulated based on basic uncertainties of the model and data. The flowchart of the proposed algorithm is elaborated by Algorithm A.
Algorithm A: proposed algorithm Start 1) input the sub-minute training samples 2) normalization For each bootstrapping sample in total days Wavelet decomposition in the sub-minute scale For each frequency Establish and training the CNN End End 3) input the solar irradiance testing samples 4) normalization and wavelet decomposition For each frequency Forecasting using CNNs End 5) wavelet reconstruction 6) deterministic solar irradiance prediction 7) anti-normalization 8) mean prediction End

Performance criterion
In the study, comprehensive indices for the mean deterministic forecast performance about the mean absolute error (MAE) and root mean-square error (RMSE) are employed as follows: (1)

data description
The data used for the prediction method are based on the solar irradiance measurements of GHI obtained from Oahu measurement Grid at National Renewable Energy Laboratory (NREL) radiometer array in Oahu, Hawaii, USA (21.31° N, 158.09° W) [11]. A map of the radiometer array from the official website is found in Figure 1. Data sample rate for GHI is every 1 second outputs and is from 5:00 to 20:00 HST from 18/03/2010 until 31/10/2011. The array has 18 monitoring stations to measure GHI with the silicon detector, distributed in an area of approximately 1 km × 1.2 km. Firstly, the station DHHL_8 is extracted as the data source due to the lower data error rate and the time period from 7:00 to 17:00 is split to reduce the zero intensity of solar irradiance. The dataset is divided into training, validation and test set to perform the study. The training day set and the validation day set are randomly chosen from 18/03/2010 to 31/07/2011 based on the ratio of 0.9/0.1. the training set is used for training the various CNN models. the validation is used to choose the optimal hyper-parameters and to proceed the early-stopping while training the CNNs. The test set is from 01/08/2011 until 31/10/2011 to estimate the proposed model against some other models. In order to better perform the study, the data is preprocessed by two sub-processes: 1) Normalization: to evaluate the proposed models in a sub-minute scale, all the data are normalized based on x/ . is the maximum of the whole dataset. 2) Forming the 'graph': to predict the following sub-minute solar irradiance, taking the example at one second resolution, the 5×150 'graph' matrix are built and similar to one grey-scale map. As shown in the below 'graph' matrix, the first time period (T0) from t+0 to t+149 is selected as the first line of 'graph'. The second line is formed by the second time period (T1) from t+60 to t + 209 and the rest can be in a same manner. (3)

Design of Bootstrap -WT-CNNs predictive model
A key element to build a prediction model without any other input parameters except for solar irradiance is to fully explore the inherent relevance. In this paper, the date number is set for bootstrapping. During each sampling, 70% of days are chosen and a total of 30 samples are taken. the overall model structure consists of multiple CNNs to dig the feature of wavelet 'graph' matrix to predict the next period wavelet coefficient. So the wavelet 'graph' matrix (MW) is: Where AC is the matrix grouped by five approximate coefficient vectors, DC1 is the matrix grouped by five detailed coefficient vectors from the max level and so on. WT is the wavelet transformation.
To illustrate the detailed process, the AC is chosen as an example. For 30 bootstrapped datasets, 30 CNNs for each wavelet 'graph' matrix is independently trained and estimated. And the estimated process is to predict ac vectors for the next time period T5 and can be expressed as: For each wavelet coefficient 'graph' matrix, 30 independent CNNs are used to predict coefficient vectors for the next time period. So inverse wavelet transform (IWT) reconstitutes the following time period solar irradiance sequence at one second resolution. And the process is expressed as: Where the superscript means the coefficient vector is built based on the i-th bootstrap sampling dataset. G is the calculated solar irradiance sequence from t+300 to t+449. Especially, the predictive solar irradiance in sub-minute scale is from t+389 to t+449. Lastly, the real prediction values are represented by the average value.

hyper-parameter selection and optimization
The proposed model needs to be tuned Whether it's a parameter or an internal structure. The following hyper-parameter for WT and CNN are optimized: 1) wavelet basis function: for the same signal, different wavelet basis functions will give different results. this paper deals with the discrete solar irradiance series. The rate of irradiance change is quite different in different weather. The characteristic of Daubechies wavelet is that with the increase of order, the larger the order of vanishing moment is, the better the effect of frequency band division is. But, it will weaken the features in time domain, increase the computation amount and make the real-time performance worse. So the moderate Daubechies wavelet order of 6 is chosen.  5 2) number of convolution layers and convolution kernel size. limited by the size of 'graph' matrix, the convolution kernel size is set as the frequently used size of 3. For the number of convolution layers, this paper confirms the 4 layers based on experiments by trial-and-error.
3) General learning rate. Normally, too big learning rate does not decrease the gradient and may lead to divergence or oscillation. Too small learning rate may cause a slow drop and increase the number of iterations. And by trial-and-error from 0.1, 0.01, 0.001, 0.0001, 0.00005 and 0.000001, the learning rate of 0.00005 shows a better performance. 4) Dropout. Dropout is a better skill to reduce overfitting and improve the training performance. And by trial-and-error from 0.1, 0.2, 0.3, 0.4 and 0.5, the dropout rate of 0.1 shows a better performance. 5) number of dense layers and kernel size. the dense layers' depth and the size of each layer are parameters that needs to be tuned in order to obtain a model that can correctly generalize. After experimenting, four layers and a basic size of kernel of 1024, 512, 256 and 128 are confirmed. But in special condition, the output length is bigger than 128 and the parameter may be adjusted appropriately In addition, the tips of batch-normalization and dataset shuffle is added in the CNN structure

Training
In this study, the objective algorithm and other auxiliary are developed based on core i7 @ 4.0GHz and 16GB memory computer. And the GPU is based on NVIDIA GeForce GTX 1660 with 6GB memory. The CNNs is trained on both CPU and GPU by minimizing the mean absolute error by the optimization algorithm 'Adam' based on Python 3.7 library Tensorflow-gpu v2.0.0 For training each CNN, about average 3 minutes are spent and all prediction process of one-time period is lower than one second.

Results and discussion
To explicate the accuracy and effectiveness of the proposed model, sub-minute with one second interval is firstly conducted. Sub-minute scale prediction is used to forecast the GHI in the next minute. Figure 2 describe the further details in a second scale from 11:50:00 to 11:59:59. Accordingly, Figure 2 shows the values of evaluated indices at the four days. To be illustrated, in the testing days, the absolute clear days is near-nonexistent. During the relatively stable time period like in figure 2(a) from the whole day A, the value of MAE is 1.63 and RMSE is 2.0 W/m2. From the whole day, the solar irradiance distribution in day C and day D are very confusing by naked eyes, which makes the time sequence features extracted from t+0 to t+389 full of non-determinacy compared to t+390 to t+349. Generally, the CNNs learn the habitual nature in the past 8 minutes to predict the approaching one-minute solar irradiance distribution, which means the tendency and features of the last 8 minutes have the greatest effects on the predication results. But the factors, such as cloud movements effects the instantaneous solar irradiance changes in just a few seconds. Like in figure 2(b, c, d), the real solar irradiance can change about 800 W/m 2 and this is not an easy work to dig mutation information just from the solar irradiance time sequence. during the unstable time period like in Figure 2(

Conclusion
In this work, a new approach for sub-minute solar irradiance forecasting is developed based on the ensemble method of bootstrap method, wavelet transformation and convolutional neural networks. This technique evaluates next minute's solar irradiance solely on the potential solar irradiance interaction hidden in the time-frequency domain in a second level minute by minute. The irradiance data in time sequence are linked with each other in the last few minutes in a 'graph' matrix. Under the help of wavelet transformation, the information under the time-frequency domain are extracted by the convolutional neural networks to improve the prediction result just based on the only solar irradiance inputs. Overall, the performance showed a better accuracy in a one-second scale especially in a stable or slow irradiance transformation process. And the following next work theme will focus on introducing some new variables to solve the inefficiency while facing the timeline mutation.