Short-term power load comparison based on time series and neural networks considering multiple features

In order to achieve a certain balance between power supply and production demand, as well as to ensure the operation of social machinery and multiple markets, accurate power load forecasting is indispensable. A load prediction method based on the Convolutional Neural Network (CNN) - the Long Short-Term Memory (LSTM) and the CNN-gated Recurrent Unit (GRU) is proposed. The multi-feature load forecasting is constructed by the date factor, the weather factor, the load factor and the electricity price factor. The advantage of CNN in feature extraction of the data set is used to establish a high-dimensional relationship with load, optimize the input LSTM and GRU network model, train each group of the neural network model, and output load prediction value. Combining the load data of a certain area with its meteorological factors, the CNN-LSTM and CNN-GRU methods are tested and compared with the single-network models of Back Propagation (BP), LSTM and GRU. By comparing it with other popular algorithms, it is proved that the presented model has superior computational efficiency and prediction accuracy.


Introduction
The report of the International Energy Agency [1] pointed out that, global electricity demand has returned to growth for two consecutive years after experiencing a slight decline in 2020.Global electricity demand will grow by about 2% year-on-year in 2022, which is basically the same as the average growth rate (2.4%) during the five years (2015-2019) before the outbreak of the novel coronavirus, and significantly lower than the growth rate (6%) in 2021.The extreme weather in many regions of the world in 2022 highlights the necessity of strengthening power supply security, so accurate power load forecasting is crucial to enhance the flexibility of electricity systems [2], ensure the continuous safety and reliability of power supply, and maintain the balance of electricity trading becomes crucial.
The paper on load forecasting is separated into traditional and modern load forecasting methods, respectively.The establishment of relevant mathematical theories, the completion of computer programming languages and computing software [3][4] and the high-speed replacement of CPU provide a reliable and convenient platform for load forecasting, which also makes the combination of traditional and modern forecasting methods proposed [5].The neural network method is a hot topic at present.In [6], LSTM and GRU neural networks are used to predict short-term load, and it is proved that the GRU neural network has better accuracy.In [7], a bidirectional cyclic multilayer neural network model is proposed, which uses GRU and LSTM neural networks as the forward and reverse prediction parts of the bidirectional cycle respectively.In [8], the LSTM model weight is optimized through the Attention Mechanism to improve the model prediction performance.In [9], multi-step load forecasting is proposed based on time-aware CNN (TCMS-CNN), which extracts features from the power load sequence to enhance the effectiveness of forecasting.
In this article, CNN is linked with LSTM and GRU, which is denoted as CNN-LSTM and CNN-GRU respectively.We aim to use a variety of meteorological factors as input features to form a highdimensional prediction feature vector model to realize the load forecasting function in this paper.Only historical data is used in most short-term power load studies, and adding weather factors helps to improve prediction accuracy.Other than that, the temporal characteristics of the power load data are also retained in this paper, and the load data and CNN data using meteorological factors are convolved to better obtain additional features of the data, making the input data more closely correlated with the output data, and then LSTM and GRU are introduced again to predict the load of electricity.The results of this paper will help the power system departments to make more accurate forecasting.

CNN-LSTM hybrid prediction model
One-dimensional convolutional neural networks [10] are used to obtain time series data.After analyzing the one-dimensional convolution operation and pooling operation, a combined neural network model of CNN-LSTM is built.The input data is a multi-feature power load data of 48-time steps, of which the features are 6, so the input data dimension is 48 × 6.
A one-dimensional neural network is selected to perform a one-dimensional convolution operation on the above data.The convolution kernel is 7, so the data channel is 7, and the length and width are N × 1. One-dimensional pooling operations are performed on the convolved data to obtain valid new features of the data.Finally, the newly extracted feature structure is input into the LSTM neural network.The LSTM neural network performs time series learning, and the Dense layer strengthens the learning data features to predict and output the predicted value.

CNN-GRU hybrid prediction model
The GRU is developed from the LSTM, which is a variant of the recurrent neural network (RNN).RNN is better at processing time series data and can guarantee the continuity of the input signal by looping.GRU is simplified compared to the LSTM network, and the GRU network consists of two gate functions: new gates and resetters.It can effectively model dynamic time series data in volatile and uncertain data, and the new features extracted from CNN are put into GRU to continue learning the demand pattern and cycle change characteristics in load data.
The processed data is also put into the CNN network, convolved and pooled to obtain new data features.The CNN-GRU neural network is built by Keras, and the network contains two onedimensional convolution layers and two one-dimensional pooling layers.There are two GRU neural network layers, with each layer of neurons being 10.Adam optimizer is selected to optimize the parameters, and the learning rate is 0.01.Adam can optimize the output value with the trained load data.

Configuration of the experimental environment
The experimental environment is based on Windows 11 and RTX3060-6G graphics cards.The language of the algorithm model was chosen as Python 3.9, and the software architecture model of TensorFlow framework combined with Scikit-learn algorithm framework and keras framework was used, and the Adam optimizer was added to plot the implementation through a matplotlib tool.

Analysis of power load data and time series
The data of a certain area from 2016 to 2020 is selected in this paper, which contains dry bulb temperature, dew point temperature, wet bulb temperature, ambient humidity, electricity load, electricity price, etc., as shown in the following Figure 1 and Figure 2: From the analysis above, we can learn that weather characteristics have certain influencing factors on electricity consumption, and the data show obvious cyclical changes, which have a strong correlation with human life and production.Therefore, the electricity consumption data at the previous moment has an impact on the subsequent electricity consumption data, which is a typical feature of time series.The annual peak of electricity consumption is from December to February and from June to August, showing a feature of seasonality.It can be seen that weather factors have a certain impact on electricity consumption, and power load data has obvious timing characteristics.
The LSTM and GRU neural networks are both three-dimensional, and the input style is to input the sample number, the time step, and the feature.The sampling interval of the experimental data used in this paper is half an hour, and 48 points are collected every day.Therefore, the weather characteristics, electricity price characteristics and load characteristics of the first 48 sampling points are taken as the input characteristics of the model to predict the power load value of the 49th sampling point, and the value is continuously sliding.

Model parameter selection
To verify the scientificity and effectiveness of the obtained load forecasting model, a total of more than 80, 000 data for five years from 2016 to 2020 was selected from a region in China, and 80% of the data was selected as the training set and 10% as the validation set and test set, respectively.The CNN-LSTM and CNN-GRU hybrid neural network models proposed in this paper were trained, and the prediction results were compared and analyzed with those predicted by BP and LSTM in terms of prediction accuracy and efficiency under the same environment and conditions.
In the paper, a one-dimensional convolution is used depending on the length of the sequence and its feature positions.This structure of the network model is relatively simple, allowing both a reduction in the number of weights and the direct input of temporal load data into the model, which can reduce the complexity in terms of obtaining features and reconstructing the data.Pooling makes it easier to learn the network and also avoids a degree of overfitting.
Before training the model, the data is cleaned and normalized to the used data.The different features of the load data also lead to different physical scales, which may make some physical quantities have larger values, and these features will be mistaken with more important features in the neural network calculation, and also make the neural network run slowlier, thus increasing the unnecessary consumption of time.Therefore, before the data prediction, the data should be pre-processed by normalization, and all the data features should be normalized to 0-1 using the softmax function.In addition, the corresponding features and labels of the training and validation sets are set to load the corresponding data and labels so that the subsequent neural network can better learn the features of the relevant data.The error calculated by using the output value of forward propagation and the real labels is backpropagated to update the weight parameters of the network so that the neural network can continuously learn the useful information in the load data.512 data is trained into the grid at a time, and LSTM and GRU are trained separately.The load value at the corresponding time and the temperature, humidity and electricity price at the predicted time are taken as input features, and the predicted load value is output at that moment.TensorFlow is used to fit and predict LSTM and GRU.After 40 iterations, the prediction model loss curve is presented in Figure 3 and Figure 4.In the GRU training process, the training matrix inputs the GRU network to predict the desired output results; the output results are compared to the real load value of electricity; the error value of the neural network is calculated.If the error meets the set requirements, then the optimal weights can be obtained and the training can be completed.If the error does not meet the requirements, then the weights are updated according to the gradient descent method, and so on repeatedly until the error meets the requirements or reaches the maximum number of iterations set, and then the training process can be completed.
The conventional GRU neural network uses the stochastic gradient descent algorithm to iteratively update the weight of the neural network.The early convergence speed is slow and the accuracy is easy to decline.Therefore, the Adam optimization algorithm is selected and the learning rate attenuation strategy is added in this paper.The Adam optimizer was chosen, with a learning rate of 0.01.As the number of iterations increases, the learning rate decays as a fractional decay.This method is also used to reduce the problem of convergence curve oscillations in the iterations in order to make the model more stable.

Predict comparison and the analysis of the results
The values predicted by the models were back-normalized, and the results of the comparison between the predicted and real results of the CNN-LSTM and CNN-GRU neural network model algorithms are respectively shown in Figure 5:  1 that CNN-GRU has the best effect, followed by CNN-LSTM and BP.The prediction effects of CNN-LSTM and CNN-GRU models were compared with those of single BP, LSTM, and GRU, and the results are presented in Table 1 1, CNN-GRU has the best effect, in which the average value of MAPE decreased by 0.0108%, 0.0096%, 0.0083%, and 0.0063%, respectively, compared with the other four neural network methods; the RMSE method decreased by 91.7107%, 77.6129%, 66.2675%, and 57.2504; the MAE method decreased by 81.4119%, 76.3765, 69.9261 and 50.0143% respectively compared to the other four methods; the R-squared reached 0.9920, with an improvement of 0.0106 compared to the CNN-LSSTM method, and the fitting effect was better than the other four methods.The combined analysis shows that the method has a significant downward trend in MAPE, RMSE and MAE indicators, indicating that the model has a greater improvement in performance in prediction.

Results
The power load forecasting including temperature, humidity, load and electricity price is constructed.The CNN-LSTM and CNN-GRU networks optimized by convolutional neural networks for long and short neural networks and gated recurrent neural networks are studied.The network is used to predict the load of 0.5 h, 1 h and 24 h in the future, using GRU cyclic network to consider the timing of load characteristics, so that the model has better fitting regression ability and higher prediction efficiency.Experiments show that the proposed method can not only enhance the accuracy of the overall prediction results, but also has a faster model training speed.
From the analysis above, we can learn that weather characteristics have certain influencing factors on electricity consumption, and at the same time, the electricity load data has typical time series characteristics, and the data shows obvious cyclical changes, which have a strong correlation with human life and production.Therefore, the electricity consumption data at the previous moment has an impact on the subsequent electricity consumption data, which is a typical feature of time series.
In this paper, statistics were not made under a large number of experiments, which may be accidental.In follow-up studies, the optimal algorithm will be further sought to optimize the model, and more experiments will be carried out to compare and analyze the model to improve the prediction effect.

Figure 1 .Figure 2 .
Figure 1.Comparison of weather characteristics and power loads

Figure 3 .
Figure 3. Loss value of CNN-LSTM network Figure 4. Loss value of CNN-GRU network

Figure 5 .
Load prediction comparison chart of neural network models In this paper, root means squared error (RMSE) is used because it shows how well the predicted values are distributed around the mean and the mean absolute percentage error (MAPE), as MAPE shows whether the results predicted by the time series model are accurate.It can be seen from Table