Research on Adaptive Selection Algorithm for Multi-model Load Forecasting Based on Adaboost

With the development of energy Internet technology and artificial intelligence algorithms, smart grids have an increasing demand for the accuracy of load forecasting. This paper uses Support Vector Machine (SVM), Artificial Neural Network (ANN) and K-Nearest Neighbors (KNN) regression models to forecast electricity consumption of local residents and wind power generation of selected enterprises respectively. The electricity load of residents is divided into workday load and weekend load for comparative analysis. Based on Adaptive Boosting (Adaboost) algorithm, this paper proposes an adaptive selection method which further improves the prediction accuracy of the regression model, by selecting appropriate number of sub-classifiers and setting training weights of training data and sub-classifiers to adjust the distribution of training samples. Simulation results demonstrate that even in scenarios of poor data quality, the prediction accuracy of the model with the proposed Adaboost-based algorithm can be enhanced by up to 10%.


Introduction
In recent years, the growth rate of power demand has exceeded the growth rate of primary energy consumption, and electric energy substitution is the basic trend of global terminal energy utilization. In 2018, China's total electricity consumption increased by 8.4% over the previous year, and grid-connected wind power generation increased by 20.1%. It is estimated that China's primary energy consumption structure from 2035-2040 will be dominated by coal, oil and gas and new energy [1]. Load forecasting is an indispensable part of power system planning, operation and dispatch. The accuracy of load forecasting is closely related to the scientific rationality of power grid planning and operation. Improving the accuracy of load prediction can improve the economy and stability of power grid operation and dispatch while ensuring that users can obtain safe and reliable power supply.
Power system load forecasting is generally divided into four types: ultra-short-term load forecasting, short-term load forecasting, medium-term load forecasting and long-term load forecasting. The load forecasting mentioned in this article belongs to short-term load forecasting which refers to forecasting the load of the next day or week. The formulation of daily and weekly power generation plan in power plants is affected by the prediction results.
With the rise of artificial intelligence research trends, researchers have successfully applied SVM [2,3], ANN [4,5], Random Forests (RF) [6,7], Extreme Learning Machine [8] and other machine learning algorithms into predicting power system load. A novel prediction method based on neural network and chaotic intelligent feature selection is proposed in [9]. The proposed feature selection method selects the best candidate input set as the predicted input data. The phase space reconstruction theory of the embedding theorem is used to screen the candidate features, and the correlation between the measured candidate input and the target value is analyzed. The prediction engine uses a Multi-layer Perceptual layer (MLP) to mix Levenberg-Marquardt (LM) and Differential  [10] compares the performance of ANN and RF in hotel HVAC energy consumption prediction. The results show that these two models have considerable predictive power and almost the same applicability in building energy applications. [11] shows a new support vector regression algorithm by mixing the three algorithms of Empirical Mode Decomposition (EMD), Genetic Algorithm (GA) and Particle Swarm Optimization (PSO). The regularized structural risk function is optimized by the hybrid algorithm, which effectively improves the accuracy of shortterm load forecasting. [12] indicates a load forecasting model based on K-nearest neighbors (KNN). The model only uses limited temperature forecasts and integrates three individual KNNs with different neighborhood rules through weighting factors. The results show that the load prediction results of the KNN algorithm in a limited information environment are still practical.
However, the previous studies mentioned above tend to use a single forecasting model or a simple weighted mixture of several single models. In addition, the selected model training data is sufficient and of high quality. The impact of poor data quality in regression model accuracy is ignored.
This paper selects the precise electricity load data of residential users in a certain area and the wind power data of poor quality in certain enterprises. The load prediction performance of a single SVM, ANN and KNN model in two scenarios is simulated and compared. Then, based on the Adaboost theory, an adaptive selection algorithm is proposed to make adaptive enhancement on the single prediction model respectively for higher accuracy by selecting appropriate number of sub-classifiers and setting training weights of training data and sub-classifiers. By adjusting the distribution of training samples in time, the training process of the model pays more attention to the moment when the prediction error rate is higher. Finally, the results of adaptive enhanced load forecasting in different scenarios are analyzed, and the accuracy and practicability of the proposed adaptive selection forecasting algorithm are verified.

Multi-model Load Forecasting Algorithm
In this section, single prediction model of SVM, ANN, and KNN is introduced and applied to forecast residential load and wind power generation.

SVM Model
The core of the SVM algorithm is to find a hyperplane that separates data into different classes. The algorithm classifies the data by detecting an optimal data. The hyperplane sought must meet the maximum boundary between all classes. The data points closest to the hyperplane are called support vectors.
In the SVM regression model, a kernel function is introduced to linearize the training samples. Through the mapping mechanism, the model converts the more complex vector inner product operations in the highdimensional space into the operation of the kernel function in the low-dimensional space. The model also introduces a loss function H to describe the deviation between the predicted value and the actual value after regression. The optimization function is defined as follows: where x denotes the output data, g x denotes the regression function, w and b denote the vector parameters, x M denotes the nonlinear mapping function, ] and] denote the slack variables, C denotes the balance variable, H denotes the loss function.

ANN Model
ANN is a calculation model that connects artificial neurons with the weights (coefficients) that make up the EI2E 2020 IOP Conf. Series: Earth and Environmental Science 610 (2020) 012005 IOP Publishing doi:10.1088/1755-1315/610/1/012005 3 neural network. The process of matching experimental data with neural networks is called training. The neural network is trained by iterative adjustment of weights. This paper uses a multi-layer perceptron (MLP) model based on error back propagation algorithm and gradient descent theory to predict short-term load, and the loss function uses a square error function. Assuming that x denotes the input data, y denotes the output data, k and N denote the time coefficient. Figure 1 shows the structure of the ANN prediction model. Figure 1. Structure of ANN model The ANN model needs enough data to describe the connection between the input and the output, and the peak value of the input data affects the accuracy of the output. The main parameters of the ANN model are the size of the input vector, the number of layers and the number of neurons in each layer.

KNN Model
The core of the KNN algorithm is to find the K nearest neighbors to the new sample. The model measures sample similarity through a distance function, and uses most decision rules to classify and regress new samples. The algorithm relies on user-defined examples and user-defined distance metrics. In this paper, the Brute algorithm is adopted to search the neighbors of the target sample, and the weighted average of the nearest neighbors are applied to calculate the final prediction value. The flow chart of KNN algorithm is presented in Figure 2.

Adaptive Selection Based on Adaboost
In this section, an adaptive selection algorithm is proposed to adaptively enhance the single prediction model mentioned in section 2 based on Adaboost algorithm. By choosing different numbers of sub-classifiers, setting the weights of the training data and the training weights of sub-classifiers, the distribution of training samples in the current iterative training session can be adjusted in time. Thus, the model pays more attention to the moments when the prediction error rate is high, and further improves the prediction accuracy of the original forecasting model during the training process.

Adaptive Selection Algorithm
The Adaboost algorithm improves the voting mechanism of algorithm decision-making based on the traditional boosting algorithm, so that the algorithm pays more attention to the error rate of the sample when it uses sample data for training. Based on the improvement method of the voting mechanism in the Adaboost algorithm, the proposed adaptive selection algorithm introduces weight variables to change the distribution of samples according to the error rate. The case study in section 4.2 proves that such improvement is beneficial to improve the accuracy of regression prediction.
In order to explain the process of the algorithm in detail, N sets of sample data T are introduced in this section. j w denotes the single sub-classifier such as SVM, ANN and KNN.
where j G denotes the regression error rate of the jth classifier, j D denotes the weight of the jth classifier. The training sample weights need to be updated for enabling the classifier 1 j w to pay more attention to the data points that the classifier j w predicts incorrectly. The sample weight update formula is defined as follows: After the weight update, the iterative loop training enters the next round. Once the loop ends, the various classifiers are combined according to their weights to obtain an adaptively enhanced prediction model. The combination formula is as follows: Table 1 shows the pseudocode of the adaptive selection algorithm. Increase the weight of data that is incorrectly predicted by the sub-classifier j w 7: Reduce the weight of data that is correctly predicted

Data Processing
The case in this paper collects one-year electricity load data of 27 residential users who participated in the electricity household condition assessment project in region A of China, and the generator output data of the wind farm in region B of China. The start and end time of resident load data is from 00:00 on November 24, 2012 to 23:30 on November 23, 2013. The load data is collected every half hour, and the daily load data includes 48 time points. Since the residential load data in half an hour is relatively small, only about 0.3kWh, in order to improve the prediction accuracy as much as possible, this paper adds the electricity load data of 27 residential users at the same time to obtain a collective electricity load data set. This collective data set is used as the model historical training data set. The start and end time of wind power data is from 0:00 on March 1, 2018 to 23:57 on March 6, 2018. Wind power data is collected every 3 minutes, and daily wind power data includes 480 time points. Since the daily wind power data of the selected wind farm after 12 o'clock has a large degree of missing data with poor data quality, only the data of the first 240 time points of each day are selected for predictive analysis in the following case study.
This paper selects the root mean square error (RMSE) and the prediction error rate w as the evaluation index of the prediction result. The specific evaluation formula is defined as follows > @ > @

Result Analysis
Considering two different scenarios of resident load forecasting and wind power generation forecasting, this paper first uses a single model of SVM, ANN and KNN to make forecast respectively. Resident load forecasting scenarios are subdivided into workday load and weekend load. The SVM model kernel function selects the radial basis kernel function. The ANN model is a three-layer neural network, the number of hidden layer neurons is set to 10, the excitation function uses a modified linear unit function, the weight adjustment uses the gradient descent algorithm, and the maximum number of iterations is set to 10,000. The nearest neighbor K of the KNN model is set to 5, and the Distance rule is used to determine the weight of the nearest neighbor samples.
Then, in order to further improve the prediction accuracy, this paper proposes the adaptive selection algorithm based on Adaboost to enhance the single model. The number of sub-classifiers (num_estimators) is set to 10 and 30 for comparison. For a single model, num_estimators=0. Finally, use the enhanced prediction model to predict the residential load and wind power generation. Figure 3 shows the results of the residents' workday load forecasting. The blue curve in the figure represents the distribution of the true value of the predicted point, and the red curve is the distribution of the predicted value. Table 2 indicates the summary of numerical analysis on forecast error rate w and RMSE.  It can be seen from Figure 3 and Table 2 that the SVM model has the lowest prediction error rate among the whole prediction results of a single model. After adaptively enhancing a single prediction model, the increase in the number of sub-classifiers leads to a decrease in the prediction error rate of the model. When the number of sub-classifiers is 30, the enhanced SVM model performs best, the prediction error rate is reduced by about 3%, the RMSE is reduced by about 4.4%, and the deviation between the prediction result and the true value is further reduced. The effectiveness of the adaptive selection algorithm based on Adaboost is proven. Figure 4 shows the results of residents' weekend load forecasting.   It can be seen from Figure 4 and Table 3 that the prediction error rate of the SVM model is still the lowest among the prediction results of a single model. After adaptive enhancement of a single prediction model, the model prediction error rate is generally reduced, which proves that the prediction results are indeed optimized. However, the error rate of residents' weekend load forecasts is higher than that of weekday load forecasts. The reason may be that the residents' electricity load on weekends has strong randomness and volatility. The forecasting difficulty of weekend load is higher than that of working days.  Figure 5. Wind power generation forecast results Different from the resident load forecasting scenario, the wind power generation scenario is more random. This paper selects poor quality wind power data, which only contains 6 days of data. Figure 5 shows the curve of wind power forecast results. Table 4 indicates the summary of numerical analysis on forecast error rate and RMSE. In this scenario, the results show that the improvement of prediction accuracy is higher than that of the resident load scenario.  Figure 5 and Table 4 indicates that the model prediction error rate is reduced generally after adaptive enhancement. The prediction accuracy of the enhanced SVM model is improved by about 12%, and the RMSE is reduced by about 18.5%. The result proves that the adaptive selection algorithm can achieve higher accuracy improvement in the case of poor data quality.

Conclusions
This paper focuses on two typical scenarios of residential load forecasting and wind power generation. SVM, ANN and KNN machine learning models are used to predict load and power generation. The adaptive selection algorithm based on Adaboost is proposed to enhance the single prediction model for further improvement of prediction accuracy. The conclusion is summarized as follows: 1) The Adaboost-based adaptive selection algorithm effectively improves the prediction accuracy of a single model. As the number of sub-classifiers increases, the prediction accuracy of the Adaboost-based adaptive selection model is significantly improved.
2) In the scenario with poor data quality, the prediction accuracy of the model with the proposed Adaboost- based algorithm is improved by up to 10% which indicates a better generalization performance of the adaptive enhancement prediction model than the single model.