Long Short-Term Memory (LSTM) neural networks in predicting fair price level in the road construction industry

Running a tender procedure in the construction industry a client expects receiving reasonable prices in the tenders. However, the market competition and the contractors’ will to win a contract to work out a profit, sometimes make the prices far from expected. They can be really low, almost impossible to be kept during the contract execution. Oppositely, the offered prices can be far above a client’s expectation. If they are extremely high, a client has to decide, to accept one of them (even bid-rigging is suspected), or to cancel the procedure. Nevertheless, cancelling a procedure means the considerably postponed the start of the construction. The analysis of almost 400 completed tender procedures (in the Polish road construction industry) and the market trends proved that unexpectedly high prices do not always have to point the collusive behaviours of the offerors. As aforementioned analysis is based on past cases, there is a need to propose the predicting tool, to help clients making decisions in the new tender procedures: to accept the high price as a market-based, or to reject all of the offers as a bid-rigging is suspected. The above-mentioned analysis are based on the simple moving average tool. For predictions the autoregressive tool is proposed – long short-term memory (LSTM) neural networks.


Introduction
The unfair pricing (namely bid-rigging) -leading to a forced choice of the overpriced offer -happens all over the world (it is described e.g. in [1], [2], [3]). The Polish market is also not free from this unwanted phenomenon (some examples in the construction industry are described in [4]). The importance of the problem made national and international organisations publishing manuals, how to recognize and avoid bid-rigging in public procurements [5], [6], [7], [8]. If a client observes that the offered prices are much higher than expected, two decisions can be undertaken. Cancelling the procedure -then the expected results are postponed, as well as, the external financing of the project can be lost. Another possible decision is to accept one of the tenders, even if the unexpectedly high price is offered. Then, a client should be sure that the high level of the prices offered is based on other -than collusion or bid-rigging reasons. Otherwise, accepting the offer would be a loss of public funds (the amount exceeding the market price). There is not so long time to work-out this kind of decision. That is why, several bid rigging indicators has been found and published (e.g. in [5], [9], [10], [11]). One of the indicators (to be applied in the road construction industry), based on the relation of the tender price to the client's estimation is introduced in [12] (it was named there -price to estimation). As the price level reflects the market trends (which are observed by the contractors), the simple moving average IOP Publishing doi:10.1088/1757-899X/1015/1/012060 2 (SMA) [13] of is calculated on the 30 days basis. Then based on Bollinger bands concept [14], [15], lower and upper limit of reasonable, market bid prices are defined on the base of 30 ( ) values (see figure 1).
If the upper limit is not exceeded, even a certain bid price is high, the reason for that can arise from the market trends. The monitored period covers 3 years starting from June 1, 2014. Assuming a real case, when the choice of the best tender should be done by a client, the SMA curve, as well as, the upper and the lower limits are unknown, as the choice of the previous procedure was done several days ago. The maximum period between announcing the winner in the procedures recorded is 25 days. For details see table 1 and the histogram in figure 2. The total number of analysed procedures is 383, but 100 of them had a place simultaneously (2 or more in one day). Table 1 contains data concerning periods between 282 tender procedures (being single ones on certain days).   In case of 12 days' time, the possible error can reach -0,2 to +0,2 values, while SMA30(PtE) values vary from 0,38 to 0,83. So, even for 12 days' time the error can be significant (over 50 %). Therefore, to provide clients with the tool allowing for predicting the market level of the bid prices (with possibly low error), the model based on long short-term memory neural networks (LSTM) is introduced.

Data collected
Several types of data are collected about tender procedures for building the sections of roads in Poland. The contracts over 10 million PLN were searched in the period starting from July 1, 2014 to June 30, 2017. Together with the values of the orders, the dates of clients' announcements (about the winners of the procedures) are collected. For over 400 contracts, the clients' estimations of the expected bid prices are available for 382 tender procedures. Then, the price to estimation ratio ( ) is calculated for each case. Based on these data, the 30-days simple moving average is calculated. It consists of 343 cases starting from Aug 1, 2014.

LSTM neural networks
Unlike feedforward neural networks (applied e.g. in [16], [17]), Long Short-Term Memory (LSTM) neural networks introduced in [18] are a variation of recurrent neural networks. They are able to capture long-term dependencies in the data. The unique feature of LSTM networks is that each layer contains four gate sub-layers capable of removing and adding information to the data based on the context.
The main flow of the information in the LSTM is called the cell state, which conveys the data through the neuron with minor changes. As LSTM is a recurrent neural network, the updates to the data and the weights of the network happen in a loop, such that the refining of the weights is dependent mainly on the number of epochs. Parallelly to the cell state, the neuron conveys weights managing the information gain and loss of the cell state. The first sub-layer -forget gate layer -combines the input with the IOP Publishing doi:10.1088/1757-899X/1015/1/012060 4 weights and a sigmoid activation, which results in a vector of values between 0 and 1 regulating how much of the information should be dropped (forgotten) from the data conveyed in the cell state. The network is trained to capture the relevant information from previous data which improves the prediction of new data. The next two sub-layers -input gate layer and activation layer -control which elements of the data in the cell state should be updated with new information. The input gate layer similarly to the forget gate layer uses a sigmoid activation to choose entries to be updated, and then the activation layer generates candidate values that should be added to the cell state. Having generated the vectors of elements to forget and to update, the network generates new data on the cell state as a linear combination of the previous data and the candidate data weighted with the outputs of the sigmoid functions in the sub-layers. Finally, the last sub-layer -output layer -generates new weights which will be used in the next epoch. A third sigmoid function is used to again choose what will be present in the output, and values of that vector are multiplied by the new values in the cell state after applying a chosen activation function.
The popular applications of LSTM neural networks are in language recognition (e.g. [19], [20]). However, predictions made with LSTM of the market processes and trends can also be found [21].

Input data preparation to LSTM neural networks
There were 343 announcements of the winners in the tender procedures in the 1064 days. There were 100 days with two announcements in the same days. So, there were 821 days when SMA30(PtE) couldn't be calculated, and the periods between the announcements vary. As the LSTM networks have to work on data with equal time spans between the variable values, it was decided to interpolate SMA30(PtE) values for each day without announcement. The following two method are applied: constant methods (where SMA30(PtE) remains unchanged from the lately recorded, up to the date of the next announcement of the winner) and linear method described below (see formula (1)).  where: int interpolated value r real value n number of days between k and k+1announcements These processes allowed for making predictions of SMA30(PtE) with the use of LSTM neural networks. As the values of SMA30(PtE) vary from 0,38 to 0,83, there is no need to standardize them.

Results achieved
The dataset consists of 73,5 % cases where the variable is not real, but interpolated. This implies the decision of calculating prediction errors only for days with announcements i.e. for days when the real value of the variable could be calculated. The first tries of forecasting SMA30(PtE) brought two important conclusions. The forecasting based on more than 15 days period does not bring an increase in the accuracy of predictions. Applying the dataset with linear interpolations provides much lower errors of predictions. All the results presented below are achieved for the dataset with linear interpolations, based on a 15-days historical period for predictions. Data concerning the last six months of analysed cases creates the validation dataset (approximately 1/6 of the total number of cases). So, the presented errors are calculated based on real cases from the validation dataset. The time horizon of the forecasts varies from 1 to 25 days. The forecast done for the 25 th day comprise 25 values of predicted SMA30(PtE) from day 1 to day 25. The general rule is confirmed: the longer the forecast horizon, the bigger the error of prediction. For each time horizon LSTM networks are run separately, 50 times for each time horizon. There are different types of errors presented in table 2 concerning the farthest day of the forecast.  As the values in the dataset are not standardized the values of the errors RMSE and MAE are the same non-unit as the forecasted variable. Each from these types of errors can be presented as a "heat map". While MAE can be very easy to apply in real application of the invented prediction model, the MAE "heat map" is presented in figure 4. The results for the last day of predictions are compared to the highest potential error in figure 5. The level of lowering of MAE seems very good, it is to remember that for the case without LSTM predictions the highest daily gradient of SMA is assumed (based on recorded data). To compare these approaches (considering the latest recorded SMA or calculating the predictions w LSTM network), the absolute errors (AE) are calculated for the validating dataset. There were 73 procedures in the first half of 2017 with the winner announced. The number of days between announcements is presented in table 3. Table 3. Time span between announcements of the winner for the procedures in the first half of 2017 and number of procedures with a certain time span form the latest announcement Time span  1  2  3  4  5  6  7  11  14   Number of  procedures  25  12  10  5  7  8  3  2  2 Then, for each period category, the mean absolute error (MAE) is calculated for the case, where the latest SMA value is assumed as a current one. They can be compared to the MAE calculated based on predictions form LSTM networks. It is presented in figure 6. Except for the 1-day period and for the 11-days period, the mean absolute error is lower, when the SMA is left unchanged from the latest announcements of the winner of the procedure. For the validating dataset -even the error values of the predictions are low -the short horizon of the forecast, and a rather stable trend of SMA, made prediction's accuracy not satisfactory. The total absolute error for these 73 procedures is 1,926 based on the latest SMA values, while for the predictions it is summed to 2,623.

Conclusions
The monitored price level in each tender procedure for the road construction in mid-2014 to mid-2017 differs. Sometimes the unexpected peaks of the price-to-estimation ratio are smoothed by the application of the simple moving average. It is necessary to estimate if the price level can be evaluated as marketbased. It was suspected that even the period between the latest procedure and the new one is short, considering the latest value of SME may cause significant error in evaluating the price level in the new tender procedure. The errors produced by LSTM neural networks are low. When compared to the maximum recorded daily change of SMA, the absolute error can be lowered more than twice (see figure  5). Nevertheless, it was proved for the validating dataset, where the daily changes of SMA were not significant, that lower errors can be made if SMA is assumed on the latest recorded level (except for one-day span and 11-days span) i.e. without any advanced predictions.