Automatic Prediction of Road Angles using Deep Learning-Based Transfer Learning Models

The construction of robust and complex roads requires a considerable amount of resources, time, and effort. These efforts get manifold, especially when a road is to be constructed on rugged terrains. The use of Artificial Intelligence (AI) can be seen in almost all the research sectors. The efforts and the cost involved in road construction can be reduced by the induction of AI-based techniques for the estimation of road construction-related parameters that are required to predict the tentative cost incurred in the whole road construction project. In this article, the angle, which is one of the crucial parameters that aids in estimating the total cost and time required for the whole road construction project is predicted using the variation of three transfer learning-based deep learning models viz. VGG-16, DenseNet-121, and DenseNet-169. The proposed VGG16 based CNN model performance is computed and compared in terms of the evaluation metrics like Mean Square Estimation (MSE), Mean Absolute Error (MAE), and Correlation Coefficient (R2). Based upon the simulation results conducted, it was observed that VGG-16 has yielded road angles with less difference error.


Introduction
Road angle estimation and prediction is a crucial task in both the autonomous transportation system and road construction. Road angle estimation is equally essential in self-driving cars while making smart decisions in real-time driving. However, the road angle estimation must be accurate, reliable, and available in all possible terrains. In the construction of roads, the estimation of crucial road parameters like road-angles, slopes, elevation, type of terrain, etc., can be used in accurate pre-project cost estimation. There are various ways of computing the abovementioned parameters, including the road angles. However, strict adherence to accuracy, reliability, availability, and real-time predictions are essential requirements for such applications. Deep Learning is an emerging and robust field of Machine learning that has already proven its worth in numerous useful domains and application areas. The powerful models of deep learning are already being used in solving various complex problems of road-construction and intelligent transportation systems (ITS). For example, in ITS, various deep learning models are deployed for tasks like pothole detection, object detection, intrusion and threat detection, traffic-lights recognition, number plate recognition, and many other similar applications [1] [2]. Similarly, in road-construction, deep learning and machine learning techniques are used for performing high precision and time-bounded tasks like estimation of shear strength of soil, compressive strength of concrete, structural health monitoring, etc., which are directly or indirectly linked with the lives of people [3] [4] . These tasks can be performed manually also in laboratories. However, computing the important road construction related parameters automatically using machine learning and deep learning models will not only improve the accuracy but will also reduce the cost and time in the calculation of these parameters. Machine learning and deep learning models are used interchangeably in different fields based upon the requirement of the work to be done. Although deep learning models are the subset of its predecessor Machine Learning, deep learning has various reasons and a lot to offer, making it the first choice of researchers in various applications. In deep learning, the need to manually select the relevant features from the feature set is not there. The deep learning models automatically find the best and suitable features on their own. Thus, it provides significantly improved accuracy, which can be the problem if erroneous and less significant features get selected. In this article, considering the abovementioned massive strength as well as vast applicability of the Deep Learning models in different fields of ITS and road construction, attempt has been made to explore the possibility and applicability of deep learning, especially transfer learning, in the estimation and prediction of road angles which is an important parameter in both ITS as well as in road construction during the pre-project cost estimation.

Motivation of this Work
ITS is the future of the existing transportation system that will improve the driving and on-road experience and provide a plethora of additional benefits like reduced waiting time, improved fuel efficiency, reduced congestions, and on-roads accidents, smart parking, and much more. Providing good roads is vital for a smooth driving experience. Various cost and other parameters are usually involved during road construction, which is both time-consuming and cost extensive. So, the technology of Artificial Intelligence (AI), which is being used in most research and application areas considering their accuracy and performance, can be used in ITS for tasks like road quality identification and estimation of road-construction parameters like slope angle, elevation, etc. In this article, use of the strength of powerful AI technology for estimation of road parameter like angle of the road, which is equally important in self-driving autonomous cars and road construction cost estimation, has been explored. The estimation of road parameters like road angles for self-driving cars and road construction cost estimation needs to be highly accurate, considering the fact that any error in the estimation can lead to accidents and claim people's lives.

Contribution of this Work
The main contributions of this work are as follows: 1. Three Transfer Learning Models viz. VGG-16 with fine-tuning, DenseNet-121, and DenseNet-169 are used for the prediction of road angles. 2. Comparison of the performance of the abovementioned three transfer learning models in prediction of Road angle prediction is made to find the best-suited model for this task. 3. Future Possible Enhancements in the prediction of road-angles that can further improve the CNN-based model performance are presented.
The rest of this paper is organized as follows: In Section 2, a Literature review of existing related work is done. In section 3, our proposed transfer learning models for road angle prediction are presented in which various transfer learning models used for predicting road angles, their network architecture, and the various hyperparameters used for the training and testing of the proposed models are presented. The discussions on the performance of the proposed model is also presented in Section 3. The concluding remarks to this work along with the future enhancements that can be incorporated for enhancing the performance of models are given in Section 4.

Literature Review
In this section, various existing related work on automatic estimation of different road-related parameters like angle, elevation, slope, texture, quality, etc. Using Artificial Intelligence required during the construction of roads and highways are reviewed and discussed. Authors in [5] have tried to estimate the speed and steering angle using deep learning in which the features are learned automatically using the existing driving data without any involvement of humans. This automatic prediction of a vehicle's speed and steering angle can be deployed and used in autonomous driving. In this work, a deep belief network (DBN) is being used as the learning model, and real-world driving data are used for the training of the model. Based on their simulation results, the authors concluded that the DBN-based automatic learning and prediction model performs well in terms of accurate prediction compared to the traditional manual learning methods. Similarly, another road parameter, i.e., slope, which is important to estimate during autonomous driving and in the Intelligent Transportation system, has been predicted using a slope-estimation algorithm proposed by authors in [6]. Data received from three sources viz, Global Positioning System (GPS), onboard-deployed sensors, and longitudinal vehicle model, are used for the estimation of the slope of the road. A probabilistic data association filter (PDAF) is used for the estimation, which integrates the values received from the abovementioned used measurement sources. An Artificial Neural Network-based model has been proposed by authors in [7] to estimate the sideslip angle of a vehicle in different road scenarios as well as to identify the condition of different roads. The estimation model for the sideslip angle of the vehicle is considered for three types of road conditions, i.e. icy, dry, and wet, using three distinct regression models. Road`s quality identification was performed using pattern recognition neural network-based classifier. The proposed ANN model is trained using the self-created dataset prepared via high quality and high-performance recording instruments in different road conditions. The predicted angle is compared with the actual angle, and an accuracy of about 98.6% was attained by the proposed ANN-based model. Authors in their work in [8] have tried to predict the on-road steering angle on different road scenarios to assist autonomous driving, which is a step towards ITS. Different artificial intelligence-based models, including LSTM (Long Short-Term Memory), ResNet, 3D-CNN, are used in this work by the authors for the estimation of the on-road steering angle. The models were trained using the real-world driving data of Udacity containing high-quality images of road captured from the capturing device deployed on the car. In this work, the aim was to predict the angle of the steering using the provided images. Similar attempt to showcase the capability of CNN in aiding the self-driving cars and bringing the concept of autonomous self-driving cars one step towards reality, authors in their work [9] have proposed a CNN based model which learns and trains itself automatically using the existing images from simulated car. The proposed CNN model automatically learns the unique features and then predicts the required steering angle for a particular road scenario that further allows for driving the car. Similarly, another work where Artificial Intelligence can be used for cloning of driver behavior and the accurate estimation of road-related parameters is presented by researchers in [10]. In this work, researchers have tried to used various variations of auto-encoders along with the Recurrent Neural Network (RNN) for cloning driver`s driving behavior and for automatic driving of a car. Based on the above literature, it is quite evident that various models and techniques of Artificial Intelligence (AI) can provide promising results in autonomous driving and Intelligent Transportation System. AI models can also be used for driver`s driving behavior cloning and accurate estimation of various road-related parameters like angle, slope, speed, etc., which is equally important for both autonomous driving as well as for road construction.

Dataset Used
There are many public datasets available related to road quality estimation. For this task, a specific type of dataset was required in which drone images of the road are given as input to the prediction model to estimate the slope, angle, and elevation. However, there was no such type of dataset publicly available. So, to demonstrate the effectiveness of the Artificial Intelligence, especially the transfer learning models in the prediction and estimation of road angles of the input road images, Udacity driverless car dataset [11] has been used in this work for both the training and testing of our proposed transfer learning models . The self-driving car dataset of Udacity comprises of 100,000 video frames. Each video frame is approximately about 40 seconds long in reasonably high quality (720p and 30 frames per second). To explain the tough driving trajectories, GPS information, captured from cell phones, is also included in these videos. These videos were taken from different sites in the United States. The Udacity dataset comprises 223 GB of picture frames and records data on two different days from 70 minutes of driving in Mountain View, one day being sunny and the other being overcast. These datasets help us in estimating the angle of the road as shown in figure 1.  The Udacity dataset considered for this work was fetched from the following GitHub repository: https://github.com/udacity/self-driving-car/tree/master/datasets/CH2. 90 % of the above considered Udacity dataset was used for the training of the transfer learning models, while the remaining 10 % was used for testing the performance in terms of the difference between the actual road angle vs the predicted road angle. In Fig. 1, some of the sample road images used in this work, which will be given as input to our different transfer learning models, are shown.

Dataset Pre-processing and Data Augmentation
Data Pre-processing is a very crucial step that can considerably improve the performance of the AI models. Data from the dataset was pre-processed using various standard existing techniques like resizing, smoothing, etc. Each image was scaled from [0,255] into range [-1,1].

Road Angle Estimation using different Transfer Learning Models
For this work, different transfer learning models to predict road angles have been used. Transfer learning is a machine learning method where CNNs trained on a task are reused as the starting point for a second task model [12]. Transfer learning allows using high quality trained models that were trained on some existing larger datasets. The primary benefit of using transfer learning is that the transfer learning model uses the existing knowledge it has gained while solving a different problem that is identical in nature to the one being solved, rather than starting the learning process from scratch [13]. Thus, the model takes advantage of previous learning in transfer learning and prevents starting from scratch. Transfer Learning has already been used in various other classifications problems and has provided satisfactory performance in terms of training time and accuracy. Transfer learning is commonly accomplished by using pre-trained models. Pre-trained models are the models that have been trained to solve a similar problem on a broad benchmark dataset to the one we want to solve. There exist a variety of transfer learning models like ResNet, VGG-16, etc. [14]. This paper has used three transfer learning models, i.e., VGG-16, DenseNet-121. and DenseNet-169, for road-angle prediction and have compared their performance to check which model performs better. Our algorithm involved first extracting the features and then inputting the features and individual frames of video to train a Convolution Neural Network, giving us the road's angle (treating it as a regression problem). Training data comprised of input images of road and output labels contained the angle turn of the road. The model has been trained using various neural network architecture available in order to get good accuracy. All the three transfer learning models used in this work were implemented on Google Colab with Graphical Processing Unit (GPU) support.

VGG-16 Architecture with Fine-Tuning:
VGG-16 consists of 16 convolutional layers. Due to its uniform architecture, this model is very appealing [15]. This transfer learning network comprises of six blocks of convolutional with filters, each of which is followed by two fully connected layers of neurons, and a single output. The input size varies based on the area of the dataset that is considered. Dropout layers are added prior to the fullyconnected layers, for better generalisation. For regression targets, the camera is scaled in the range

Network Architecture
The overview of the network architecture used in this article for road angle prediction using Transfer Learning Models is explained in the following points: 1. Dataset: The dataset is split as a Training and Testing Set in a 90:10 ratio. The dataset used is a Udacity self-driving car dataset that contains road images and corresponding road angles. For the provided road image, our aim is to predict the road angle. 2. Three pre-trained transfer Learning models, viz. VGG-16, DenseNet-121, and DenseNet-169, has been used. The last layer is used for computation and prediction of road angle with Linear Activation function, as explained in Figure 2 (R 2 ) values are the three performance evaluation metrics that were used for testing the performance of the proposed transfer learning-based models for road angle prediction. The complete architecture of the proposed VGG-16 model for road angle prediction is explained using Fig. 2.

Evaluation Metrics
For evaluation of the performance and efficiency of different transfer learning models in prediction and estimation of road angles, three performance measuring metrics, i.e., Mean Square Error (MSE), Mean Absolute Error (MAE), and R squared error (R 2 ), are used as the Evaluation Metrics.

Mean Square Error (MSE):
Mean Squared Error can be calculated as the average of the square of the difference between the actual value of the road angle and the predicted angle value. where n is the total number of samples (angles in case), and i ranges from 1 to n.

Mean Absolute Error (MAE):
Mean Absolute Error (MAE) is defined as the absolute difference between the actual value and the predicted road angle value [16]. If the difference result is negative, then the negative is ignored in the absolute difference.

R Squared Error (R 2 ) or Coefficient of Determination
R Squared Error (R 2 ), also called a Coefficient of Determination, determines how good a model fits into a specific dataset [17]. The value of R 2 falls between 0 to 1, where R 2 indicates that the model does not fit well with that data, while the value 1 tells that the proposed trained model has fitted perfectly with the given data.

Results
All the considered three transfer learning models were trained and tested against the same dataset. Table 1 shows the performance of the VGG-16 model in terms of metrics like MAE, MSE and R 2 for the prediction of road angles. In table 2, the comparison of the performance of VGG-16, DenseNet-121 and DenseNet-169 in terms of MSE value is shown. From the computed metrics values of different transfer learning model, it is evident that VGG-16 is performing better in the prediction of road angles on the Udacity self-driving dataset in terms of the error difference and is yielding encouraging results. Figure 3 shows the graph depicting the VGG-16 predicted angle vs the actual road angle.

Discussion
It is evident that it is possible to create a model that can reliably predict the road and steering wheel angles for a given road image as input using a transfer learning-based deep neural network. After applying further pre-processing, data augmentation and hyperparameter tunings, it is possible to boost the efficiency of the prediction models. However, considering the risks as well as the cost involved in these applications, these models cannot be used as it is in the real-world transportation scenarios and requires further improvement. The models are also required to be trained, considering an exhaustive list of road and weather scenarios.

Conclusion
Good Roads are reflectors of a country's economy as roads act as communication pathways that connect different regions and are used for a smooth supply of goods that includes food, medicines, and defence supplies. However, various factors ranging from road construction to road maintenance are involved which requires huge manpower, cost, efforts and planning. AI, which is the revolutionizing technology used in most of the applications and research areas, can be used in the road construction projects and road quality maintenance. In this article, the possibility of using AI-based transfer learning models in predicting road parameters like road angle, which can also be beneficial in selfdriving cars, was examined. Three transfer learning models, viz, VGG-16, DenseNet-121 , and DenseNet-169 were trained and tested on Udacity self-driving datasets. The performance of these models was compared on the basis of metrics like MSE, MAE, and R 2 . From the result analysis, it is evident that the VGG-16 has performed better in road angle prediction in comparison to other considered transfer learning models. However, the model still requires considerable improvement for the deployment of these similar models in real-life road scenarios.

Scope for Further Improvements
The performance and accuracy of the models can be improved further in future works using the whole dataset as in this work, the models were applied on a part of the dataset considering the resource limitations. Extensive hyperparameters experimentation can also be performed, which can contribute further to the performance of the models. Also, since there is no dedicated dataset for the road`s parameter estimation, a new road dataset with attributes like road angle, elevation, slope, etc . must be prepared.