Smart traffic light control system using image processing

Traffic congestion has become a serious issue due to the growing number of vehicles in Malaysia. Traffic light control system is widely used to control the flow of road junction. Currently, most of the traffic light system used pre-time and count down timers to control traffic flow. Due to the fixed-time setting, often the system unable to handle unexpected heavy traffic flows and cause traffic jam. Thus, there is a need of adaptive traffic signals which are able to do real time monitoring to control traffic light signal based on traffic density. This study proposed an adaptive traffic light control system that uses image processing and image matching technique in controlling the traffic in an effective manner by taking images of each lane at a junction. The density of traffic in the images at each junction are compared. Results show that more time are allocated for the vehicles on the densest road to pass compared to other less dense road. Edge operation detector is used to detect the density of traffic at each lane. In this study, a comparison study was carried out by applying five different edge detectors namely Roberts, Sobel, Log, Canny and active contour. Among these detectors, Canny edge operation detector has found to be the best as it could extract actual edges with average time of 0.453 second.


Introduction
Traffic light play an important role in controlling and regulating traffic on a daily basis. Currently, there are several types of traffic lights used such as pre-timed traffic light with the timing for each signal is determined based on traffic volume and traffic patterns in each particular area [1]. Another type of traffic light is a countdown timer where a two-digit time indicator located on top of the pole above the traffic signal which is used to help the motorists to be conscious of the time left on the green phase as well as to have a better judgment of the traffic flow. Heavy traffic congestion has noticeably increased in major cities and this usually occurred at the main junctions especially during peak hours.
The common types of traffic light used in Malaysia is pre-timed traffic light with the signal timing cycle length usually falling between 45 and 120 seconds [2]. Since they are pre-programmed to wait for a fixed duration of time after every change in signal, they are independent of the traffic on the roads and remain constant throughout their operation. Therefore, even if the traffic density in a particular lane is the least, users are still required to wait for their turn to receive the green signal for a long period of time and when it is their turn to go, it makes other lanes wait for even longer durations.
Centrally controlled city traffic lights via Sydney Coordinated Adaptive Traffic System (SCATS) was one of the early proposed solutions to replace the pre-timed traffic lights in 1982 [3]. The system is slow due to the used of rented telephone lines to transmit the data of the roads to the central controller. There was a suggestion to improve traffic light system by using PIC microcontroller as intelligent traffic signal system embedded with infrared sensor to measure the traffic density [4]. The drawback of this system is that the infrared sensors only work for fewer distances, thus it may give inaccurate data when there is a heavy traffic congestion. Most of the countries in the world have road prevention maintenance strategy to provide immediately repaired on road distress. The conventional human based inspection method is too subjective and inaccurate. In Thailand, an automatic visual inspection system was developed to inspect pothole by using onboard in car camera [6]. The system reveals the road conditions in Bangkok and Bangkokian's driving styles.
Hence, the purpose of this study is to proposed an adaptive traffic light control system by detecting the density of road by image processing. The system detects the density of road and determine the time allocation for each lane. The performance of several edge detectors namely Roberts, Sobel, Log, Canny and active contour are compared and discussed in this study.

Antenna design
Image processing is one of the fast-growing technologies in engineering as well as in computer science. Image processing includes importing the acquired image using image acquisition tools, analyzing and manipulating the image accordingly and producing the output from the image processes [7] [8].
Edge detection is one of the well-developed areas within the image processing. A classical edge detection method involves the use of operators which consist of a two-dimensional filter [9]. There are two types of classical edge operation detectors namely first-order differential operators and second-order differential operators to locate certain types of edges. The first-order differential operators are Roberts and Sobel. The second-order differential operators are Laplacian of Gaussian (Log) and Canny. There is another new edge detection method known as active contour or snake.

Evaluation stage
In 1965, Lawrence Roberts proposed a cross-gradient operator using a two-dimensional mask to detect edges [9]. In Robert edge detection, the vertical and horizontal edges will be individually drawn and the result of the edge detection is then mutually determined. It contains the pair of 2x2 convolution masks [10] which are illustrated in Figure 1. The main drawbacks of Roberts edge detection technique are that it is unable to detect the type of edges that are multiplied by 45 degrees and that it is not symmetrical [10]. Moreover, the cross kernels of Roberts are relatively small, therefore, they are highly prone to noise [11].

Sobel Edge Operation Detector
The Sobel operator, also referred to as Sobel -Feldman Operator or Sobel Filter [9]. Sobel Operator uses 3x3 convolution kernels as shown in Figure 2 where each of the masks responsible in calculating the gradient in both vertical and horizontal direction. Based on the mask [9], it can be seen that the horizontal (x) orientation is expanding in the "right" direction while the vertical (y) orientation is increasing in the "up" direction.
The Sobel operator has larger convolution kernel; it smoothens the input image to a greater extent making the operator to be less sensitive to noise [10]. However, due to its larger convolution kernel, it is much slower to compute as compared to the Roberts edge detector operator.

Laplacian of Gaussian (Log) Edge Operator Detector
Laplacian of Gaussian (Log), was invented by Marr and Hildreth back in 1980 [12]. It belongs to second spatial derivative in which an image is smoothed with Gaussian filter to get rid of the noise sensitivity [13]. It is then uses the zero-crossing method operated by Laplacian operator to extract edges for the smoothed image. One of the drawbacks of the operator is it tends to weaken the image features at the same time which results in some edges are unable to be detected efficiently [14].

Canny Edge Operation Detector
In 1986, John F. Canny developed an edge detection operator using a multi-stage algorithm to detect a wide range of image edges called the Canny edge detector [15]. Canny edge operation detector takes a grayscale image as input and produces an image showing the positions of tracked intensity discontinuities as an output [15]. It works by smoothing the image first, then uses the gradient of the image to highlight regions with high spatial derivatives. It then tracks those regions to delete any pixels that are not at the maximum.

Active contour
Active contours, also known as snakes was first introduce by Kass in 1988. Active contour widely used in image processing to locate object boundaries. This method used to energy-minimize model to detect the edges forming an object boundary with given an initial guess [16]. The model minimizes the energy function from an external and internal force to form the boundary edge of an object in the image. The model is efficient but it sensitive to image noise. It has the difficulty of the initial curve placement due to the local characteristic of image gradients.

Image Matching Techniques
Image matching techniques are the techniques used to locate a pattern in a source image. Every image is identified by its unique set of characteristic features where these features are exclusive to each particular image and therefore contribute to the subsequent identification and discrimination of images [17]. Features can be characterized as a focus of interest or an "interesting" part of a picture that is used as a reference point for some computer vision calculations. Figure 3 shows the block diagram of the proposed smart traffic light control system. The process begins with image acquisition process. In this process, a high resolution 1080px of camera is used to capture images of four lanes at a particular junction. A set of empty lanes (without vehicle) images are first capture at each junction as reference image. The images with vehicles are then collected to detect the traffic density at each junction as test images. These acquired images are then process in MATLAB software using Hp pavilion laptop with AMD Kaveri A10 as the processor, 4 + 4GB RAM and 2TB SATA hardrive. The processed image of the empty lane (reference image) will be used to match with the processed image of the lane with vehicles (test image). Time allocation will be determined based on the traffic signal from image matching.  Figure 4 shows the steps in image processing block diagram. Firstly, the acquired RGB images are first converted to a greyscale image. Secondly, edge detection is applied in detecting the edges of the vehicles in the acquired images. In this study, a comparison between five edge detectors namely Roberts, Sobel, Log, Canny and active contour are performed to select the best the edge detector in this study. Thirdly, dilation operation is then applied on the edge detected images to fill in the discontinuing between the edge segments in order to obtain the outline details of the vehicles in the images. The edge detected from the best edge operation detector will be selected for image dilation process.

Methodology
Next, the test image with vehicles is then being matched with the reference image using featurebased image matching technique. The features of both of the reference and test images include edges and texture of the images are being extracted and the percentage of the image matching is calculated accordingly. Finally, the time allocation for the traffic signal is calculated based on the traffic density that depend on the estimation on the percentage of the image matching.
According to a study made by Harshita, Chandan and Poornima in 2017, the duration of green signal for each lane at a traffic light junction with an average traffic density is set depending on the percentage of the image matching as shown in Table 1 [18]. Due to this, it can be explained that high percentage of image matching represents that the traffic at that particular lane is less congested. Therefore, less duration of green signal is given to that particular lane. On the other hand, less percentage of the image matching indicates that the traffic congestion at that particular lane is high, thus, the duration of the green signal is more compared to other lanes.

RGB to Grayscale Conversion
In this study, four images at the same junction with different traffic signal are captured. The different traffic signal represents four different junctions are namely lane 1, lane 2, lane 3 and lane 4. Figure 5 shows the acquired images that were converted to grayscale image. This is to ensure that less information needs to be provided for each pixel as it eliminates all information on color, thus, leaving only the grayscale signal intensity of each pixel. Signal intensity or brightness can be measured on a different range of scale from zero intensity (black) to full intensity (white). In this study, the grayscale intensity is stored as an 8-bit integer giving the possibilities of 256 different shades of gray in one layer.

Edge Detection Technique
Edges are extracted from the grayscale image. As a comparison, there are five different edge detectors namely Roberts, Sobel, Log, Canny and Active contour were used to extract edges in this study. Figure  6 shows the output of the edge detection of the edge operator detectors and Table 2 shows the processing time taken for each edge detector. Both Roberts and Sobel edge detectors show several edges in the acquired images were missing, thus, reducing the accuracy of the edge detected images. It indicates that a high signal to noise ratio as their ability to detect the appropriate edges on the image source were very low. The average processing time for these edge detectors are 0.434 second for Roberts and 0.442 second for Sobel. Log edge detection on the other hand produced quite similar result to the original image, however some edges are not detected as shown in the Figure 6. The average processing time for Log is 0.475 second.
The fourth edge detector is Canny. Canny edge operation detector extracted most of the edges that similar to the original image compared to Roberts, Sobel and Log. The average processing time for Canny is 0.453 second. The fifth edge detector is an active contour. This detector is able to extract most of the edges. However, the average processing time is lengthy which is 1 second. It is important that the edge occurring in the image should not be missed and there should be no response at all for non-edge in order to give the most accurate result. In comparison, both Canny and active contour edge detector could extract edges most similar to the original images. Edges extracted from active contour model is accurate, however, due to its complicated operation and longer time is needed to perform the method. On the other hand, Canny is a multi-stage algorithm with Gaussian filter that could reduce noise and produce accurate edges. Due to the simple and less complicated operation, the processing time is shorter. This method offers the lowest signal to noise ratio, thus producing a more accurate edge detected image.
Considering the time taken determination for traffic signal should be short in controlling the traffic congestion hence, Canny edge detection is chosen in detecting edges. Moreover, Canny edge detector has good localization in which the distance between the pixels of the edge recognized by the detector and the actual edge is minimum. Therefore, Canny method is the best edge detection technique compared to other edge detection techniques due to its low signal to noise ratio and less time consuming.

Image Dilation
Dilation is a process adding pixels to the boundaries of objects in a way that it expands or thickens foreground objects in an image. The purpose of image dilation is to thicken the edges so that the boundaries of the object more visible and reduce the rate of error in the matching process. Figure 7 shows the image dilation from the output of best edge operation detector which is Canny edge detector.

Image Matching Technique
In this study, feature based matching technique is used to calculate the percentage of image that match between the two images. The result from the percentage of the image matching is used to identify which lane should be prioritized of having the longest duration of green signal. Figure 8 shows

Conclusion
This paper presents the development of an adaptive traffic light control system by using image processing technique. The technique analyze the density of traffic signal and time allocation is determined based on the percentage of matching. This system could reduce the use of additional hardware devices such as sensors, wireless routers etc. Furthermore, it is able to detect the presence of vehicles consistently as real-time images of the traffic are used, thus, time allocation of green signals to the lane with the highest priority is more compared to others in order to reduce the traffic congestion. Canny edge operation detection is found to be the best for edge detection in this study. The edge detector could extract edge similar to original image with reasonable processing time.