ITLCS Based on OpenCV Image Processing Technology

With the rapid growth of the number of motor vehicles in the city, traffic congestion is more serious every day, part of it is caused by the coding delay with the red light on, not real traffic jams, now we need a control system that can really change the traffic flow. In this paper, ITLCS (intelligent traffic signal control system) based on OpenCV image processing technology is proposed to adjust the timing of traffic signal according to road density, instead of setting a level that is balanced with other lanes, so that high-load traffic lanes can be used for a long time. The camera facing the roadway in the system takes pictures of the driving route, then takes pictures of the driving density of pedestrians and vehicles, and compares each image through processing technology, after the system is processed, the traffic light signal timing can be adjusted immediately, which greatly reduces the time spent on the inactive green light and can effectively deal with the traffic congestion problem.


Introduction
In recent years, my country's urban economic construction has developed rapidly, but urban problems have also followed, the primary problem is the phenomenon of urban traffic congestion. Because of the sharp increase in car ownership, the problem of urban traffic congestion has become more serious [1]. Some experts and scholars believe that problems such as insufficient transportation infrastructure, too many cars, and unreasonable traffic layout have caused traffic congestion to become more and more serious [2]; Due to the imbalance between traffic supply and demand, urban traffic congestion is caused [3]. And they also put forward a variety of methods to solve the problem of traffic congestion, such as: building new roads, using local area networks, RFID systems (radio frequency wireless signal automatic identification system), etc. However, because there is not enough construction space, so no new roads can be built. The cost of using RFID sensors and LAN systems at intersections is much higher than the existing traffic signal control system [4].
In fact, because fixed traffic signal frames are set at road intersections, traffic congestion will be caused [5]. The traffic signal control system did not timely base on the actual situation of the road, but set a fixed signal time for each road, which causes many problems [6]. In areas with a large number of people, it will have a great impact on people's life, work and even safety and health in the area,  [7]. Due to the impact of morning and evening traffic peaks, people cannot complete their work according to their expected plans, and people cannot predict their trip time [8], which directly affects more things that people have to deal with in daily life [9]. Another problem is that the vehicle will stop and start at the intersection, which directly leads to higher fuel consumption than during normal driving. The increase in fuel volume leads to the need to pay more fuel fees, at the same time a large amount of air pollution exacerbates global warming [10]. In addition, this problem will have a greater negative impact in emergency situations, and fire trucks and ambulances cannot respond at the appropriate time. Let's take the case of an ambulance as an example. When the patient on this road is in a critical condition, but the traffic jam may make it impossible for the ambulance to reach the hospital at the appropriate time. This situation will put the patient in danger of life. This is why an intelligent traffic control system is needed to effectively deal with different traffic conditions and avoid traffic jams and accidents.
The main modules of the intelligent traffic monitoring system include: model, length, traffic, route, traffic police and direction management, as shown in Figure 1. The main content of this research is: ITLCS system uses image processing technology to detect vehicle density and process lane images taken by digital cameras, including: sharpening, blurring, brightening, edge enhancement, etc.; Then, the images are grouped and analyzed by computer imaging technology to obtain the actual vehicle situation, and the time of traffic signal is controlled by computer. The OpenCV image processing technology used is a free Intel open source computer vision library.

. Preliminary research
Pranjali B and others proposed ATCS system [11] is an adaptive traffic control system combining embedded Linux board and image processing. Determine the traffic light signal time according to the road density to ensure that the signal time of serious traffic congestion is longer than that of other intersections. In this way, the timing assigned to each lane opens for a given time in a clockwise direction. Then, according to the different traffic density, each lane spends different time and repeats this cycle. The digital camera records the video and processes the video frames using the open source computer vision library to obtain the density of the vehicle. In case of emergency, GSM and GPS module system will provide special services.
Jackrit SuthakornA and others proposed intelligent vehicle and driving assistance system based on 3 open source computer vision library using neural network, which can quickly, real-time and reliably carry out traffic sign recognition (TSR) [12]. The automatic detection and recognition system of road signs will help relieve the burden of drivers and improve the safety and comfort during driving. The automatic recognition of traffic signs in automatic intelligent vehicle and driver assistance system is a very important content. The system has high classification accuracy and large calculation capacity for road signs with complex background images. S Shinde et al. introduced STC system, which is an effective solution provided by OpenCV software for intelligent signal control system [13]. The camera installed in front of the Lane takes photos of the road conditions. These images are effectively processed in order to understand the traffic flow of the road. According to the processing data of OpenCV, the 8051 microcontroller will send a control signal to the traffic LED timer to adjust the specific time of the traffic signal. Compared with sensors, OpenCV has the advantages of simple configuration, low cost, high precision and high speed. However, because there are few vehicles in rural areas and traffic congestion rarely occurs, the system is not suitable for use in rural areas.
Bilal Ghazal et al. constructed a traffic light STLCS system [14]. Generally, traffic lights monitor and control vehicle activities at multiple road intersections. The traditional control system can not supervise the variable flow of intersections. There is no correlation between adjacent traffic signal systems, traffic flow time difference, traffic accidents, emergency vehicle traffic and pedestrian crossing, resulting in traffic congestion and traffic delay. The LED device of PIC microcontroller in the system uses infrared sensors to obtain traffic density and allocate dynamic time at different levels. In addition, the remote controller handles emergency vehicles trapped on congested highways.
Anuradha G. Suratekar introduced WSN-ITCS System [15], which is an intelligent signal light control system based on image processing and wireless sensor network. The induction coil method is used for traffic detection, but the volume of the equipment is large and the installation and maintenance are not very convenient. At the same time, the current traffic control system has set the time, which is not very effective in traffic control. There is no mutual communication between induction coils. WSN has the characteristics of scalability, fault tolerance, coordination and real-time. Therefore, according to WSN, the traffic signal system can be planned and programmed with C + + language to adjust the traffic density.
In order to overcome these problems, this paper proposes a system that uses digital camera to measure traffic density and automatically adjust intersection signal time. The system relies on global system for mobile communications (GSM), global navigation satellite system (GNSS) and processing technology, in order to keep the highway line unblocked, it provides a convenient transportation method in emergency situations, which will enable the transportation system to work normally. Synchronously upload the provided traffic to the web server, and store the traffic database of the entire system, therefore, in daily use and emergency situations, the smooth and safe traffic can all be ensured.

ITLCS (Intelligent traffic light control system)
ITLCS is an intelligent traffic signal control system based on OpenCV processing technology, which provides appropriate time to adjust the traffic flow in each direction of the intersection, so that the lane with high vehicle density has a longer opening time than other lanes, and maintains the total intersection time ratio, so that the lane with low density also has a free path without waiting for more time. First of all, accurate vehicle quantity data should be obtained to reasonably allocate the time on each road. The system includes: digital camera, OpenCV software, image processing unit, wireless transceiver, microcontroller, etc. OpenCV system as image processing software; Python is the language used; Rasberry PI processes images and uses 8051 microcontroller to convert digital language into binary language; Color threshold and blob detection technology are adopted; Kalman filter algorithm is used.
As shown in the ITLCS system framework in Figure 2, the system can detect fire trucks, ambulances, emergency vehicles, etc. and take the required actions, at the same time, We use Python language programming to implement the operation. After the camera captures the image, the Raspberry Pi is used to process the image, and the time is allocated to each traffic light according to the traffic density. In an emergency, the system will consider the best. GSM and GPS transmitters are placed on the ambulance to send out emergency warning messages, and GSM and GPS signal receiver engines are installed at every intersection. Each intersection is assigned a unique code, and all codes for each route have a unique identifier, as shown in Figure 3.

Proposition 1: State Model and Process
Considering the discrete-time control process, which is described by linear equation (1) as:  (1) stands for random variable process noise, s l represents the state variable of time step 1, B and A are the state differential equation (1) of step l related to the m × m and 1 × m matrix coefficients, and control the input accordingly. Now the system measurement can be described as: In equation (2)x l represents the estimated value of step 1, random variable u l is the measurement noise of the system, and G is n × M measurement status x_ N of L × M matrix coefficient. Random variable s_ L and u_ L is different. The covariance matrices P and t of these Gaussian white noise are shown in equations (3) and (4): Then according to the above equation, equation (5) is that for all l and j, the covariance matrix from s to u is always zero:

Proposition 2: Kalman filter origin estimation
The current state of y l in step 1 and its prior state in step 1 are estimated to be y � l − , recalling y � l − and the posterior state estimation x l is y � l − when the estimation is allowed in step 1, then a priori and a posteriori calculations of the errors in equations (6) and (7).
Therefore, the before and after evaluation of the error covariance is expressed in equations (8) and (9) as: The most important factor in Kalman filtering algorithm is gain or mixing factor L, which minimizes a posteriori covariance and accurately predicts moving targets.
It can be seen from the above equation (10) that when T l → 0 is used more and more to measure the actual measured value of x l , if Q l − can be used, then G l y l − uses predictive calculation, and believes the actual measured value x l more and more. Therefore, combined with the above equation, it can be seen that the time update filter and measurement update filter are used in the form of prediction equation and correction equation of Kalman filter.  figure 4, it will continue to track it until the vehicle leaves the frame and count it. Therefore, the center of the detected vehicle must be found and connected to the vehicle in a rectangle. When we get the center of the detected vehicle, we will use the Kalman filter algorithm to track the vehicle, obtain the accurate number of vehicles in each lane, and calculate the time spent on different routes at the intersection. This database saves the traffic information and replaces the route time allocated to each intersection on the web server.

Proposition 3: Time update equation
The equation used in the programming process predicts the future state as follows: The prediction of previous error covariance:

Proposition 4: Updating the measurement equation
Kalman gain is expressed as: Update state estimates with measured values: The error covariance estimates and measurements represented by updates:  Figure 5. Backgroud Subtraction algorithm Figure 5 shows Background subtraction is a method for computer vision processing. It is processed after removing the foreground of the image. Receive the data stream of the vehicle through the digital camera, convert the video into frames, delete the background and identify the moving target. The three stages are: in system initialization, the system starts and establishes, the digital camera tracks the data stream and sends it to the processing system; Background subtraction takes a series of video frames as the center for continuous analysis and operation of background subtraction; Then, in vehicle detection, the moving vehicle or object is visible. The extracted background image is processed by background subtraction for tracking and counting.
The ITLCS system operates either "off" or in three colors (red, green or yellow) to simulate traffic lights operating in only one direction. Assuming that the system user requests to turn the light on or off, or change the color of the light, the arrow shows the possible change state, and the change points to a position that can be moved from one state to another. Therefore, this study will establish a cost-effective signal intelligent management scheme. A lane oriented camera captures the driving route, then captures the density of pedestrians and vehicles, and uses image processing software to effectively process and compare each image, so as to facilitate system users to understand the traffic density.

Work performance ratio
The multi period timing control system is used to decompose the traffic flow into several cycles in a day, adjust the signal time frame in different periods of the day, including considering the adjustment of the traffic demand of the day, and optimize and create the signal timing by using the detailed performance index method or green band time frame method. This is a real-time signal management system with easy construction, strong operability and low cost, which shows better performance, or at least long-term strategic performance. These strategies have been well adapted in different networks for many years. Figure 6 shows that the ITLCS method has a high performance ratio. Figure 7 reflects the accuracy ratio of the ITLCS system. The accuracy of describing the actual traffic environment based on the model and the control knowledge based on control feedback cannot be learned and adjusted online. Obviously, V2V, V2I, v2x communication technology and new automatic driving technology under large-scale development and implementation show the development of urban traffic control system from the period of data shortage to the information age.  Figure 6. Performance ratio Figure 7 shows the accuracy ratio of the ITLCS system. The accuracy of describing the actual traffic environment based on the model and the control knowledge based on control feedback cannot learn and adjust the control effect online. Obviously, V2V, V2I, V2X communication technologies and new autonomous driving technologies that are being developed and implemented on a large scale will greatly promote the development of urban traffic control system technology routes from the data scarcity period to the information age.  Figure 9. Accuracy ratio Volume prediction is a key component. Connecting the two factors with prediction can reflect the accuracy and resolution. The relationship and transaction between control strategy and prediction resolution, as well as the correlation error between them, are of great significance for the establishment of the system. In short, in the environment of rich traffic in the future, the theory and method of urban traffic integrated management is an inevitable choice. The proposed ITLCSmethod has high accuracy. Figure 8 reflects the level of traffic density recognized by the system, and figure 9 compares the accuracy difference between the system and the existing methods.
The most important factor in Kalman filter algorithm is gain or mixing factor L, which reduces the covariance, thereby achieving accurate prediction of moving targets. To achieve accurate prediction of moving targets, the key factors and the accuracy of vehicle prediction are as follows Shown.

Error rate
Kalman filter deduces a series of mathematical equations to predict the past, present and future situation, which is very important, and the optimal estimation is the most common. When all noise is Gaussian noise, Kalman filter minimizes the mean square error of calculation parameters. The sound is not Gaussian, and the Kalman filter is also very useful because it uses the best linear estimator and a good nonlinear estimator. For the noise data of Kalman filtering, there is the best approximation result, and the proposed ITLCS method has a smaller bit error rate in image detection, as shown in Figure 11.  Average waiting time The system proved to be able to minimize the aggregation of vehicles and shorten the waiting time of vehicles under traffic lights. In order to conduct real-time research on modern urban traffic, the author created real traffic images and combined them with micro-control and insurance companies, it is demonstrated that the average waiting time of vehicles in front of the signal lights is lower than that of the existing traffic control network. The number of vehicles is counted and the image is processed to ensure that the average waiting time of vehicles in front of traffic lights is less than that of the existing traffic management system. The efficiency of the system is improved through the technology or algorithm adopted in the project, as shown in Figure 12. Figure 13. Computational cost By testing 52 images with test marks, the calculation cost is estimated. The average processing time of each frame is 37.27 milliseconds, and the time diagram of test image processing calculation. The processing time can be divided between two different regions according to the image complexity. Because the algorithm is located as a circle or ellipse in the test image, there are a large number of potential regions in the very complex background image, which requires more processing time. This method can reduce the computational cost and make the real-time implementation simple, as shown in Figure 13.

Calculate the cost
As mentioned above, ITLCS system can effectively deal with daily traffic problems. GSM and GNSS modules are used to track emergency vehicles such as ambulances, fire engines, police vehicles and VIP vehicles. The transmitter of the module is located at the intersection of emergency vehicles and receivers. Send information when the vehicle approaches the nearest intersection, receive the information at each intersection at the same time, and arrange these emergency vehicles in time when necessary..

Conclusion
In summary, the developed vehicle tracking and counting system takes the open source computer vision library as a software tool and uses Kalman filter algorithm and background difference algorithm. Based on OpenCV processing technology, background subtraction is applied to the detection of moving vehicles, and Kalman filter algorithm is used to continuously monitor vehicles. The system uses a single digital camera that evaluates the road density at intersections. We have tested the device in different environments. Compared with other sensors, the device has a higher efficiency. This makes it possible for the system to store and update the database to provide efficient traffic flow for the entire city. This system gives emergency vehicles the highest priority and continuously monitors them after obtaining data from the emergency vehicles, in order to further handle emergency situations, the system provides better performance than other systems.