Research on Key Technology of Vehicle Electronic Image Stabilization

The study of image stabilization algorithms plays an important role in achieving electronic image stabilization. This paper firstly introduces the basic concepts, principles and common typical algorithms of vehicle-borne electronic image stabilization technology. Then, based on the characteristics of vehicle-mounted image processing, this paper analyzes the influence of the instability of vehicle-mounted image sequences on the observation results of vehicle-borne systematic images, and introduces the vehicle-mounted electronic image stabilization technology. Besides, a method to achieve the stabilization of the inter-frame translation motion of vehicle-mounted dynamic image sequences using the gray projection matching algorithm is proposed. In the vehicle-mounted imaging system, the maximum gray value in the feature image is used to estimate the inter-frame motion of the image sequence, and a partition method is used to estimate the motion vector by selecting the gray-scale maximum point as the feature point in a certain scanning order, to ensure the validity of the algorithm. At the same time, based on characteristics of vehicle imaging, the necessity and possibility of detecting long-focus picture pick-up system and compensating the rotation movement are proposed, and the method of using the maximum gray value points as the image feature to estimate the inter-frame motion of the image sequence are presented.


Introduction
The EIS (Electronic Image Stabilization) refers to the process of removing the image disturbance caused by the random motion of the camera from the input video image sequence to stabilize the image sequence, and the video caused by the random motion of the camera is expected to be eliminated after processing [1]. The implementation of EIS can ensure that the camera outputs a stable video image within a certain range of changes [2].
In order to improve the equipment's precise strike capabilities, target acquisition capabilities, command capabilities, communications capabilities, intelligence acquisition capabilities and situational awareness capabilities, the modern tanks, self-propelled anti-aircraft artillery, reconnaissance vehicles, and command vehicles have adopted optoelectronic observation equipments [3]. The vibration of the supporting platform caused by the vibration of the vehicle body will cause a large shift between the frames of the image sequence, which is reflected on the monitor that the obtained image information is unstable, the image quality is degraded, and the resolution is reduced. The jitter and instability of the image display make the observer easily fatigued and also reduce the accuracy of the measurement, resulting in misjudgment and missed judgment. Therefore, the stability of on-board camera systems is a very important issue, especially in long-focus, high-resolution surveillance and tracking systems [4].
The method of stabilizing image can be divided into three types according to its function and principle: mechanical image stabilization, optical image stabilization, and electricity substabilization. The former two methods require the addition of precision machinery, optics, and electronic 2 1234567890''"" components and other equipment, which is not conducive to miniaturization and integration, and it is difficult to meet the need for further development of modern image stabilization technology [5]. The electronic image stabilization technology is the use of electronic equipment and digital image processing technology to detect the motion displacement of the reference image and the compared image, and use it to compensate the compared image, thereby eliminating or reducing the instability of the video image frame, to obtain clarity.
The stable image sequence has the characteristics of flexibility, high precision and high intelligence. After 1990s, with the rapid development of computer technology and large-scale integrated circuit technology (VLSI), the performance of computer products has rapidly increased while the price of image processing equipment has continued to decline, which have provided favorable conditions for the development of digital image processing. As a result, the focus of people's research has begun to shift from traditional optical image stabilization and mechanical image stabilization to the research of electronic image stabilization technology [6].
This paper is mainly about the basic principle and method of applying electronic image stabilization technology. The image sequence taken by vehicle imaging equipment is considered as the researched and experimental object, and the method of stabilizing the vehicle video image sequence is studied, based on the environment of vehicle imaging and the characteristics of the obtained video dynamic image sequence.

The Principle of Automotive Electronic Image Stabilization Technology
The vehicle-mounted image processing system consists of image acquisition, image preprocessing, electronic image stabilization, image segmentation and tracking processing modules. Based on the vehicle-mounted image processing system, the stable and stable video image sequence can be obtained through electronic image stabilization processing [9], and the structural diagram of the vehicle image processing system with electronic image stabilization device is shown in Figure 1.

The Basic Principle and Method of Electronic Image Stabilization
Electronic image stabilization is a method that utilizes a combination of electronic equipment and digital image processing technology to detect the motion vectors of reference image and compared image, which can be used to compensate the compared image to remove and eliminate the instability between video image sequence frames, based on which a clear and stable video image sequence can be obtained [10]. The structural schematic of the imaging system with electronic image stabilization device is shown in Figure 2 In this section, the image frame inter-frame blur and compensation method from the image plane will be analyzed. Figure 2 shows a schematic diagram of the camera's imaging target plane, and each cell representsone pixel. Figure 2(a) represents the first frame of an object in the four areas of (00, 01, 10, 11) when the camera is imaging. Fig. 2(b) shows that the second frame of the same object is formed in four areas (11,12,21,22) when the camera is mounted on a moving carrier and a vibration with a motion vector is generated.
The image will be blurred from the monitor because the same object can generate formation of image in different areas. By detecting the motion of the camera, the pixels of the moving image plane can be compensated for the second frame image, so that the image of the second frame image plane coincides with the image of the first frame image plane, to output a stable image sequence [11].
The structure of a typical electronic image stabilization system is shown in Figure 3. As can be seen from Figure 3, the electronic image stabilization system is mainly composed of three parts: image preprocessing, motion detection and motion compensation. Electronic image stabilization is a method of directly detecting a motion vector between image frames from a given reference image and a compared image to compensate further. The relative motion vector between the scene and the photographic system is determined by the motion detection (i.e. the motion estimation system and the motion determination system). The motion estimation system processes the image through various algorithms to estimate the motion offset of the image sequence.
Where θ is the camera's rotation angle and (ΔX, ΔY, ΔZ) is the camera's displacement. According to the principle of perspective projection, the formula for the scene point image on a two-dimensional image is: In Eq.
(2), f is the focal length of the camera lens; i=1,2,..., indicates different moments. The amount of motion variation between images can be expressed by Eq.(3). sin cos In Eq.(3), s is the zoom factor, and (∆x 2 , ∆y 2 ) is the image displacement of t 2 relative to that at time t 1 .

Effect of Random Motion Speed of Vehicle Camera on Video Image
The random motion of the vehicle-mounted camera can cause the jittering of the image screen, and people have studied many methods to stabilize it and achieved certain results. Because the image blur is caused by the camera's random motion, the different speed of the camera makes cause the ways causing the image blur are also different, namely, the so-called intra-image blur and inter-frame blur of the video image [13].
For the image blurring caused by high-frequency vibrations, the image must be processed one frame by one frame, because each frame image in the video image sequence is blurred, that is, the entire image function space is blurred. On the contrary, the processing of the image blur caused by the low-frequency vibration must be performed on the image between frames, that is, the entire image sequence is processed. Because the cause of the blur is different from the content of different image frames, it can be said that the image sequence was blurred by pairs.

Vehicle-Mounted Electronic Image Stabilization Method
In the imaging process of the vehicle-mounted imaging system, since the imaging platform is established on the vehicle-mounted system, the vehicle traveling, vibration, attitude change and relative target displacement of the vehicle will affect the imaging system [15]. Therefore, an effective stabilization algorithm is the key to the stability of the image sequence in the electronic image stabilization system of the vehicle image system. In the process of realizing real-time stabilization technique for image sequences, the algorithm as simple and effective as possible should be sought. This section first analyzes the characteristics of vehicle-borne dynamic image, and based on experimental and theoretical analysis of the vehicle-borne image sequence, a stable, fast and effective electronic image stabilization algorithm for vehiclemounted video image sequences is proposed.

Vehicle-Mounted Gray Projection Algorithm
The gray projection algorithm is a perspective projection or orthogonal projection set of time-varying sequence images (i.e. dynamic images) of a 3-dimensional moving object, which is essentially a sequence image in which the gray level of the image changes. The algorithm first uses the projection method to find the motion vector between frames, and then performs average filtering on the motion vector sequence to conduct smoothing processing to remove the high-frequency dithering part. Besides, in order to control the error at the beginning of compensation to propagate to the subsequent frame, the motion vector corrections are performed to achieve the motion compensation of the image sequence. Therefore, it is feasible to use the gray projection algorithm for the electronic image stabilization technology of vehicle-mounted imaging equipment.

The Methods of improving accuracy
To solve the aforementioned problems, this paper proposes a solution to enhance the image sequence preprocessing, and the gray-scale projection algorithm is used after partitioning the image. First, the local motion vector and the global motion vector need to be calculated, then the motion vector is processed to remove the invalid motion vector and determine the effective motion vector, which can improve the performance of the algorithm applied to the vehicle-mounted electronic stability system.
The procedures that the local motion vector is estimated by the gray-projection method are presented as follows: at first, the image need to be partitioned and the local motion vector is calculated by using the projection algorithm for each sub-region, and the small moving object in the image sequence can be determined based on the local motion vector. As shown in Figure 7, the vector V2 is different from the motion vectors of other subareas, which is maybe due to the presence of a small moving object. Of course, how to determine the size of the sub-region is an important issue. If there are more regional divisions, it is theoretically possible to determine the range of motion of a moving object more accurately. However, the information contained in the small area is little, which makes the change information of the projection curve is not abudant, and the accuracy of projection matching will be reduced. Considering the practical application, an image can be divided into four sub-areas for a 640×480 resolution infrared captured image as shown in Fig. 8, which can meet the requirements of the image stabilization system better. The grayscale projection algorithm needs not to perform correlation calculation on each pixel of the image, but the grayscale projection curve of the image is used to perform a cross-correlation calculation to find the motion vector between the image sequence frames, and thus the calculation speed of this algorithm is high, which can meet the speed requirements of 50 vehicle-mounted electronic stability image processing per second. Through the image histogram equalization preprocessing, the image with poor contrast can be handled and the robustness of the algorithm can be enhanced. According to the characteristics of the motion, the characteristics of the local motion vector of several consecutive image frames and the global motion vector can be used to identify the scanning motion of the camera and the presence of moving objects in the background easily and effectively, thereby improving the algorithm accuracy of detecting the motion vector in practical application.

The Realization of Stabilizing Vehicle-Mounted Video Image Translation Motion
After capturing the image, the frame image needs to be stored in the image buffer as required firstly, and the selected current image needs to be input into the image motion vector detector, which is crucial for the image stabilization algorithm.
In order to calculate the motion vector of the image in the detector, the image needs to be partitioned firstly: the local motion vector needs to be calculated at first, and then the global motion vector is obtained, to remove the image variation caused by the moving target from the image. A reference image is needed to calculate the amount of motion of the image, and there are several ways to select the reference image, which can be selected in the image sequence according to the specific requirements.
Afterwards, the current image and the reference image are correlated to obtain the motion vector of the image. The output of the obtained image change amount is divided into two ways: the one is integrated into the integral element, and the other way is input to the motion decision unit, which can make the camera's artificial scanning movement extracted combined with image motion integration or using a sensor method. Finally, the detected camera's shake amount is input into the image correction link by the motion determination unit, to perform the motion compensation on the image.

Conclusions
This paper studies the theory and algorithm of electronic image stabilization technology in detail. Aiming at the characteristics of the vehicle-mounted imaging equipment and imaging environment, an effective image stabilization algorithm suitable for vehicle-borne video image electronic image stabilization was developed. The proposed technical scheme was validated through a large number of experiments. The image sequence of a vehicle-mounted camera system was written in a real-time operating system, and the real-time stable software has a stable image stabilization effect. Firstly, the motion relation expression between the vehicle-mounted camera and the image is deduced as the theoretical basis of the electronic image stabilization algorithm. The influence of the sensor movement speed and the exposure time of the imaging system on the imaging quality is analyzed, and the necessity of stabilizing the electronic image using the vehicle-mounted imaging system is proposed.
Secondly, the principle and implementation method of vehicle-mounted electronic image stabilization are introduced. The main algorithms of image stabilization are studied, and the principle of selecting image stabilization algorithm in the design of vehicle-borne electronic image stabilization system is given, which is based on the characteristics and requirements of vehicle-mounted imaging system.
Finally, the gray projection algorithm is used to achieve the stability of the translational motion of vehicle-mounted imaging equipment. Aiming at the characteristics of the vehicle-mounted imaging system and the imaging environment, the image preprocessing method is proposed, and the factors affecting the vehicle-board electronic image stabilization are analyzed. The effective method of identifying the image jitter, scanning motion and detecting small objects in the background is provided further, so that the image jitter can be eliminated correctly and the better image compensation effects can be obtaind.