Method of multispectral image fusion using wavelet spectra

The method of image fusion for images of the same scene obtained in different spectral ranges with the formation of an integrated monochrome image is considered. The fusion method is based on the wavelet decomposition of the original images. A strategy for fusing detail coefficients by comparing the proportions of their magnitudes for all original images is proposed. The fusion procedure does not require any thresholds. The algorithm does not introduce additional distortions and collects all the necessary information from the original images. The proposed algorithm can be used in optoelectronic systems for automatic image processing.


Introduction
Multisensor systems for target detection and tracking are now widely used primarily because of their ability to work in conditions of strong hindrances, which can be natural or artificial. Each individual channel is not able to meet technical requirements in conditions of poor visibility, masking targets and the use of electronic countermeasures. However, the efficiency of the system is increased by common use of information from different channels. The composition of optical-electronic equipment usually consists of television, thermal (multi-band), laser and other channels [1,2].
The information fusion is divided into three levels: pixel fusion, feature fusion, and decision fusion. The pixel fusion is low level integration, which is applicable for the sensors with compatible receiving rate, dimension and format of data. The algorithms for image fusion can be divided into two groups: pseudo-color image generation algorithms and true-color image generation algorithms. Pseudo-color images are mainly used for remote sensing data processing and in medical applications [3,4].
In this paper we consider the fusion of images of the same scene obtained in different spectral ranges to form an integrated monochrome image. The main problem of compiling images for visual display is «saving a picture» [5]. The important details from all compiled images should be presented in the resulting image, and the compilation procedure should not introduce distortions. The visual quality of the compiled image should be at least as good as the original image when other channels of the system are blocked. The compilation method should work with any number of observation channels, regardless of their spectral range and combination.
Earlier methods for compiling monochrome images were based on a weighted summation of the original images with some weights [6,7]. The fusion of essential details was provided by adding of maximum brightness contrasts from all channels to this linear combination.
In the later algorithms, significant brightness differences in each of the original images are detected and combined in the integrated image. In recent years, many methods based on multi-scale transforms have been proposed for image fusion. These methods include contourlet transform [8], curvelet transform, the discrete wavelet transform [3,6,[9][10][11], and various pyramid algorithms. The wavelet transform differs from the Fourier transform in that it allows local spectral analysis. Spectral wavelet IOP Publishing doi:10.1088/1757-899X/1019/1/012058 2 coefficients correspond both to the amplitudes of different frequencies and to different spatial regions of the image. The advantage of wavelet analysis for image fusion is the ability to use images of various spatial resolutions to reduce the amount of information processed.

Image fusion algorithms based on the wavelet transform
We consider the image fusion methods based on the wavelet transform described in [3] and [6,10]. The wavelet system Haar, which requires a minimum of calculations, is chosen as the basis in these works. A separable two-dimensional basis, obtained by integer translations and binary scaling of basic functions, is used.
All original images are processed by two-dimensional wavelet decomposition: each row is divided into low-frequency and high-frequency halves, and then each column is processed similarly. The result is four half-sized arrays from the original: Wϕ are approximation or scaling coefficients, W H , W V , and W D are detail or wavelet coefficients.
When the arrays of coefficients are formed, there are several ways to restore the resulting image. The approximation coefficients for the fused image can be taken from one (pre-selected) original image. They can also be obtained by averaging or weighted averaging the approximation coefficients from all original images. The detail coefficients can be formed from the maximum magnitudes of the corresponding coefficients for all original images. They can also be selected based on some feature. In any case, the brightness range of the original images should be normalized before processing.
References [6,10] proposed that the approximation coefficients were formed as an average of the coefficients of the two input images, and the detail coefficients were composed of maximum magnitudes. In reference [3] the correlation of the approximation coefficients for input images were tested. The array of approximation coefficients is formed from the maximum magnitudes, if the correlation is low, and, otherwise, is a linear combination of coefficients from two input images. High frequency component of transformation changes more intensely, and the authors [3] used the fusion strategy based on variance for the detail coefficients. The detail coefficients are selected from the image, where the variance of the coefficients calculated over the local neighborhood is greater.
Fusion weighting coefficient can be calculated as where σ1 and σ2 are respectively standard deviations within the field (from 3×3 to 7×7) from detail coefficients for the first and second images. If Tvar is less than the specified threshold t, then the detail coefficient is taken from the image where the standard deviation value is maximum. If Tvar is greater than the threshold value, then the detail coefficient is a linear combination of coefficients from two input images: 1 ), /( , where W i are the detail coefficients for the fused image, W1 and W2 are the detail coefficients for the first and second original images, i = H, V, D. The described algorithms were simulated. The simulation results showed that when the detail coefficients are composed from the maximum magnitudes [6,10], the synthesized image contains too many small details. This complicates the analysis of the fused image and makes the method unstable with respect to noise. On the other hand, the formation of approximation coefficients [3] from the maximum magnitudes is possible only for images with similar brightness. When fusing TV and thermal images, artifacts in the form of light spots appeared in the resulting image. The fusion strategy for high-frequency component of wavelet transform according to formulas (1), (2) gives good results, but it requires thresholds setting for comparing fusion weighting coefficients with them. During the simulation, the thresholds were calculated by preprocessing the image. Thus, the fusion procedure became two-pass.
We propose a new fusing method for detail coefficients, which does not require threshold values. The method is based on comparing the proportions of the detail coefficients. If the considered point belongs to the edge, the magnitude of one of the detail coefficients will be greater than the others. The greater the difference, the more visually significant the boundary will be. On the other hand, if there is a point from the neighborhood with constant brightness, the magnitudes of the detail coefficients are almost equal. We introduce the fusing rule as The original image and the image with added Gaussian noise are shown in Figure 1. The results of merging these images are shown in Figure 2. Approximation coefficients were taken from the first original image. The detail coefficients were formed from the maximum magnitudes -the result is shown in Figure 2a, according to equationes (1) and (2) -the result is shown in Figure 2b. The fusion result image by the proposed method according to equation (3) is shown in Figure 2c.
The correlation coefficients are calculated for the first original image and all fusion results. They were 0.7842, 0.7844, and 0.9462 for the images in Figure 2 a -c. Thus, the considered method is more resistant to noise than the compared methods.

Experiment and result analysis
The original multispectral images for the fusion by the proposed algorithm are shown in Figure 3. Image fusion can be performed at any level of the wavelet decomposition. The fusion result image according to the considered algorithm using a one level of the wavelet decomposition is presented in Figure 4. The fusion result using two levels of decomposition is shown in Figure 5. Note that only the detail coefficients were combined. The approximation coefficients were taken from the first original image.    The resulting image contains details that are indistinguishable in one of the ranges. Using two levels of decomposition can significantly reduce processing time. However, increasing the number of decomposition levels can be used to compile images with large objects. The mosaic effect appears on small objects, and smooth curved lines become jagged.

Conclusion
The result fusion images according to the considered algorithm are convenient for the operator's perception. The algorithm does not introduce additional distortions and collects all the necessary information from the original images. The fusion procedure does not require thresholds. The proposed algorithm can be used in optoelectronic systems for automatic image processing.
This paper does not consider the possibility of further improving the quality of the generated images, in particular, due to directional filtering during wavelet decomposition or filtering of wavelet coefficients [12]. Further studies are aimed at adapting the algorithm by combining approximation coefficients to improve the quality of the result.