Fast Linear Prediction Hyperspectral Image Visualization Algorithm

How to generate a color image with as much information as possible from hyperspectral images on a trichromatic display is a subject of great concern. In a sense, effective bands containing important information can be extracted from hyperspectral data for image visualization through the method of band selection. In this paper, two improved methods are proposed for the linear prediction (LP) band selection algorithm. According to the information entropy, the band with the greatest difference in information domain is selected as initial band.Then, the max pooling method is used to select the pixel which can better reflect the image features. Finally, the band similarity is used to select the most distinctive band for hyperspectral image visualization. The experimental results are measured by the metrics such as feature separability, standard deviation, operation efficiency and image displaying detail.The final experimental results show that fast linear prediction (LP) algorithm is an effective method for hyperspectral image visualization.


Introduction
In recent years, hyperspectral images have played an important role in the field of remote sensing and developed rapidly, which have attracted extensive attention from academia and industry.Since its high resolution makes target detection and classification more accurate, hyperspectral images have been widely used in earth science, medical image analysis, atmospheric environment and other aspects. However, displaying hyperspectral images is a challenging task [1],because hyperspectral images contain high dimensional data and can not be displayed directly on an RGB tricolour display. High dimensional data often leads to complex calculations, while adjacent bands of hyperspectral data are highly correlated, which resulting in data redundancy. Therefore, dimensionality reduction is often required in order to visualize hyperspectral data [2].
From the perspective of dimension reduction, RGB color representation can be divided into two categories [3]: one is to project data into low-dimensional space through linear or nonlinear transformation, including principal component analysis (PCA) [4], independent component analysis (ICA) [5], ISOMAP [6]and other methods. In addition, it also includes some PCA-based improved visualization techniques, such as NAPCA [7] and INAPCA [7]. Liao et al. proposed a hyperspectral image visualization based on manifold alignment [8], which projected multiple data sets into a shared embedded space, where the matching points between them were aligned in pairs. But this transform-based approach changes the meaning of the data, and the process is irreversible. Another method is to extract the most representative band from a large number of hyperspectral bands to display the image, such as one bit transform(1BT) [9]. First, unstructured bands are removed from the original hyperspectral image data according to the total number of transformations, and then three bands are selected from the remaining well-structured bands for visualization. Normalized information  [10]follows the criteria of information retention and improves the amount of information retained by visual results. LP algorithm [11]is a widely used unsupervised band selection algorithms, in which generated color display algorithm of information are very abundant, since the process of initializing band selection traverses all bands, the whole process will be time consuming. Based on the above considerations, this paper puts forward a kind of initialization band selection method based on information entropy, and through the max pooling before LP band selection method to extract the representative pixel. All of these greatly improve the operation efficiency. After quantified by objective indexes, the results show that our proposed method works well in improving image contrast and the computational efficiency of algorithm has also been improved significantly.
In addition to the dimensionality reduction method, some other methods have been proposed in recent years. For example, a new hyperspectral image visualization method based on optimization [12]can obtain a hybrid optimization strategy through the compromise between the fidelity of raw hyperspectral data and the expected high-contrast result mapping. The image fusion method based on bilateral filtering [13]use the edge retention feature of bilateral filtering to retain the tiny scenes in the image through the layered fusion scheme. Based on the image decolorization method [14], the hyperspectral images were divided into several subsets, and then the image decolorization method was used to conduct progressive fusion of bands. Based on the moving least squares visualization method [15], the transformation of each pixel is estimated by solving the unique weighted least squares problem of each spectral feature, so as to produce visual images with colors similar to the corresponding RGB images.
The rest of this paper is arranged as follows. The second section introduces the improved LP method, the third section conducts experiments, and the fourth section draws conclusions.

Initial Band Selection
The initial band selection is very important to the execution of the whole algorithm. Theoretically, two bands that differ the most and contain the most information should be used. Information entropy [16], proposed by Shannon in 1948, is often used as a quantitative index of information content of a system. Its essence is a mathematical measurement of uncertain phenomena. In hyperspectral images, it is generally considered that each band is denoted as a random variable X and all pixels in the band are treated as events of X. The entropy calculation formula of random variable X is as follows: Where x is an event of X , () X  is a set of possible values, () X px is the probability density, b is the logarithmic order, usually two. It indicates that the more likely the event is to occur, the less information it carries.It can be used as a measure of the amount of information in a system. The greater the amount of information contained in a system, the greater the value of information entropy will be. That is the higher the entropy value is, the richer the information contained in the band will be.In the process of initializing the band, the band with the most information and the greatest difference will be selected after comparing the entropy value of each band, which can avoid the complex calculation caused by traversing all bands.

LP Band Selection Algorithm and its Improvement
In this paper, the LP band selection algorithm proposed by Du Qian et al. [11] is selected for the visualization of hyperspectral images.LP algorithm is an unsupervised band selection algorithm, which compares the differences between different bands by similarity, and can jointly evaluate the joint differences between multiple bands.
Suppose there are two initially selected bands B 1 and B 2 in the subset  ,which contain N pixels respectively, and the most different bands from them can be predicted by B 1 and B 2 : Where X is an N×3 matrix, the first column of X are valued as ones, the second column includes all N pixels in B 1 , the third column includes all pixels in B 2 , and y is an N×1 vector containing all pixels in B.The band with the greatest error min e caused by using the best parameter in a is considered to be the band with the least similarity to B 1 and B 2 , and will be selected as B 3 for  .In this way, B 4 and B 5 can be selected until enough bands are selected.
The color displaying method using LP algorithm has good robustness in image quality and category separability, but the choice of representative pixels introudces unnecessary executing time.Thus, how to efficiently select pixels with more information, is a critical problem.Considering that the pooling operation based on neural network can maintain the most important information while keep the performance of the algorithm,our work introduced this method into our algorithm.
Pooling is an operation in a convolutional neural network. The idea comes from the visual mechanism, and it is a process of abstracting information [17].The method is to divide the graph into several regions and use matrix window to scan the regions and output the target pixel value.Common pooling methods include max pooling, average pooling and so on.The max pooling is to pick the maximum value of the image area as the value after pooling, which can better maintain the features on the image texture. The average pooling is to calculate the average value of the image area as the value after pooling, which can retain the features of the overall data and highlight the background information. The pooling core is a filter that can filter out unimportant pixels. The step is that the filter starts from the top left of the input array and slides on the input array successively from left to right and top to bottom, with the number of rows and columns slid each time.
The max pooling operation can expressed as:  In the pixel extraction of LP algorithm, using the principle of pooling, the purpose is to extarct important information in hyperspectral image pixels, while eliminat some irrelevant detail information, considering the average pooling cannot highlight important information,we chose max pooling in pixel extraction operation.
In addition, in the process of pooling, the filter and stride size are not fixed, and their size will determine the final performance of pooling. Although larger filter and stride will shorten the running time of the algorithm, it will also greatly reduce the visualization quality of the image.Therefore, larger pooling kernel and stride dose not mean better performance. In order to restore image details more clearly and maintain the separability and standard deviation of class features, appropriate filter and stride length should be selected.

Datasets
Indian Pines was the first test data used to classify hyperspectral images when the Airborne Visible Infrared Imaging Spectroradiometer (AVIRIS) imaged a pine test site in northern Indiana in 1992. The scene consisted of two thirds of agriculture and one third of forest or other natural perennial plants.The image size is 145×145 pixels, the wavelength range is 0.4-2.5 μm, the spectral resolution is 10nm, after excluding 20 water-absorbed bands, the data set contains 200 bands.
Like Indian Pines, the Salinas data was captured by the AVIRIS imaging spectrometer, a high resolution image of California's Salinas Valley, consisting of vegetables, bare soil and vineyards, with a resolution of 3.7 meters.We used the remaining 204 bands after excluding the 20 water absorption bands, each band containing 512×217 pixels.
Pavia University data is part of the hyperspectral imaging system developed by Germany's Reflective Optical Spectrographic Imaging System (RoSIS-03) in Pavia, Italy, in 2003.The spectral imager continuously imaged 115 bands in the wavelength range of 0.43-0.86μm with a spatial resolution of 1.3m.Among them, 12 bands did not contain information and were eliminated, so the image formed by the remaining 103 spectral bands was used.The data size is 610×340 pixels, which contains a large number of background pixels. There are only 42,776 pixels that contain ground objects, including nine categories of ground objects, such as trees, asphalt roads, bricks, pastures, etc.

.Evaluation Index
At present, there is no known correct results for comparison of hyperspectral image visualization, so there is no unified index to evaluate the visualization results. This paper chose the separability of features (SF), standard deviation(SD) [14] and the band selection process running time, in order to improve the computational efficiency,we us the sampling method to select the part of the pixels to calculate.
(1) SF represents the resolvable degree of different pixels mapped to the color space.Mainly by calculating the average distance between all the pixels, the greater the distance, the higher the resolution is.SF is defined by the following formula: , ii d x y is the distance between two pixels in the RGB color space, N is the number of pixels, and lager SF means better feature separability.
(2) SD is the standard deviation, which is used to measure the contrast of image display results.SD is defined by the following formula: x is the average value of pixels in the image, and i x is the value of the i th pixel.
(3) To assess the efficiency of the improved method, experiment was performed on an Inter(R) Core(TM) I7-7700 3.6 GHz CPU with 8GB of RAM.

.The Visualization Results of the Fast Method
In order to compare the influence of different size filter and stride lengths on the display image effect, this experiment selected filter =2, stride =1, stride =3, stride =1, filter =3, stride =2 and LP algorithm without pooling operation respectively for comparison. Speedup Ratio(SR)represents the percentage improvement over the original algorithm, and bold represents the performance improvement.On the right is a close-up view of a certain part of the image. Objective evaluation indexes of the visualization results are listed in the table.
It can be seen from Table 1 that no matter which kind of filter and stride is used, the improved LP algorithm will improve the computational efficiency, and also improve the image contrast. Figure 2 shows the selection Filter = 2, the stride = 1, in terms of visual effect and without pooling algorithm which shown in the image are basically the same, the Filter = 3, stride = 1 can very clearly see the contrast of ascension, and in the image on the reduction degree also does not have too big change, and when the Filter = 3, stride = 2, although also has a certain degree of improvement in the contrast, but very serious fuzzy on display details.  The second experiment is Pines, and the visualization result is shown in figure 3.In this case we can see, when filter = 2 and stride = 1, even class separability declined, the time and processing detail show good result. When the filter = 3, stride = 1, both SF and SD indicator are superior to other two methods.When the filter = 3, stride = 2, both objective indicators and display effect are the worst.From the close-up view, it can be seen that when filter=3 and stride=2, the details displayed in the image can hardly be seen, while the other two methods have good restoration.  The third experiment was conducted on the Salinas. As shown in figure 4, as the previous two experiments, when filter=2, stride=1, and filter=3, stride=1, the image has achieved good results in overall display and detail processing. When filter=3, stride=2, some spatial details of visual results still will be lost. As shown in table 3, when filter=3 and stride=1, the SF and SD results are both better than other methods.  Three experimental results show that the pooled LP visualization algorithm is effective. Among them, the selection of filter and stride plays a very important role in the image display. Too large filter and stride will shorten the running time, but they will not perform well in the display effect and evaluation index. The image display and detail display will have higher restoration, and the operation efficiency and contrast ratio can also be improved to some extent. In general, the effect obtained by selecting filter=3 and stride=1 is better than that of other filter and stride length.

Conclusion
In this paper, an improved hyperspectral image visualization method based on linear prediction is proposed. Firstly, the bands with high information content and the greatest difference are selected as the initial bands by using the different entropy values between bands, and then the most representative bands are selected by using the improved linear prediction algorithm after pooling operation for visualization. The experimental results show that the visual display of bands selected by the improved linear prediction method can not only avoid initial band traversal search, but also can improve calculation efficiency in the process of the band selection, and improve the image contrast of the original algorithm to some extent in the final visual display, which is an effective and fast hyperspectral image display method.