Satellite image super-resolution reconstruction based on ACGAN and dual-channel dense residual network

For solving the problem of simple information image super-resolution reconstruction, this paper proposes a method based on ACGAN and dual-channel dense residual networks. Firstly, the different information represented by the images is classified to form different types of datasets. Focusing on image feature information, ACGAN is used to learn image feature information to generate images with this kind of feature information in this type of dataset so as to achieve image enhancement. Secondly, the study designed and proposed a dual-channel dense residual network to train the enhanced image dataset and achieve image super-resolution reconstruction. Experiments show that the method proposed in this paper not only can obtain higher reconstruction image quality than other methods but also obtain higher PSNR and SSIM than others. From this point of view, the application of this technology will have a far-reaching impact on the research of image super-resolution reconstruction with simple and sparse feature information.


INTRODUCTION
In recent years, the research on SISR reconstruction based on convolutional neural networks and deep learning has become the mainstream trend, and the current research mainly focuses on natural images.However, in the current research on image super-resolution reconstruction, rich image feature information is mainly used, but there is a lack of effective research on simple images with less feature information.This paper proposes a composite method-an image super-resolution reconstruction method based on Auxiliary Classifier Generative Adversarial Network (ACGAN) and a dual-channel dense residual network (DCDRN) to research SISR for small target satellite images.This comprehensive method is divided into two parts.First, through the ACGAN structure, the image feature information of different categories of datasets is used as the core learning content to generate new images of corresponding categories to achieve image information enhancement.The second is to use the dilated convolution design to construct the dilated convolution residual block and use the residual block design to construct a dual-channel dense residual network to achieve image super-resolution reconstruction.The role of dilated convolution is to expand the receptive field of the residual module to the feature image without introducing additional parameters, retain more image features, and deepen the network structure.
The research content of this paper is as follows: (1) The ACGAN model is proposed and constructed, and it is used to generate more images with similar feature information for images with simple feature information in different categories of datasets so as to enhance the feature information of images.
(2) The residual block is improved, and a dilated convolution residual block is designed and built.The block is used to propose a dual-channel dense residual network to achieve the target images superresolution reconstruction.
The parts of the article are arranged as follows.Section 2 describes the technical development background of convolutional neural networks and deep learning technology research on SISR reconstruction.In Section 3, the study proposes a comprehensive method for image super-resolution reconstruction of ACGAN and dual-channel dense residual network.The experimental content of the method proposed in this paper and other network model methods on the corresponding datasets are in Section 4. And in Section 5, we present final conclusion and future work.[1] and Enhanced Deep Residual Networks (EDSR) were applied [2], as well as the SRDenseNet model constructed by the subsequent DenseNet structure [3].Lai et al. combined a series of structural features and combined with the image pyramid structure to propose a more complex Laplacian pyramid [4] image super-resolution structure to achieve multi-scale super-resolution reconstruction of images.

RESEARCH RELATE WORK
Goodfellow et al. used the residual structure to design and build a generative adversarial network (GAN) for the first time.By training the network, the generator and the discriminator reached Nash equilibrium to achieve network stability.The main feature of GAN is that it can generate a large amount of relevant data to achieve quantitative enhancement of research objects.Ledig et al. first used GAN to study single-image super-resolution and proposed the SRGAN algorithm model.With wide application and continuous development and improvement of the GAN model [5], a series of algorithm models such as BiGAN [6], BGAN [7], CycleGAN [8], and DCGAN [9] have emerged according to different uses.The ACGAN [10] model applied in this paper is also one of the GAN technology models, which mainly classifies and generates image features.
In recent years, with the continuous development of SISR research technology, the single algorithm network model has been gradually eliminated due to its simple structure.Complex deep networks and composite structures algorithmic models have gradually become a research trend.The image superresolution reconstruction technology proposed in this paper is also comprehensive.First, the target image is enhanced through the ACGAN, and then the image is reconstructed through the network to generate a high-resolution image.

PROPOSED METHOD
This study proposes a single-image super-resolution reconstruction method in the form of a multinetwork synthesis.First, image enhancement is performed on the target image, followed by a dualchannel convolutional neural network, and finally, high-resolution images are output.The entire image processing process is shown in Fig. 1.

Improved residual block
In this study, the residual block is improved and redesigned; the dilated convolution form is introduced during the convolution processing inside the block.A new residual block is designed by using the dilated convolutional layer.The advantage of dilated convolution is that it can enlarge the receptive field on the feature image without additional parameters and at the same time, can deepen the structure of the neural network.The structure of the designed dilated convolution residual block is shown in Fig. 5.In the residual block, convolution and dilated convolution are densely connected in series.The image information of the previous layer is passed to the next layer, like flowing water layer by layer, and is fused so that the block output the image information after all layers are completely superimposed and fused.
The degraded images are reconstructed in the two residual network channels, respectively.Before entering the residual module network, the shallow convolution of two standard convolution layers is required to extract more feature details so that the network can be trained more easily, and the process is as follows:   1 ( ) H   and 0 ( ) H  are convolution functions in corresponding layers, respectively.After shallow feature extraction, it enters the residual network composed of dilated residual modules.After intensive feature fusion among convolutional layers in each residual block, each channel transmits the feature information output by each residual block to the feature fusion layer for image feature fusion processing, that is, , , , After the convolution layer of 1 1  convolution kernel fuses the output features of each module with block features, each channel outputs the corresponding reconstructed image  ( 1,2) , which is: The parameter  in ( 6) is the mapping function of the residual network for each channel.After the two-channel residual networks output the reconstructed images respectively, they finally fuse and upsample the output to reconstruct the high-resolution image Î , where  indicates up-sampling.

Evaluation standard index
The performance of the SISR network model is determined by the quality of the generated image.The generated image reflects the quality of the image through relevant technical indicators.At present, the image evaluation indicators are peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM).The higher the PSNR value is, the closer the SSIM is to 1, indicating that the generated image quality is better, and the algorithm model performance is better.

Experimental environment
The hardware working platform used in this paper is a computing workstation equipped with 16 GB RAM NVIDIA GeForce Quadro P5000 GPU, Intel 3106 CPU for 1.70 GHz, and 64 GB ROM.The software on the experimental platform is 64-bit Windows 10 Professional Edition, CUDA 10.0, and cuDNN 7.4, and it is compiled in the Pycharm compilation environment with Python.

Datasets
In this research experiment, the public data sets of Set5 and Set14 are used to compare the performance of the comprehensive method proposed in this paper and the classical model; the Tiangong-1 hyperspectral full chromatographic satellite images are used to show the performance of the text method and the classical model on satellite image super-resolution result.

Parameter settings
In this experiment, the satellite image objects were divided into 4 categories, and each category had 60 gray-scale images of 128 128  hyperspectral full spectrum chromatography satellite images.In the dual-channel residual network, each channel consists of 8 dilated residual modules with a total of 64 layers, the initial learning rate is 4 10  , and the batch size is set to 8.  Ⅰ.It is obviously found in the public dataset that the comprehensive method has a higher PSNR value and SSIM value than other methods, and the comprehensive result is better.

Results and analysis
In the Tiangong-1 satellite image dataset of this research object, for the architecture target image category among the four categories separated, the proposed method and other methods are used to perform image super-resolution reconstruction with an amplification factor of 2  and obtained different images, respectively.The results of different super-resolution reconstruction images are shown in Fig. 7, and the PSNR value and SSIM value of the obtained image are listed in Table Ⅱ.
In addition, we find that without the enhancement of ACGAN, image super-resolution reconstruction only using the dual-channel dense residual network can indeed obtain a good result.Still, the degree of improvement is small.At the same time, the comprehensive method of image enhancement through ACGAN and image super-resolution reconstruction through a dual-channel dense residual network can obtain higher improvement results.

CONCLUSIONS
We introduced the relevant neural network methods and technical developments of single-image superresolution at the beginning of the article.Then we proposed a comprehensive method for image superresolution reconstruction by ACGAN and dual-channel dense residual network.Afterward, through the experiments, we obtained the relevant experimental data results.Finally, through comparative analysis, it is proved that the method proposed in this paper has higher evaluation index results and better image quality.

Fig. 1
Fig. 1 The flowchart of image super-resolution reconstruction proposed in this study 3.1 ACGAN structure ACGAN was proposed by Odena A et al. at the 2017 International Conference on Machine Learning.As a technical variant of the GAN model, the main function of ACGAN is to generate images according to requirements so as to achieve the purpose of image data enhancement.Fig.2is the ACGAN structure.In image generation, the network does not need to label the images in the original image sample library but adds category discrimination.The process of generating images by the network is as follows: firstly, the images are classified in the original image samples C , and are sent into the ACGAN model according to the categories; then, in the generation network structure G , the generation network generates( , )  G C Z under the received noise signal Z get fake image fake X

Fig. 2 Fig. 3
Fig. 2 Schematic diagram of ACGAN structureIn ACGAN, the generator and discriminator have different structures and functions.The structure of the generator and discriminator in ACGAN is shown in Figs.3 and 4respectively, where the generator network structure is densely connected by 15 processing blocks, replacing the convolutional layer connection mode of GAN.This method can effectively connect image processing information closely,

Fig. 5 Fig. 6
Fig. 5 Structure diagram of dilated convolution residual block3.3Dual-channel dense residual networkIn this part, a dual-channel dense residual network model is designed and constructed by using the improved dilated residual block.Fig.6shows the proposed dual-channel dense residual network model structure.

7 Fig. 7 2 
Fig. 7 The image results obtained by the proposed method and other model methods for scale factor 2  super-resolution reconstruction of architecture images The first to introduce the convolutional neural network (CNN) method for SISR technology research is the Super Resolution Convolutional neural network (SRCNN) model method proposed by Dong et al. in 2014.In further research, Dong et al. improved the structure of CNN and proposed a further FSRCNN technology model to improve the quality of generated images.With the continuous increase of network structure, deep network structure and deep learning technology appear.To solve the problems of unstable network training and image information distortion caused by too deep a network structure, He et al. proposed the concept of residual for the first time.They proposed the residual network structure to ensure the network is stable and easy to train without more parameters.Kim combined the deep convolutional neural network and residual network to propose a VDSR image super-resolution reconstruction model with a 20-layer network.Under the influence of this technical idea, a series of network model algorithms such as Deep Recursive Residual Network (DRRN)

TABLE I .
THE RESULTS OF THE PROPOSED METHOD AND DIFFERENT IMAGE RECONSTRUCTION METHODS ON DIFFERENT PUBLIC DATASETS

38.57/0.9617 34.12/0.9217 ACGAN+ Dual- channel 2  39.26/0.9705 35.49/0.9353
The proposed method and other model methods are used to perform the reconstruction experiments with the parameter 2 on Set5 and Set14.The results of the method proposed in this study and other algorithm model methods are listed in Table

TABLE II .
THE PSNR AND SSIM VALUES OF DIFFERENT METHODS OBTAINED BY PARAMETER 2