Transfer Learning with Convolutional Neural Networks for SAR Ship Recognition

Ship recognition is the backbone of marine surveillance systems. Recent deep learning methods, e.g. Convolutional Neural Networks (CNNs), have shown high performance for optical images. Learning CNNs, however, requires a number of annotated samples to estimate numerous model parameters, which prevents its application to Synthetic Aperture Radar (SAR) images due to the limited annotated training samples. Transfer learning has been a promising technique for applications with limited data. To this end, a novel SAR ship recognition method based on CNNs with transfer learning has been developed. In this work, we firstly start with a CNNs model that has been trained in advance on Moving and Stationary Target Acquisition and Recognition (MSTAR) database. Next, based on the knowledge gained from this image recognition task, we fine-tune the CNNs on a new task to recognize three types of ships in the OpenSARShip database. The experimental results show that our proposed approach can obviously increase the recognition rate comparing with the result of merely applying CNNs. In addition, compared to existing methods, the proposed method proves to be very competitive and can learn discriminative features directly from training data instead of requiring pre-specification or pre-selection manually.


Introduction
Automatic ship recognition is an important link of the marine surveillance systems, which has been used in fisheries management, marine rescue, traffic control, military operations etc. In particular, Synthetic Aperture Radar (SAR) is often employed in ship recognition due to its ability to penetrate various weather and light conditions. OpenSARShip [1], a benchmark dataset composed of different types of SAR ship chips, is publicly available for development and evaluation of SAR recognition methods. Traditional ship recognition system is divided into three key points, image preprocessing, feature extraction, classification and identification. Image processing mainly includes speckle noises filtering and selection of interested areas containing recognition targets. From the segmented images, feature extraction is used to compute features for each type. In present, the most basic used features for SAR ship identification are geography [2], moments [3], scattering statistics [4,5] [6], which have shown a great success in the field of optical image classification. However, the successful application of this technical needs amounts of training data to adjust the numerous parameters of CNNs automatically. This property prevents the application of CNNs to SAR images for the limited accessibility of annotated training data.
To address the forementioned issue, we introduce a transfer learning approach. Transfer learning, which aims to learn knowledge from related sources to complete target tasks [7], has been proved to be quite successful in the past years and is becoming a hot research area in deep learning [8,9]. Our work is to leverage available datasets MSTAR to extract features that are useful for ship recognition. Specifically, we start with a CNNs model pre-trained for object recognition on MSTAR and learn a modified network on SAR image dataset. The experimental results demonstrate that the recognition performance can be improved by introducing a transfer learning procedure. Besides, the CNNs based method learns discriminative features automatically and achieves impressive performance in object recognition compared to existing methods. Below in Section 2 we present our proposed method CNNs with transfer learning (CNNs-TL) and discuss experimental results in Section 3.

Transfer learning with CNNs
The proposed method is a combination of a CNNs and transfer learning. In this section two different parts are described independently and the combination of the two systems is shown in the last part. Readers can refer to [10] for the detailed description of CNNs and find further information about transfer learning in [7].

Convolutional Neural Networks
Convolutional Neural Networks (CNNSs) requires very large numbers of parameters learned from training [8]. CNNs typically involve alternating convolutional layers and pooling layers followed by fully connected layers, where convolutional layers extract features and pooling layers generalize features. The convolutional layers will have n filters of size k × k × q where k is smaller than the dimension of the input and q normally vary for each kernel.
In this paper we built a VGG-like convolutional model as shown in Table 1. This network contains a total of 7 layers with learnable parameters, which composed of 2 convolutional layers, 2 pooling layers and 3 fully connected networks. Each convolutional layer has filters of size 3 × 3 and a Rectied Linear Units (ReLU) activation. effective method when employing CNNs to many fields. CNNs model is well suited for this transfer learning setting since the features learned from a task may be useful for other task. Specific to our ship recognition problem, the layers trained on MSTAR can be reused to extract the mid-level features of images in OpenSARShip dataset effectively.

Transferring CNN weights
Our network training pipeline involves three main steps, as shown in Figure 1. First, we start with a CNNs network pre-trained on the MSTAR dataset. This network is trained to classify source targets given the entire images rather than pre-selection features as input. Each image contains one ground target occupying large portion of the image. Next, we load all the parameters of former 6 layers trained on the MSTAR. Finally, we fine-tune on the OpenSARShip dataset after fixing parameters of different numbers of layers, as illustrated in Figure 1. The images in OpenSARShip, however, depict different scales and distributions with MSTAR. It is noteworthy that the retrain portions of the network can be adjusted to achieve better recognition performance. After network training, we use the test samples to evaluate the transfer learning results.

Experiment and discussion
In this section we first describe details of datasets and experimental environment. Then we give experimental results of our proposed CNNs-TL on the target task. To estimate the effectiveness of our proposed method, we finally compare the results with other traditional recognition methods.

Benchmark Data and Environment
In this paper we use OpenSARShip dataset as our target domains. The OpenSARShip dataset consists of 11346 SAR ship chips collected from 41 C-band Sentinel-1 images. In our paper 5432 samples (2716 VH/VV channel chips respectively) cover three types of ships (tanker, container ship and bulk carrier) are used to perform our target task. The original data are downloaded from the Sentinels Scientific Data Hub [11] and the corresponding AIS messages are acquired at the same zone in a 20 min window whose center is the SAR acquisition time [12].
Our source MSTAR dataset consists of X-band spotlight-SAR images for multiple targets. For each target, images collected at two different depression angles (15° and 17°) are both used for model training (2987 images totally).
The benchmarks were performed on a dual-socket system running Centos release 6.8, including two Intel Xeon E5-2640 server CPUs running at 2.4 GHz with 10 cores each, and was equipped with a total memory of 128 GB. The GPU acceleration was performed on the NVIDIA Tesla K80.

Recognition Accuracy Analysis
To display the advantage of the application of transfer learning, we arrange a group of comparison experiments, which are CNNs alone and CNNs+TL respectively. CNNs alone and transfer learning results with different fixed layers. We mixed three types chips and splited them with 70% for training and 30% for testing. In addition, 5-fold cross validation method was used to search the optimal parameters.  Figure 2 shows the 3-class targets confusion matrix of CNNs alone and CNNs+TL. These results show that a simple transfer learning procedure results in about 2.2% increase in recognition performance (VH Channel). As shown in Figure 2, the proposed method achieves better correct recognition accuracy than CNNs alone among all the three categories. As shown in Table 3, we compare the final recognition accuracy of CNNs-TL and some traditional methods results of the published paper [12]. Our proposed method CNNs-TL achieved rates of 77.63% (VH Channel) and 79.12%(VV Channel) respectively, which are better than the other methods.

Conclusion
In this paper, we proposed a method based on transfer learning for SAR ship recognition. Experiments were conducted to explore the influence of the application of transfer learning, and a recognition rate of 77.63% on VH Channel dataset (79.12% on VV Channel dataset) was obtained on the OpenSARShip dataset. The experimental results demonstrate that a simple transfer learning procedure could yield better SAR vehicle recognition results. Additionally, our model can be productively effective even when data on key outcomes of interest are not sufficient. Furthermore, our approach can easily be generalized to other remote sensing tasks and has great potential for SAR image classification.