Multi-Instance Multi-Label with Application to High Resolution Remote Sensing Images

In this paper, we seek to solve the problem of remote sensing image classification using Multi Instance Multi Label Learning (MIML) framework, where each image contain multiple regions (instances) corresponds to multiple objects (labels). MIML framework is more efficient than traditional learning framework for complicated objects such as satellite images. Existing MIML algorithms such as MIMLboost, MIMLsvm and MIMLfast have been found useful in scene classification and can provide multiple labels for each instance in complicated objects. The proposed approach was performed in two steps: In the first step, we process the dataset using segmentation and feature extraction. Images are ambiguous because they contain numerous objects so we consider that each image is a bag and various blocks in the image are instances. In the second step, we apply MIML to classify each block of images. According to the experimental results, the proposed method outperforms the state-of-the-art methods.


Introduction
Recently, MIML attracts much attention in machine learning community. The MIML formalization is learning the model function for dataset , where is set of instances.
is set of associated labels { } such that n and m are sizes of and respectively . With the development of satellite technology, the plentiful information provided in high resolution images totally changes the perspective of satellite image classification. HR image as one individual object with only a single semantic label. Actually, with more information the HR image contain numerous semantic meaning and multi-labels emerged from image segmented regions instead of the entire image. Feng [1] introduced an algorithm of multi-instance semi-supervised learning with hierarchical sparse representation. Liu and al [2] presented the key instance detection task and propose a new framework as solution based on the relationship between instances. T. Gä rtner [3] used multiinstance bags after extraction of feature vector in order to prepare multi-instance kernel function. In order to predict one label of multi instance bags, MIML framework was proposed by Zhou et al. [3,4], which has advanced toward becoming another exploration area. MIML framework provides a useful formulation for HR satellite images. In MIML HR images are described as bags, each bag contains many instances associated to segmented regions of the original image. Labelling a bag as negative required that all its instances are negative. We defined a bag as positive if at least one from its instances is positive. The rest of this paper is organized as follows: In section 2 we introduce the related works for both remote sensing image classification and other application of MIML framework. In section 3 we discuss the experimental results with preparation of dataset and evaluation criteria. The experiments and result analysis are given in section 4. Section 5 offers our conclusion.

MIML
Multi Label Learning (MLL) [5,6], can be used in classification problems, where each sample can be mapped to various classes at the same time. The main function of MLL is to learn the classifier: . In a set of MLL training set: { } is the i-th instance and is the label set of . In [7], Chaudhuri B proposed semi-supervised graph-theoretic method related to the approach of multi-label image retrieval problem. This approach diffuses the labels in the training images using a label correlation propagation technique to the unlabelled image in order to map them with multiple labels. In order to recuperate the images out of archives with the same classes of regions as the image in question. MLL has been applied in many real world fields, such as medical, banking, text classification and ecological studies [8,9]. Numerous methods are proposed in order to solve the task of MLL for instance: ML k-Nearest neighbour, ML decision tree and ranking SVM are proposed in [10].
Multi Instance Learning (MIL) model was formalized by Dietterich et al, in which they propose algorithm for dry activity prediction [11]. The main function of MIL is: From a set of MIL training set ; is the label of the bag . Jaume amores [12], present comparative studies about multi-label solutions of MIL.
Zhou et al [13,14] introduce two MIML algorithms, MIMLboost and MIMLsvm algorithm respectively and their application in dataset scene classification. Li et al [15] proposed a Key Instances Sharing among Related labels (KISAR) algorithm which can find key-instance activating several labels in a bag. In [16] they propose the RankLossSIM approach to optimize with label ranking loss .To effectively deal with large dataset and minimize time complexity of MIML, MIMLfast algorithm is proposed with an approximated ranking-loss by using stochastic gradient descent (SGD).

Remote sensing Images
Remote sensing data is about the object or phenomenon through devises sensor without contact the objects physically. The use of satellite sensors is to detect and classify ground objects for the interest of numerous fields: including geography, geology, military, intelligence and humanitarian application.
In fact remote sensing is referred to satellite or aircraft sensor devises, according to the process of emission/reception of signals, remote sensing is divided into active (signal is emitted by satellite and reflection is used in the sensor) and passive (reflection of sunlight is detected by sensor).

Feature extraction methods
 Discrete Cosine Transform (DCT) is applied for each block after subdividing the image into regions of 8x8 pixels. The precision of classifiers on DCT features extraction is better than statistical and texture feature extraction techniques. The main idea of DCT is that the intensity of an image is often high in the low frequency elements. Generally the DCT only uses the coefficient with the largest values to reconstruct the original image [17].  Local Binary Pattern (LBP) corresponds the functions of the descriptor, i.e. a local neighbourhood is started from gray scale of middle pixels into a binary Pattern [18].  Gabor texture features is efficient for analysing remote sensed imagery [19]. They were shown to surpass other texture features in the images classification. Gabor method uses a set of scales and orientations filters to extract the texture features [20]. We summary the general process of object detection and mention the step of previous feature extraction methods in Figure 1.

Proposed method
Remote sensing images are complicated where each sample can correspond to multi-instances and multi-labels at the same time. For example, a satellite image contain several region, each one can be associated to an instance. Simultaneously the image can have the part with buildings and trees. In this paper we apply MIML into remote sensing images. When we learn the model by numerous bags; each bag has several instances and several labels as well.
MIML is the continuation of classical supervised learning (MIL and MLL). In MIL [21] an object presented by several instances is attached to one class label. The training dataset used under the form , where is set of instances is and number of instances, is the label of . In MLL [22] an object presented by one instance is attached with numerous class labels. The training dataset used under the form , where is an instance and is an array of labels in class label L. MIML framework provides the mapping between objects instances and corresponding labels as show in figure 2. The classification on label is defined by the function : , is a m d matrix, is m-dimensional weight vector corresponding to label , d and m are the sizes of the feature space and the shared space successively. The prediction of instance x on label as: (1) According to Dietterich, a bag is positive equivalently it contains au minimum one positive instance, the prediction of a bag X on label is defined by the following function: . The basics parameters of gradient descent are as follow: Such that is the step size of SGD, , and all column are standardised to Norm L2.  -; 9: v = i 10 : update as Eqs. 4 to 9, and perform normalization; 11: break 12 : until stop criterion reached 13: OUTPUT: Relevant labels set for the test bag X test is: Figure 3 show the scenario of image classification and the illustration according to selected approaches (MIL, MLL, MIML). MLL Principe associate the labels (building, trees and water) to the input image by founding the class objects in the image and fill the matrix instance-label by 1(-1) if it exist (not).MIL approach link multi sub images with one label if and only if at least one class object exist in the numerous sub images. Finally MIML provide the opportunity to label the image linking each sub image with its conform label and this is the developed approach to classify the image with complicated objects.

Data collection
The dataset we used contains 2000 high resolution images affiliated to the classes: ground, water, buildings, trees and roads. We collected those images from Google earth in the area of California USA, more than 90% belong to multiple classes in the same image. We obtained 2000 sub-images after segmentation of original images. Totally we use 1800 samples for training and 200 for test, for each sample we divide it into four regions and extract the features using dct features. This project required to have labelled dataset in order to train the model and evaluate its performance. The images are resized to be a uniform scale for the object extraction.

Multi label Performance criteria
Multi-label classification problem needs special criteria for evaluation different from traditional signal label classification that uses just accuracy as performance evaluation criteria [23,24]. In the experiment part we employ five criteria Hamming loss (h.l), One-error (o.e), Coverage (co), Ranking loss (r.l) and Average precision (a.p) to check the performance of the proposed MIML classification method. These criteria are defined as follows:  Hamming loss: calculate how frequently an instance-label pair is wrong classified.
is the difference between two sets .The less value, the improved achievement . Ranking loss: Calculate the medium portion of labels that are not correctly requested.
Where the de notes the supplementary set of in C. The performance is ideal when RankLoss 0, The less value, the improved achievement.  Average Precision: Calculate the medium portion of labels classified to a certain label which actually is in . Table 1 describes the experimental results of many feature extraction methods in order to compare the evaluation performance.   . Performance criteria (h.l,o.e,r.l,co,a.p) versus size of dataset according to three following methods MIMLfast, KISAR and MIMLsvm . It is essential to evaluate the effectiveness of the model trained with MIML approach, for this purpose we calculate the five MIML criteria as shown in Figure 4. Finally we conclude that MIMLfast give more accurate result than two other methods (MIMLsvm, KISAR) especially with LBP feature extraction method.

Conclusion
MIML approach is the appropriate for Remote sensing object recognition thus it is created for learning the complicated objects. MIML can be comprehended by transformation MIML problem into its corresponding in classical supervised learning using MIML or MLL as intermediate. Although, this transformation may miss information in ignore the relation between labels. When taking in consideration the relation between labels in common area and finding sub-ideas for complex labels. Comparison results demonstrate the efficiency of proposed method in terms of classification accuracy. Future work will cover the optimization of hamming loss function by implementing active learning in the process of training the model.