Graph Extraction of Batik Image Using Region Adjacency Graph Representation

Image feature extraction done in image analysis based on the traditional pixel is not extracted effectively. This is due to the extraction that only represents the content. A very promising approach and the challenge are to extract a graph from an image that represents the content and their relationship. The results of graph extraction are obtained from the process of image segmentation. The selection of an appropriate segmentation method from many graph-based image segmentation methods is interesting to be reviewed. The method of super-pixel segmentation is one way to divide the image into regions. The regions obtained in segmentation are represented by vertices and edges represent connections between adjacent regions. This representation is called Region Adjacency Graph (RAG). In this paper, graph extraction of “batik” images has been successfully performed where its representation uses RAG and segmentation uses the Minimum Spanning Tree method. The results of graph extraction have been tested in a simple image retrieval process using the VF2 graph matching method.


Introduction
Recently, graphs have been increasingly used as a representation way for image processing and analysis [1]. Graphs are used to represent information consisting of entities that are described as vertices and relationships between entities that are described as edges. In the field of pattern recognition, these representations have been used since the late 1970s to recognize objects. In addition to images, 2D / 3D shapes, documents, symbols, and characters, these representations are also used for chemical or biological structures, semantic web and web, social and economic networks, and many others. [2].
There are several reasons why image processing and analysis is carried out based on graphs. The first reason is its simple representation which is suitable for efficient method development. The second is that different types of images can be represented flexibly. The third is the reuse and redevelopment of graph theory algorithms and theorems in the field of image analysis and the fourth is the addition of high-level information to low-level representations (related to the semantic gap problem) [3].
Image segmentation is the initial stage in image graph extraction. It will form a homogeneous set of regions. Each region has a set of connected pixels and all pixels in a region have one feature set [4]. Extracting a very comprehensive image feature that can reflect image content is still a challenging problem [5].
In this paper, the extraction of the batik image graph is performed. It is expected to become the basis for analyzing the image of batik based on the graph method. The resulting graph extraction will be tested using graph matching through a simple image retrieval example using the VF2 algorithm. This can contribute to maintaining one of Indonesia's cultures by providing a suitable batik graph dataset. The IOP Publishing doi:10.1088/1757-899X/1077/1/012006 2 contribution of this research is to become a model for forming a graph dataset of batik images. This dataset can be used to quickly and accurately classify and recognize various types of Indonesian batik.
In the Great Dictionary of Indonesian Language, batik has the meaning of illustrated fabric that is made specifically by writing "Malam" on the cloth, then processing it in a certain way. Batik is a familiar thing for Indonesian people. Batik is one of the unique archipelago heritage that must be preserved and has also become a world cultural heritage since 2009 which was confirmed by UNESCO. The uniqueness of batik is shown in various kinds of motives that have their own meaning.
The paper will be organized into several sections. The first part is an introduction that explains graph extraction. The second part contains the related work of research on this topic. The third part is the theoretical basis that underlies the extraction of image graphs such as graph theory terminology, segmentation, region adjacency graph, and VF2 algorithm. The fourth part is the results and discussion which is followed by the testing phase on image retrieval. The last section is a conclusion and suggestions for future work.

Related Works
There are several advantages of graph-based data structures that can be used to find relationships between objects [6] and can be applied to statistical methods and machine learning in graph theory [7]. Promising research in this area is interactive visual data mining together with graph-based data analysis [8] Many methods have been proposed to obtain graph representations of objects. It depends on the purpose of the application. The difference between all these methods is in the scale of operation, but generally, in all these methods, the parts of an object in the image are represented by one node [2]. Holzinger extracts the graph structure from the image using HTML canvas and Javascript which is run on a Web browser [5]. Sharma determined the similarity of the histological image from the breast biopsy image using inexact graph matching where the results of the retrieval image made were superior to the histogram-based technique. [9] There are methods that use a different node for each image pixel. These methods form a graph with a large number of vertices where the edges have a regular structure. An example is the Boykov and Kolmogorov algorithms which perform graph-based segmentation [10]. On the other hand, there are nodes that represent the whole object, or important parts of the object, which are generated from several levels of segmentation. Nodes represent a homogeneous region (e.g. Graph Adjacency Region Tremau and Colantoni) [11].

Definition and Terminology
There are a lot of literature and works that have been done related to segmentation based on the graph method. In this section, we briefly discuss the method for this work: some basic things about graph theory, segmentation whose region is used as a node for the Region Adjacency Graph representation, Minimum Spanning Tree algorithm, and graph matching using the VF2 algorithm

Graph Terminology
A graph is a set of elements and relation pairs between two elements. Nodes represent elements and and the edge i with eij ∈ E, and also denoted eij = {vi, vj} [1].
There are several graph terminology as the basis for processing and analyzing images with graphs Here are some basic definitions of graph theory such as adjacent, incidence, subgraph, and graph matching [1]: 1. Adjacency: two vertices (vi and vj) connected by an edge are adjacent if ∃ eij ∈ E or ∃ eji ∈ E.
2. Incidence: an edge eij is incident to a vertex vi that is one of its endpoints.
vi ∈ V' and vj ∈ V' }. . Graph matching is a comparison between two graphs with certain criteria to match the nodes and edges of both graphs. There are several basic concepts related to graph matching, such as graph isomorphism and subgraph isomorphism a. Graph isomorphism definition : let G1 = (V1, E1) and G2 = (V2, E2) are two non-directed b. Subgraph isomorphism definition: a subgraph isomorphism from graph G1 to G2 is an isomorphism from graph G2 to subgraph G3 from graph G1 (G3⊆G1).
The graph formation can use 4-neighbor or 8-neighbor pixels. The Euclidean distance formula is used to find the RGB weight between two pixels [12] :

Segmentation and Region Adjacency Graph
Segmentation and RAG are included as ways to reduce the number of graph nodes. Segmentation will divide the image into several homogeneous regions which are interconnected. The partition graph is the basic principle of the graph-based segmentation method. This means that a search for the set of subgraphs (SG1, SG2, ..., SGn) of graph G will be performed [4]. The formation of a graph with a region adjacency graph representation is done by taking the centroid region as a node and the relation between two neighboring regions as an edge as seen in Figure 1. In this way, the number of vertices is reduced because the pixels in a region are represented by only one node.
The region that consists of pixels that have proximity or similarity is also called super-pixel. Superpixel is a meaningful pixel grouping and is used as a primitive for further computing or processing [6]. Image processing and analysis using super-pixel bases can increase the processing speed compared to using pixel-based due to the number of vertices that have dropped dramatically.

Minimum Spanning Tree
Minimum Spanning Tree (MST) is a spanning tree with the smallest weight value compared to all spanning trees on the graph. [13]. Kruskal and Boruvka are MST algorithms that are widely used in the graph-based segmentation method. The complexity of the MST algorithm is polynomial and the algorithm is greedy. Basically, the MST algorithm MST is easy to understand and fast in processing. It is a graph-based clustering where the graph is an undirected graph. The edge weight is the color distance of two related vertices. The grouping is done by removing the edges graph with certain criteria to form a subgraph. The important criteria are gestalt principles such as similarity and proximity [14].
The efficient graph-based image segmentation method proposed by Felzenszwalb and Huttenlocher is MST based on the Kruskal algorithm [14]. This method uses the global differences between the two neighboring subgraphs and the local differences within each subgraph. Meanwhile, the segmentation using the Boruvka algorithm and the Felzenszwalb criteria was done by Ali Saglam. The method developed is to detect objects based on color and cluster size. [12] as seen in Figure 2.  (2) is Felzenszwalb's criteria for grouping similar pixels in one region [12]: Internal Difference (Int (X)) is the difference between the edges with the highest weight on subtree X. Dif f (X, Y) is the difference in the smallest weight of the two neighboring subtrees X and Y. | X | is the size of the subtree X or the total number of vertices and k is a constant. Two subtrees will be combined if these criteria are met.
The following is an algorithm for merging regions in MST: 1. Sort the weights of all edges e ∈ E ascending from smallest to largest 2. For all sides (e ∈ E): a. Check that the pixels connected to the edge e are in one region. If it is connected then continue processing to the next edge. b. If it is not connected then check if there is a boundary between the two observed regions.
(Find Int(X), Int(Y) and Dif f(X, Y)) There is a limit if the minimum edge weight connected to the region is greater than the minimum of the maximum internal weight of the MST edges. c. If there is a limit, then go to the next edge. d. If not, then combine the region into a new region.

Graph Matching using VF2 Algorithm
Finding the exact correspondence between two graphs can use the exact matching method. Structurally, ensuring that the mapping between the vertices of the two graphs must maintain the incident edges between these vertices. This means that if two vertices in one graph are connected by an edge when it is mapped to two vertices on another graph, it is also connected by an edge. Ullmann's algorithm [15] and the VF2 algorithm [16]. Ullmann's algorithm is the basic algorithm of the tree search class of algorithms for subgraph isomorphism. Ullmann is the first depth-based search algorithm commonly used with complex heuristics to reduce the number of states visited. The main problem is the space complexity of O (n 3 ) which becomes impractical when dealing with large sparse graphs.
The Ullman algorithm is developed into the VF2 algorithm which can be used for graph isomorphism. VF2 is an algorithm commonly used in exact matching and is designed to solve pattern matching problems. In more than a decade, VF2 has been the best algorithm and is widely used because of its space efficiency and simplicity. Status-space is the representation used in VF2 and several conditions in each state will be checked to reduce the search tree.

Results and Discussion
It will show the process of graph extraction of the batik image using the Region Adjacency Graph representation model, and the testing of the results in image retrieval.

Data Preparation
The data prepared are synthetic images and real images. Synthetic data consists of 200 digital images with a size of 64 x 64 pixels. There are 4 different data classes, each totaling 50 images per class. This synthetic data will be tested in the graph extraction stages so that the details of the RAG formation can be known. Figure 3 is an example of synthetic image data that represents each image class.
Synthetic data was created using Adobe Photoshop CS6 64 bit. Some shapes with specific colors are drawn manually on page 64 x 64.  The algorithm of MST has been tested for real images and artificial images. The running time for this algorithm runs linearly with the number of edges. This method can measure regional boundaries by IOP Publishing doi:10.1088/1757-899X/1077/1/012006 6 comparing two quantities: based on differences in the intensity of the regional boundaries that are opposite, and based on the intensity between neighboring pixels in each region.

Experimental Results
The following is the result of a trial of the graph extraction process from an image. From the original image, the segmentation process is carried out and then extracted to get the region adjacency graph from the image. Graph feature extraction is obtained based on the label value given to each region resulting from segmentation. The label value provided is an integer value of 0 to (n_region-1). Each image will IOP Publishing doi:10.1088/1757-899X/1077/1/012006 7 have one Region Adjacency Graph. Through these stages, there are several features that are generated from each graph, including: 1. Node: nodes will be formed representing each region of the image segmentation results 2. Centroid: Each region will have one centroid which is the midpoint value 3. Mean Color: Each region will have an average pixel value in RGB 4. Area: Each region will have a total of RGB pixels which is the area of the region 5. Edge: each pair of vertices from neighboring regions will form an edge 6. Weight: Each edge of the graph has a weight which is the value of the Euclidean distance between two neighboring regions The graph of the RAG of each image will be stored on the storage media in a "*.gpickle" file format using the "networkx" library Furthermore, from the resulting RAG, image retrieval testing was carried out using the VF2 algorithm. An overview of the process of retrieval is as follows and can be seen in Figure 6: 1. The user first enters a query image 2. The query is extracted from its graph feature (RAG). Graph extraction has also been carried out on image data stored in the database. 3. Comparisons will be made between the query image graph and the graph database image to look for similarities using VF2 4. After the comparison process, several isomorphism images will be selected 5. Then the data retrieval that has been selected is conducted  The following is an example of an image retrieval that meets the isomorphism requirements created using artificial images (figure 7) and batik images ( figure 8). The number of retrievals can be determined by the user (1..n).
The next stage is performance evaluation using precision and recall values of image retrieval. Precision is the level of accuracy of the answers given by the system to the information requested by the user (Precision = TP / (TP + FP)). Meanwhile, recall is the success rate of the system in retrieving information (Recall = TP / (TP + FN)). The evaluation uses 200 synthetic images consisting of 4 classes and 180 real images consisting of 3 classes. Testing is carried out on all images. Table 3 are the evaluation results.
Based on table 2, the matching and retrieval evaluation performance, which includes precision and recall value on artificial image data, has good results. The values are 96.2% and 84.5%. However, a good precision result which is 87.8% is obtained for batik core motif data, while the recall result is 19.4%. This is due to the resulting RAG graph variation and noise so that the graph comparison does not provide the desired value. An isomorphic RAG graph due to rotation gives the correct result.

Conclusions and Suggestions
It has shown the steps of graph extraction from batik images. The extraction of graphs from other types of images can follow the model that has been made. A simple image capture prototype has been created to test RAG batik. The Minimum Spanning Tree method is a technique used in segmentation. Many segmentation techniques have been developed, both graph-based and non-graph-based. One technique can work well with certain sets of images but can fail to segment other sets of images. Selection of the right technique to obtain Region Adjacency Graph (RAG) is an important and decisive step for extracting the graph from the image An image Graph with RAG representation requires further exploration stage. There is a lot of information and features stored in this graph. The utilization of existing information is an important problem in the scope of graph-based classification and becomes the next challenge in image analysis with graphs. Various image analyses with graph methods have been developed for classification tasks including graph comparison techniques such as graph edit distance, kernel matching, or greedy matching. Image retrieval can also be done using this graph information by applying the graph matching technique Although the graph is considered as a powerful representation, there is one major drawback that is the lack of a suitable method that can be used for classification and retrieval. Computational complexity is also a big problem in graph matching techniques. The graph mining method can be said as a way to overcome this deficiency.