Periodic Pattern Detection of Printed Fabric Based on Deep Learning Algorithm

Printed fabric patterns contain multiple repeat pattern primitives, which have a significant impact on fabric pattern design in the textile industry. The pattern primitive is often composed of multiple elements, such as color, form, and texture structure. Therefore, the more pattern elements it contains, the more complex the primitive is. In order to segment fabric primitives, this paper proposes a novel convolutional neural network (CNN) method with spatial pyramid pooling module as a feature extractor, which enables to learn the pattern feature information and determine whether the printed fabric has periodic pattern primitives. Furthermore, by choosing pair of activation peaks in a filter, a set of displacement vectors can be calculated. The activation peaks that are most accordant with the optimum displacement vector contribute to pick out the final size of primitives. The results show that the method with the powerful feature extraction capabilities of the CNN can segment the periodic pattern primitives of complex printed fabrics. Compared with the traditional algorithm, the proposed method has higher segmentation accuracy and adaptability.


Introduction
Printed fabric is one of the most important textiles in the field of textile industry, it is generally constructed from periodic primitive pattern (PPP), which is commonly consist of the same elements. At present, the PPPs are generally extracted by experienced workers, which is extremely costly and timeconsuming. In addition, human eyes are unable to recognize the unit pixels accurately since the unit pixels that make up a digital image are tiny. To meet the needs of digital manufacturing in textile industry, it's essential to find a way to accurately extract the PPP from the printed fabric automatically, so as to accelerate the pattern design process or to improve the accuracy of defects detection [1].
There have been a large amount of work in the field of periodic pattern analysis. Generally, they can be mainly divided into as following 3 categories: (1) spectral-based methods, (2) statistical-based methods, (3) traditional learning-based methods. For example, early in 1980, Terzopoulus et al. [2] tried to analyze periodic textures structure by using the gray-level co-occurrence matrix (GLCM). However, it was computationally intensive and had poor robustness [3]. The approach using autocorrelation function to calculate peaks was proposed by Lin et al. [4]. This method had a certain tolerance to local distortion, but it required to smooth the autocorrelation function, which led to long calculation time [4]. Oh et al. [5] used distance matching function to analyze periodic texture's structure, which has a small amount of calculation consumption. However, the DMF method is very sensitive to noise such as geometric distortion. A method of using sum and difference histogram (SDH) for texture analysis was  [6]. The advantage of this method was that it greatly reduced the calculation time and had a stable effect. Matsuyama et al. [7] used Fourier transform to analyze the textures. The period of repeat pattern primitives could be determined by the pulse distribution of Fourier spectrum. However, the peak of Fourier spectrum was not prominent when the pattern contained fewer periodic primitives, which made the method based on Fourier transform can only segment the periodic pattern primitives of the woven fabric. Kuo et al. [8] converted the printed fabric into a full-color image, and employed the fuzzy mean clustering algorithm to segment the pattern to obtain the pattern of the printed fabric. Finally, the proposed approach using the Hough transform completed the segmentation of periodic pattern primitives. However, this method was only suitable for patterns with relatively clear color boundaries, but got trouble in effectively identifying patterns with rich colors and complex textures.
The above methods often require a manual design of feature extractors, which have poor adaptability and are only effective for some simple periodic pattern primitives. Therefore, it is hard to adapt to complex printed fabrics, which is unable to meet the needs of textile industry. In recent years, the application of deep learning algorithms has become increasingly widespread, which has promoted the rapid development of computer vision, especially in image classification [9], object detection [10], and semantic segmentation [11]. However, few researchers apply it to the pattern primitive's extraction of printed fabrics. Sun Y. et al. [12] found that the use of convolutional neural network can replace traditional hand-designed feature extractors through visualization, alignment and key point prediction. Lettry et al. [13] proposed a method of using the position information in the feature layer of the CNN to find the spatial periodic rules, which has a excellent impact on the extraction of periodic pattern primitives in natural scenes. However, the primitives of periodic patterns in natural scenes are relatively simple [14], which are usually composed of regular geometric objects with clear boundaries. The methods in natural scenes cannot be used directly due to the abundant texture and shape features. Li J. et al. [15] found that CNNs can implicitly learn spatial location information. The CNN learns the characteristic information of the primitives, and uses this characteristic information to segment the periodic pattern primitives of the printed fabric. In order to segment fabric primitives, this paper proposes a novel CNN method as a feature extractor, which enables to learn the pattern primitive feature information and determine whether the printed fabric has periodic pattern primitives. Furthermore, the network is used to calculate the activation peak and segment the periodic pattern primitives of the printed fabric.
The main contributions of this work are summarized as follows: in the first part, we proposed a classification network for printed fabrics and used the transfer learning algorithm for training, so that the convolutional neural network can learn more abundant features. In the second part, we used the convolution layer of the trained classification network as the feature extractor. Through calculating the activation peak, the network effectively segmented the pattern primitives. The results show that our algorithm has better segmentation effect than the existing methods.

Pre-processing
The input belongs to the printed fabric patterns, which is captured by industrial cameras. However, an obvious problem here is that the printed fabrics are placed manually during the collection process, and the surface of the printed fabrics is prone to noise such as dust and wrinkles. We therefore utilize a preprocessing step to align the patterns.
According to the previous research [16] that the range of fabric inclination angle α does not exceed 20°, we used the Radon Transform with a suitable inclination angle. In order to improve the segmentation effect of periodic primitives, we made use of the Non-Local Variations algorithm from Dekel et al. [17], which was based on the structural differences of the periodic regions in the whole image. The input image I reduced the small changes between the periodic structures after T transformation processing, which can be expressed as: where the J is the corrected image, N is the noise signal. This paper set the parameter iteration=20 according to human experience.

Feature Extraction
The goal of this step is to utilize the activations of filters in the convolutional layers, rather than traditionally hand-crafted. In this section, we proposed a CNN model based on spatial pyramid pooling. The printed fabrics were divided into two categories according to whether the patterns have periodic patterns. The two types of fabric patterns in the fabric data set are shown in figure 1. This paper puts to use a spatial pyramid pooling module, which allowed the AlexNet to learn multi-scale features and extract abundant features information.

Network Structure.
Networks with more zero padding can implicitly learn the spatial location information, which makes use of the location information of patches [12]. We did not choose novel deep convolutional neural networks such as VGG-16 or ResNet34, which stacked more layers and deepened the model depth. The network to deepen is prone to overfitting because of the simple samples learning. In addition, the above networks will increase the amount of computation of periodic primitives' segmentation. Therefore, we used AlexNet as the base feature extract network. In order to make the network better adapt to segmentation tasks and avoid overfitting, fc1 was removed. Additionally, the last pooling layer was replaced as the Spatial Pyramid Pooling (SPP) [18] layers, which helped the network learn more spatial features. The improved network structure is shown in figure 2. There are a total of eight layers, specifically composed of five convolutional layers, a spatial The module structure used in this paper is shown in figure 3. The input image passed through the last convolutional layer to obtain the feature map, where the size of the feature map was q×q. Then the feature map was used as the input of the spatial pyramid pooling layer, which would be divided into different patches with the size k×k.

Periodic Pattern Primitive Extraction
The shallow layers of a convolutional neural network usually learn low-level features, such as edges. As the network deepens, it learns higher-level features representation, such as objects and complex forms. Therefore, CNN can be considered as a set of filters with different levels of abstraction, and these filters will be activated when specific features appear in the image area. In this section, the convolutional layer of the previously trained classification network was used as a feature extractor to segment the periodic pattern primitives of the fabric. Its advantage was that it does not require any prior knowledge of human beings and can learn independently, which helped to adaptively generate a hierarchy of features of the input image.

Acquisition of Displacement
Vector. The input image would generate regular local peaks in each feature layer of the network, and each set of peaks corresponded to a set of displacement vectors. Therefore, the displacement vector that appeared most frequently in different feature layers was most likely to be the size of the periodic pattern primitive. The Hough voting algorithm was used in this paper to vote for each pair of displacement vectors and determine the appropriate size of primitives.
When the pre-processed image was input to the trained network, regular activation peaks would be generated after the convolution operation, which were obtained by the non-maximum suppression algorithm. In this step, the 5 convolutional layers of the improved AlexNet in section 2.  The activation peaks that are most accordant with the optimum displacement vector [19] contributes to pick out the final size of primitives. Therefore, we fused vectors linking pairs of activation peaks by a Hough-like voting space H: 2  RR . The filters in each layer have different sizes. Therefore, it is incapable of directly voting for the displacement of different filters. Therefore, it is assumed that each vote obeys a two-dimensional normal distribution centered on , mn s , and l  is the covariance matrix of  (4) where (x, y) corresponds to the pixel coordinates of the input image. In this step, the optimal spatial displacement vector * s was obtained, which contributed to the size of the periodic pattern primitive. The formula is as follows:  ( 6) where the parameter  is a flat prior and  means the radius of the neighborhood considered at lth layer as the close surrounding of the selected displacement vector.
The filters with weights larger than * × l f w  would be kept, where * l f w was the highest weight among the filters. And the parameter  was set to 0.65 according to experience [12].
In order to obtain the relative position of the displacement vector, the two-dimensional rectangular coordinate (x, y) of the filter needs to be simplified first. The formula is as follows:  ( 8) Finally, the IPM algorithm [20] was used to obtain the centroid coordinates of the periodic pattern primitives. Therefore, the final periodic pattern primitive was segmented by the centroid coordinates and the primitive size which was determined previously.

Experiments
In this section, the experimental details are discussed, and the classification results are analyzed later. Then three different types of images are tested for segmentation. In order to better analyze the performance of the algorithm in this paper, a requisite analysis is carried out with the other algorithm. All experiments were performed on a workstation configured with Intel Xeon Gold 5115 CPU(2.50GHz),

Image Classification Results
A total of 600 images of printed fabrics have been collected in this paper. There are 400 images in the training set, including 200 fabrics patterns with periodic primitives and 200 patterns without periodic primitives. The experiment adopted the transfer learning method, and used the pre-trained AlexNet network in the Pytorch deep learning framework to initialize the parameters. For the reason of small samples, operations such as translation and zoom were used to augment the image data. Finally, the number of training images was 4000, and the number of testing images was 50 of each kind. Adam was the optimized algorithm in this experiment, and the overall learning rate for the training was 0.001.
The results are shown in table 2. After transfer learning, the classification accuracy has been significantly improved. The vital basis for segment the periodic pattern primitives of printed fabric is whether the primitives in the image recursively appear after a certain distance. Therefore, Spatial Pyramid Pooling (SPP) module contributed the network to learn multi-scale features and improve the accuracy of the network classification of whether it contained periodic primitives. Then the convolutional layers would be feature extractors for subsequent primitive segmentation.

Segmentation of Three Kinds of Periodic Primitives
If the primitive contains plenty of elements and complex features, segmentation will be very arduous.
According to the number of elements contained in the periodic pattern primitives, the printed fabrics were divided into three categories. As more elements the periodic primitive contained, the more difficult it was to segment. The HCDH method [17] is one of the researches on the periodic primitives of fabric. This algorithm converts the color image into a gray image, then it uses the sum-difference histogram (SDH) algorithm to calculate the homogenous features, which contributes to determine the size of the periodic primitive according to the distance between two adjacent local maximum homogenous values. However, it is only limited to patterns with simple periodic primitives, and does not take into account the plentiful texture characteristics of the fabric and the colors. The input image size was 400 × 400. Figure 5 shows the result of simple images. The method in this paper and the comparison algorithm could both accurately segment the complete periodic primitives.
However, the HCDH algorithm is invalid for the complex periodic primitives, which contains plentiful elements and features. Obviously, the homogeneity function reaction of the HCDH method is almost a straight line on the curve, and the primitives of the printed fabric pattern cannot be obtained according to the interval between the peaks. In deep learning, the shallow layer can usually obtain lowlevel features such as edges, while the deep layer can learn more complex feature information, such as complex shapes. The result of our method is shown in figure 6.
It is of great significance to be able to identify periodic primitives with color features. The elements in these primitives have the same shape but different colors. However, the input image of the HCDH algorithm is a gray image, which is impossible to learn the color features. The image we input to the network contains three channels, and the method in this paper can extract the color features of the image. The result of our method is shown in figure 7.

Quantitative Evaluation
The effects of the two algorithms are quantitatively analyzed in this section. Cosine similarity is an important index to evaluate whether images are similar. Therefore, if the segmentation effect is precise, the similarity of adjacent periodic pattern primitives will be very high. We judge that whether the segmentation effect is good according to the cosine similarity value of adjacent periodic pattern primitives. Based on experience, the segmentation effect is regarded as accurate if the cosine similarity value greater than 0.95.
The segmentation accuracy of our algorithm and the HCDH algorithm in the data set is shown in table 3. There is no comparison here on fabrics with color features because the HCDH algorithm cannot identify the primitives. Additionally, the simple samples are further divided into 2 categories. Simple periodic primitives A means the single primitive has just one element while primitives B means the single primitive has 2 or 3 elements. On the first type of fabrics, the segmentation accuracy of our algorithm is 10% higher than that of the HCDH algorithm, and it is 14.3% higher on the second type of fabrics. As the fabric becomes more complex, the more difficulty it is to identify. On the third type of fabric, the segmentation accuracy of the HCDH algorithm is only 10.5%, which is almost impossible to identify, while the segmentation accuracy of our algorithm is 89.5%. The proposed method in this paper has high accuracy and adaptability, which benefits from the powerful feature extraction ability of CNNs.

Conclusion
In this paper, a CNN model for classification of periodic printed fabrics is proposed, and the SPP module is added to AlexNet to improve the feature extraction ability of fabrics consists of PPPs. The results show that the classification algorithm has high classification accuracy. Then, from each pair of activation peaks in each filter, a set of displacement vectors can be calculated. The activation peaks that are most accordant with the optimum displacement vector contribute to pick out the final size of primitives. Finally, the Hough voting and the IPM algorithm are used for complete primitive segmentation. Compared with the traditional algorithm, the results show that the algorithm in this paper has higher accuracy and adaptability. Although the performance of it has reached a high level, it takes a longer time compared to the traditional method and has obvious limitations in practical textile industry. Therefore, in the future, we will further optimize the algorithm to reduce computational consumption, which will be suitable for practical applications.