1. Introduction

Texture is a very important attribute in the field of computer vision and image processing. Numerous methods of texture analysis have been developed over the years. However, analyzing existing textures in the real world is a major challenge. A good real world texture classification method should have a highly discriminative and robust to variances such as rotation and scales. Among the many texture classification methods, local binary patterns (LBP) [1] and the gray-level co-occurrence matrix (GLCM) [2] are the most popular approaches. According to the research results of [3] [4], LBP is superior to GLCM in texture analysis performance. Therefore, there are many LBP-based methods have been proposed to enhance discrimination and improve robustness. Hafiane et al. [5] proposed median binary pattern (MBP) that uses the sign of the intensity difference compared to the local median within a neighborhood. Tan et al. [6] also proposed local ternary pattern (LTP), which extends original LBP to 3-valued codes. Liao et al. [7] employed the dominant local binary patterns (DLBP) to extract dominant patterns from textures. Jabid et al. [8] proposed local directional pattern (LDiP), which computes edge response values by using Kirsch masks in all eight directions at each pixel position and then generates a code, instead of the intensity. In [9], completed local binary pattern (CLBP) was proposed by Guo et al. to combine multiple LBP type features for texture classification by joint histogram. Zhang et al. [10] proposed local derivative pattern (LDeP), which extracts high-order local information by encoding various distinctive spatial relationships contained in a given local region. The local tetra pattern (LTrP) utilized first-order derivative calculations in the vertical and horizontal directions to encode the relationship between reference pixels and their neighbors by Murala et al. [11]. Dubey et al. [12] developed a local bit plane decoding pattern (LBDP), which is generated by finding a binary pattern using the difference between the intensity value of the center pixel and the local bit-plane transformed value of each image pixel. Alaei et al. [13] showed the fast local binary pattern (F-LBP) method, which is derived from the vertical & horizontal, and diagonal & off-diagonal pixels in a 3×3 patch size, separately. Banerjee et al. [14] proposed a texture descriptor called local neighborhood intensity pattern (LNIP), which considers the relative intensity difference between a particular pixel and a central pixel by considering its neighbors and generates a sign and a magnitude pattern. In the above mentioned, six indicator texture descriptors — namely, LBP, GLCM, LTP, LDiP, LDeP and LTrP — will be compared with the proposed method IFS-MCMLBP.

The rest of this paper is organized as follows: in Section 2 briefly introduces LBP, MCM and IFS. Section 3 presents the proposed IFS-MCMLBP descriptor including IFS texture generation, motif, LBP and feature fusion. Section 4 describes similarity measure, classifier and cross-validation. Section 5 gives the experimental analysis on various datasets along with the other comparative methods. Finally, Section 6 concludes the whole paper.

2. Related work

 2.1. Local Binary Patterns (LBP)

Ojala and colleagues [1] first proposed the local binary pattern (LBP) texture operator, which is just the first-order circular derivative of patterns and which works with the 3×3 neighborhood. In Fig. 1(a), each pixel is compared with its eight neighbors by taking the difference of the center pixel value; resulting strictly negative values are encoded with 0, and other values are encoded with 1. LBP is defined as

(1)

(2)

where g_i and g_c denote the gray values of the neighbor and central pixel, respectively; i is the index of the neighbor; P is the number of neighbors; and R is the radius of the circular neighborhood. Fig. 1(b) shows the neighbors for P = 8 when R = 1, P = 16 when R = 2, and P = 24 when R = 3. Fig. 1(c) shows an example of the LBP operator. For each given pixel, a binary number is obtained by concatenating all binary values in a clockwise direction starting with the binary value of the pixel’s top-left neighbor. The corresponding decimal value of the generated binary number is then used for labeling the given pixel.

 

              

(a)                                                           (b)                          

 

(c)

Fig. 1. (a) Neighbors for LBP pattern, (b) circular symmetric neighbor sets for different P=8 when R=1, P=16 when R=2 and P=24 when R=3 neighborhoods, (c) an example of the LBP operator.

 

2.2. Motif co-occurrence matrix (MCM)

Jhanwar and colleagues [15] have proposed the motif co-occurrence matrix (MCM) for content-based image retrieval. The MCM is derived from the motif-transformed image, which is calculated by dividing the whole image into non-overlapping 2×2 pixel patterns. Each grid is replaced by a scan motif, as shown in Fig. 2, which minimizes the local gradient while traversing the 2×2 grid that forms the motif-transformed image. These motifs are defined over a 2×2 grid, each depicting a distinct sequence of pixels starting from the top left corner.

 

Fig. 2. Illustration of scan motifs to traverse a 2×2 grid.

 

We can derive a transformed image from an original image, as shown in Fig. 3(a). An 8×8 image, as in Fig. 3(b), shows the corresponding 4×4 transformed image.

 

(a)                                              (b)

Fig. 3. (a) An 8×8 image (b) Motif transformed image from (a)

 

 2.3. Intuitionistic fuzzy set (IFS)

IFS theory, which is an extension of fuzzy set (FS), enhances images and helps to recover important structures that are not properly visible [16]. Atanassov [17] pioneered construction of IFS, which is defined by three feature functions as the degree of membership, degree of non-membership, and degree of hesitation or uncertainty.

Definition: An IFS A in universe X is an expression given by

(3)

where ,  are the membership and non-membership degree of an element x to the set A with the condition:

   for each

(4)

 

For each IFS in X, we call  the degree of hesitancy of x to A,  for each . The illustration of these degrees is shown in Fig. 4.

 

Fig. 4. Relationships between membership, non-membership, and hesitation degrees.

 

IFS is helpful in modeling vagueness or uncertainty, and important applications of IFS have been developed in many diverse areas, including medical diagnosis [18], pattern recognition [19], image processing [20], and decision making [21].

3. The proposed IFS-MCMLBP descriptor

Based on intuitionistic fuzzy set (IFS) theory, a novel descriptor IFS-MCMLBP is proposed by the fusion of Motif Co-occurrence Matrix (MCM) and Local Binary Pattern (LBP) as shown in Fig. 5. The IFS texture is used to model vagueness or uncertainty, and the MCM method for extracting microtexture information, whilst the LBP method plays the role of a global feature. And detailed information about it will be described in the following subsection.

 

Fig. 5. The framework of the proposed descriptor.

 

3.1. IFS texture generation

In the beginning, a RGB (red, green, and blue) image is converted into an HSV (hue, saturation, and value) image. Then, intuitionistic fuzzy image processing of color texture is applied to each channel of HSV to generate the image’s membership, non-membership and hesitancy components.

Suppose channel image A of size M×N pixels has L gray levels ranging between 0 and L−1. When applying IFS for image processing [22], an image can be considered as an array of fuzzy singletons. An intuitionistic fuzzy image is written as

(5)

where x is the pixel value at (i, j) point, i=0, 1,…, N-1,   j=0, 1,…, M-1.

With the condition

(6)

then

(7)

where

: membership degree

: non-membership degree

: hesitancy degree

 

Vlachos [23] represents the membership degree of image by  where x_max and x_min are the maximum and the minimum gray levels of the image. Sugeno’s intuitionistic fuzzy generator (IFG) [24] constructs non-membership degree  as

(8)

where parameter λ > 0, and hesitancy degree

(9)

By varying the λ > 0 parameter, different intuitionistic fuzzy set can be obtained. As λ is not a fixed value for all images, the optimum value of λ is obtained by maximizing fuzzy entropy [25]. Given an examples, the texture image C57 of CURet texture database is used for representing IFS texture images when λ is 3.6. Fig. 6(a) depicts C57 image along with their corresponding membership μ, non-membership ν, and hesitancy π images.

3.2. Motif feature

According to Section 3.1, a color texture image is processed by IFS texture generation, which will generate three μ, three ν and three π component images, respectively, for a total of nine. Then the nine IFS component images are individually converted into MCM. Next, we compute the histogram of each MCM. The histogram has 6 bins, representing the distribution of motif 1 to motif 6. Since motif 0 only represents a homogeneous texture, so motif 0 is not considered. Furthermore, the histogram of nine MCMs is concatenated in series to form a 54-dimensional feature vector of motif as shown Fig. 6(b). In brief, the motifs’ histograms can be easily implemented and computed from that image, and the histogram’s shape provides many clues to the image’s microtexture features.

3.3. LBP feature

As shown in Fig. 6(c), a 256-bin histogram of LBP is extracted out of the IFS texture images including three μ, three ν, and three π images one by one. Then, all the histograms are concatenated into a one 2304-dimensional feature vector for representing the global information of the texture image. This also makes this global feature more robustness for changes in rotation and scale.

3.4. Feature fusion

Since different features may be represented by different importance, it is possible to give an appropriate weight by experimental approaches when combining them. However, some studies have also pointed out that traditional feature fusion methods that simply concatenate several features may be better or more robust than using a single feature.

The aforementioned motif feature and LBP feature represent local texture information and global texture information, respectively. This paper uses traditional feature fusion methods to simply concatenate together as shown Fig. 6(d).

 

Fig. 6. (a) Texture image C57 from the CURet texture database and showing IFS texture images including membership μ, non-membership ν, and hesitancy images. (b) Concatenate three μ, three ν, and three π motif histograms to form a 54-dimensional feature vector of motif. (c) Like (b), forming a 2304-dimensional feature vector of LBP. (d) Fusion local feature motif and global feature LBP.

 

4. Classification

4.1. Similarity measure

Many measures have been proposed for discriminating the dissimilarity between two histograms, such as Manhattan, Euclidean, d₁, Canberra, and χ² (Chi-square) distance. In this paper, the χ² distance function is chosen in the experiments due to its excellent performance in terms of good recognition rates which is calculated as

(10)

where Q={q_i} and T={t_i} (i=1…n) are two histograms with n bins, and the histogram is used as feature representation; q_i is ith bin (i.e. ith feature) of testing image Q,  t_i is ith bin of training image T in database.

4.2. Classifier and cross-validation

The k-nearest neighbor (k-NN) classification [26] is one of the simplest but widely using pattern recognition algorithm. An object is classified by the distance from its neighbors, with the object being assigned to the class most common amongst its k nearest neighbors. This paper use 1-NN approach as a classifier in this work, i.e. the texture image is classified to the class of its nearest neighbor.

The sample data was split into training and test sets using the leave-one-out cross-validation (LOOCV) [27], for N samples, a total of N trials are conducted. In each trial a sample is taken out from the data set and kept for testing and the others are used for training. This procedure was repeated for all samples and the accuracy rate obtained as the percentage of classified samples out of the total number of samples. This methodology is superior to random partitioning of data to generate training and test sets as the resultant performance of the system may not reflect its true ability for texture classification.

5. Experiments and results

In this study, the proposed method compares performance with other well-known or state-of-the-art approaches, namely LBP_8,1, LBP_16,2, GLCM, LTP, LDiP, LDeP and LTrP. Experiments performed on four benchmark of texture images, the Colored Brodatz Texture (CBT) database, the Columbia-Utrecht Reflection and Texture (CUReT) database, the Outex database, the Vision Texture (VisTex) database in MIT university. All the experiments are run in Matlab environment.

5.1. Experiments on texture database

The CBT database consists of 112 periodic or non-periodic 640×640 texture images, parts of which are shown in Fig. 7(a). Each texture image is divided into 25 non-overlapping sub-images for experiments, resulting in 112 categories, each category contains 25 images, and 11,200 total samples for size of 128×128 pixels.

The CUReT texture image database contains 61 texture of real-world surfaces as shown in Fig. 7(b), 205 images per class, acquired at different viewpoints, illuminations, and orientations. There are 103 images shot from a viewing angle of 22.5 and 45 degrees, in total 6,283 (=61×103) images are selected. The size of each texture is 640×480 which is divided into nine 128×128 non-overlapping sub-images, and then a database of 56,547 samples is obtained.

This paper use a commonly test suite Outex_TC_00013 (TC13) of Outex database as shown in Fig. 7(c), containing 68 classes with 20 texture images per class, in total 1,360 (=68×20) images. Each texture class is collected under ”inca” illuminations with resolution of images at 128×128.

The VisTex database includes 40 classes texture images, parts of which are shown in Fig. 7(d). The size of each texture is 512×512 which is divided into sixteen 128×128 non-overlapping sub-images, and then a database of 640 samples is obtained.

 

Fig. 7. Sample textures from the (a) CBT, (b) CURet, (c) Outex, and (d) VisTex database.

 

5.2. Comparative analysis of the accuracy

This paper employed the nearest neighbor (1-NN) classification and χ2 (chi-square) distance, while the accuracy estimate uses leave-one-out cross-validation (LOOCV). Experiments were performed on aforementioned four texture datasets and compared to seven prior art LBP_8,1, LBP_16,2, GLCM, LTP, LDiP, LDeP and LTrP. Fig. 8 illustrates the accuracy rates of all these descriptors and following are noted.

(1)   Clearly, it is evident that the proposed IFS-MCMLBP descriptor provide better classification performance (average accuracy of 96.08%) compared to other state-of-the-art descriptors. However, for the VisTex database, the proposed descriptor shows a classification accuracy of 98.44%, which is the same as the LDeP descriptor but less than the LTrP descriptor.

(2)   The accuracy rate of all other descriptors are less than 85% on the Outex(TC13) texture. However, the IFS-MCMLBP has achieved a classification rate around 87%.

(3)   GLCM performance is very low on four texture databases when compared to other descriptors.

 

Fig. 8. Classification accuracy (%) on CBT, CUReT, Outex(TC13), and VisTex databases using different descriptors.

 

5.3. Comparative analysis of robust rotation

For the analysis of rotation robustness, all 112 texture images are rotated by 19 different angles (i.e., 0°, 10°, 20°, … , 180°) from the CBT database. Here we use the nearest neighbor interpolation method for rotation of image. Each rotated image is divided into four non-overlapping 320×320 sub-images. Four 128×128 sub-images are then cropped from the center of the 320×320 sub-images. Hence, we obtain a rotation test set with a total of 8,512 (=112×19×4) sub-images. Fig. 9 shows an example of different rotation angles from 0 to 180 degrees using 10-degree steps.

 

Fig. 9. Rotation angles from 0° to 180° using 10-degree steps for image D01 from the CBT database.

 

The results of classification accuracy with respect to rotation factors for seven tested descriptors (the GLCM classification accuracy is too low, not considered) are shown in Fig. 10 and Table 1 for clear comparison on CBT database. The proposed descriptor IFS-MCMLBP has the highly stable accuracy (average 95.68%) than the other descriptors under at each rotation angle, that confirms the robustness of the proposed descriptor. Thus, the other descriptors only obtain high accuracy at the rotation angles 0⁰, 90⁰ and 180⁰, where the image distortion is minimal.

 

Fig. 10. Texture classification results for seven tested descriptors on the rotated CBT database.

 

Table 1. Average classification rates (%) of different descriptor at each rotation angle on CBT database

 

5.4. Comparative analysis of robust scale

For the analysis of scale robustness, all 112 texture images are scaled with scaling factors of 0.5 to 1.5 with 0.1 intervals (11 scales for each image) from the CBT database. The bicubic interpolation and antialiasing methods are used to scale image. Each scaled image is divided into four non-overlapping 320×320 sub-images. Four 128×128 sub-images are then cropped from the center of the 320×320 sub-images. Hence, a scaled test set was obtained with 4,928 (=112×11×4) sub-images. Fig. 11 shows an example in different scale factors: 0.5, 0.6, 0.8, 1.0, 1.2, 1.3, 1.4 and 1.5.

 

Fig. 11. Example scaled test set for image D66 from the CBT database; scale factors are 0.5, 0.6, 0.8, 1.0, 1.2, 1.3, 1.4 and 1.5.

 

The classification accuracy results, with respect to scale factors for seven tested descriptors, are shown in Fig. 12 and Table 2. It is obvious that the proposed descriptor IFS-MCMLBP has the most stable accuracy (average 88.73%) of the seven tested descriptors under multi-scale factors. Only in the texture image with a scale factor of 1.5, the accuracy of the IFS-MCMLBP descriptor is 64.06%, which is slightly lower than 67.63% of the LTP descriptor. Moreover, the enlarged texture image could miss texture structures, and the shrinked texture image still remains original structures. Therefore, the accuracy of the former is worse than the latter.

 

Fig. 12. Texture classification results for seven tested descriptors on the scaled CBT database.

 

Table 2. Average classification rates (%) of different descriptor under multi-scale factors on CBT database

 

6. Conclusions

This paper has presented a new method for texture classification by fusing the MCM and LBP features based on intuitionistic fuzzy set. The LBP describes local textures, while MCM emphasizes global microstructures and uses IFS to recover invisible important structures. As shown in the experimental results, the proposed IFS-MCMLBP method has highly stable accuracy and robustness to rotation and scale, and outperforms the existing methods. For the future possibility, the color information also can be incorporated by combining color histogram features on hue and saturation channels into the proposed global or local feature representation scheme for better performance. In addition, the proposed method can also be extended to other research fields such as content-based image retrieval (CBIR), object recognition, remote sensing and medical imaging.

References

1.             T. Ojala, M. Pietikäinen and T. Mäenpää, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971–987, 2002.

2.             R. M. Harlick, K. Shanmugam and I. Dinstein, “Textural features for image classification”, IEEE Trans. on Systems, Man and Cybernetics, vol. SMC-3, no. 6, pp. 610-621, 1973.

3.             M. B. Neiva, A. Vacavant and O. M. Bruno, “Improving texture extraction and classification using smoothed morphological operators”, Digital Signal Processing, vol. 83, pp. 24-34, 2018.

4.             F. Alaei, A. Alaei, U. Pal and M. Blumenstein, “A comparative study of different texture features for document image retrieval”, Expert Systems with Applications, vol. 121, pp. 97-114,  2019.

5.             A. Hafiane, G. Seetharaman and B. Zavidovique, “Median binary pattern for textures classification”, Image Analysis and Recognition, pp. 387–398, 2007.

6.             X. Tan and B. Triggs, “Enhanced local texture feature sets for face recognition under difficult lighting conditions”, Analysis and Modelling of Faces and Gestures, in: Lecture Notes Computer Science, vol. 4778, pp. 168-182, 2007.

7.             S. Liao,  W. K. Law and  C. S. Chung, “Dominant local binary patterns for texture classication”, IEEE Trans. on Image Processing, vol. 18, pp. 1107-1118, 2009.

8.             T. Jabid, M. H. Kabir and O. Chae, “Local Directional Pattern (LDP) for face recognition”, in Proc. of the IEEE International Conference of Consumer Electronics, pp. 329-330, Jan. 2010.

9.             Z. Guo, L. Zhang and D. Zhang, “A completed modeling of local binary pattern operator for texture classification”, IEEE Trans. on Image Processing, 19(6), pp. 1657–1663, 2010.

10.         B. Zhang, Y. Gao, S. Zhao and J. Liu, “Local derivative pattern versus local binary pattern: Face recognition with high-order local pattern descriptor”, IEEE Trans. on Image Processing, vol. 19, no. 2, pp. 533-544, Feb. 2010.

11.         S. Murala, R. P. Maheshwari and R. Balasubramanian, “Local tetra patterns: A new feature descriptor for content-based image retrieval”, IEEE Trans. on Image Processing, vol. 21, no. 5, pp. 2874-2886, May 2012.

12.         S. R. Dubey, S. K. Singh and R. K. Singh, “Local Bit-Plane Decoded Pattern: A Novel Feature Descriptor for Biomedical Image Retrieval”, IEEE Journal of Biomedical and Health Informatics, vol. issue 4, pp. 1139–1147, Jul. 2016.

13.         F. Alaei, A. Alaei, U. Pal and M. Blumenstein, “Fast local binary pattern: Application to document image retrieval”, International Conference on the Image and Vision Computing New Zealand (IVCNZ), pp. 1–6, 2017.

14.         P. Banerjee, A. K. Bhunia and A. Bhattacharyya, “Local Neighborhood Intensity Pattern – A new texture feature descriptor for image retrieval”, Expert Systems with Applications, Jul. 2018, https://doi.org/10.1016/j.eswa.2018.06.044

15.         N. Jhanwar, S. Chaudhuri, G. Seetharaman and B. Zavidovique, “Content based image retrieval using motif co-occurrence matrix”, Image and Vision Computing, vol. 22, no. 14, pp. 1211-1220, Dec. 2004.

16.         T. Chaira and A. K. Ray, “A new measure using intuitionistic fuzzy set theory and its application to edge detection”, Applied Soft Computing, vol. 8, no. 2, pp. 919-927, 2008.

17.         K. T. Atanassov, “Intuitionistic fuzzy sets”, Fuzzy Sets and Systems, vol. 20, no. 1, pp. 87-96, 1986.

18.         S. K. De, R. Biswas and A. R. Roy, “An application of intuitionistic fuzzy sets in medical diagnosis”, Fuzzy Sets and Systems, vol. 117, No. 2, pp. 209–213, 2001.

19.         H. Nguyen, “A novel similarity/dissimilarity measure for intuitionistic fuzzy sets and its application in pattern recognition”, Expert Systems with Applications, vol. 45, pp. 97-107, Mar. 2016.

20.         V. P. Ananthi and P. Balasubramaniam, “A new image denoising method using interval-valued intuitionistic fuzzy sets for the removal of impulse noise”, Signal Processing, vol. 121, pp. 81-93, Apr. 2016.

21.         H. Nguyen, “A new knowledge-based measure for intuitionistic fuzzy sets and its application in multiple attribute group decision making”, Expert Systems with Applications, vol. 42, issue 22, pp. 8766-8774, Dec. 2015.

22.         I. K. Vlachos, G. D. Sergiadis, “Intuitionistic fuzzy information: applications to pattern recognition”, Pattern Recognition Letters, vol. 28, no. 2, pp. 197-206, Jan. 2007.

23.         I. K. Vlachos, G. D. Sergiadis, “Intuitionistic Fuzzy Image Processing”, Encyclopedia of Artificial Intelligence, pp. 967-974, 2009.

24.         M. Sugeno, “Fuzzy measures and fuzzy integrals: A survey”, In: Gupta, Saridis, and Gaines (eds), Fuzzy Automata and Decision Processes, pp. 89-102, 1977.

25.         A. D. Luca, S. Termini, “A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory”, Information and Control, vol. 20, no. 4, pp. 301-312, May 1972.

26.         T. M. Cover and P. E. Hart, “Nearest neighbor pattern classification”, IEEE Trans. on Information Theory, vol. 13, pp. 21-27, 1967.

27.         P. A. Devijver, J. Kittler, Pattern Recognition: a statistical approach, Prentice-Hall, London, 1982.