1 Introduction

Magnetic resonance imaging (MR) and computer tomography (CT) imaging of the brain are the two most commonly used tests for identifying brain abnormalities. These two methods are widely used because of their availability and the high quality of the images they produce [1]. In order to obtain better diagnostic results, MR images are processed and enhanced using digital image processing techniques. However, image enhancement is not the only process that uses image processing techniques. Rather, these are used in diagnosing, identifying, and segmenting the abnormalities. Such systems are known as computer-aided diagnosis (CAD) systems which are widely used to improve diagnostic accuracy.

The great difference in shape among the different brains of different people represents a major challenge in the computer-based diagnosis process. Therefore, the process of diagnosing a person’s brain tumour based on comparing the image with other brain images may not be reliable enough. As a result of this natural diversity, we found that the methods that use artificial intelligence algorithms, such as deep learning, to classify brain images using a specific dataset, may not be able to identify defects with sufficient accuracy when using images that may differ from the images that these methods were trained on even though the results of these methods exhibit high accuracy when applied to the dataset used.

Segmentation is another method that is used to identify the abnormality of the brain in an image. Many segmentation algorithms have been suggested and used. However, segmentation of the image of brain tumours is a very difficult task because there is a large class of tumour types that have a variety of shapes, sizes, and locations. Furthermore, the fact that different images may have different brightness levels and imaging conditions may make segmentation even more difficult.

The process of identifying tumours goes through three stages, the first stage is the process of determining whether there is any kind of abnormality in the brain, the second stage is to locate and isolate the abnormal region, and finally, the third stage is to extract the measurements of this tumour. The process of identifying and extracting the abnormal region is one of the most important challenges and therefore this research focuses mainly on this issue.

The majority of studies agree to divide the MR image processing into five stages, which are respectively, pre-processing, feature extraction, feature selection, classification, and segmentation [2]. In the pre-processing stage, the image is filtered and enhanced to improve the efficiency of the following processes using techniques and filters such as contrast enhancement and noise removal. Feature extraction and selection is crucial as choosing an inappropriate feature may reduce the accuracy of the results. Finally, classification and clustering are used to isolate and extract the tumour from the rest of the brain.

Brain tumour identification has been the focus of many studies in the past few decades. Due to the availability of resources and the large amount of published work, many reviews have been published listing the methods and techniques used. The reviews focused primarily on two types of methods: non-AI-based approaches that do not use AI algorithms such as thresholding, image segmentation, and clustering [1], [3], [4], [5], [6], [7], [8], and [9] and artificial intelligence-based approaches that use AI algorithms such as artificial neural networks, fuzzy logic, support vector machine, and deep learning [10], [11], [12], and [13].

1.1 Hypothesis and contribution

The main hypothesis of the research depends on the fact that the regions containing the tumour in the image are different from the rest of the image, which is very close to the definition of saliency. Some definitions of saliency extraction depend on the assumption that salient regions are rare in the image and with different nature of the surrounding regions. In this research, we will use a method of extracting the saliency based on the irregularity of the regions as this method depends on the basis that the salient regions in the image are rare and different from the rest of the image, which is what we need to determine the tumour area.

To demonstrate this hypothesis experimentally, several experiments have been developed and applied to a standard dataset. The results obtained were studied and discussed, and ways to improve them using other digital image processing techniques have been studied as well.

1.2 Datasets and technology

The proposed approach has been implemented using Python programming with JupyterLab version 2.2.6 on a PC with an intel core I7  processor and 16 GB RAM. The Algorithm applied to a dataset containing 250 various brain images established by selecting a subset from “Brain MRI Images for Brain Tumour Detection” [14] and “Brain Tumour Classification (MRI)” [15] datasets. Only axial plane images have been selected and all the three types: T1W, T2W and Flair have been considered.

The remaining of the paper is organised as follows: Section 2 presents the necessary background and some of the state-of-the-art existing work. In Section 3, the proposed algorithm is presented and the necessary mathematical derivation and discussion of the various measures that can be used to achieve the aim of the research. The results are presented and discussed in detail in Section 4 of this paper in addition to a comprehensive discussion of evaluation methods. Finally, we conclude the paper in section 5.

2 Background and Existing Work

This section provides a basic theoretical background related to the topic in general in addition to a discussion of the existing work.

2.1 Magnetic Resonance Imaging (MRI) of the Brain

MRI is widely used in neurology and neurosurgery since it provides very good details of the brain and has the ability to visualize it in three planes: axial, sagittal and coronal as shown in the example given in Fig. 1 [16].

 

Fig. 1. Brain MR images show the three planes: (a) axial, (b) sagittal and (c) coronal [16].

 

Three sequences are commonly used in MRI, namely T1-weighted (T1W), T2-weighted (T2W) and Flair.  T1W images are produced using short Time to Echo (TE) and short Repetition Time (TR), while T2W images are generated using longer TE and TR times. The Flair sequence is similar to a T2W but with very long TE and TR times. The three forms of sequences are shown in Fig. 2 [16].

 

Fig. 2. Three types of MRI sequences: (a) T1W, (b) T2W and (c) Flair [16].

2.2 MRI Analysis Approaches

Because of the structural complexity of brain tissue, brain tumour segmentation is a challenging and difficult task [17]. It can be divided mainly into three types based on human intervention [18], [19]:

1. Manual segmentation method: This type of segmentation is carried out by a radiologist and depends heavily on his/her knowledge and skill [3], [1].

2. Semi-automatic segmentation methods: In such kind of segmentation, the user interacts with the automatic segmentation system; the user needs to enter some parameters and provides a feedback response to the system output. The semi-automatic brain tumour segmentation methods go mainly through three main processes: initialisation, feedback response, and evaluation [19].

3. Fully automated segmentation methods: In these methods, the machine performs all operations without any user intervention. This type is known also as Computer-Aided Diagnosis (CAD) or Automatic identification.

Fig. 3 shows the main approaches used in MRI analysis.

2.3 Automatic Identification

Since the interest of this research is only in automatic analysis of MRI, we will present a discussion of its key concepts.

2.3.1 Pre-processing

To make the images more suitable for further processing and for the automatic identification of tumour, some steps are needed to be taken. These steps are usually known as pre-processing steps and include processes such as noise removal, registration, skull stripping, intensity normalisation, and bias field correction [20].

Noise removal is one of the first stages of pre-processing where the presence of noise affects image quality, which in turn leads to getting inaccurate results from other processing algorithms. The noise in the image may be caused by various reasons such as transmission system, equipment, and lighting conditions. Several types of noise have been identified in the images such as Gaussian, Poisson, Blurred, Speckle and salt-and-pepper noise. Noise removal algorithms, such as Weiner filter, Gaussian filter, and median filter are very common in image processing applications [21].

 

Fig. 3. MRI analysis approaches.

 

The second important stage in pre-processing is image registration. Image registration is the process of aligning images in a dataset with one another so that it is easy to compare and highlight similarities and differences among them. Registering images needs to determine a geometric transformation that aligns one image to fit another. It is usually used when comparing two MR images taken at different times for the same organ to study the progress of the case [22].  Bias Field Correction is another process that is used to improve the quality of the image before it goes through any further processing. Bias field signal is a low-frequency and very smooth signal that corrupts MRI images, this may cause image processing algorithms that use intensity values to be unable to produce satisfactory results. Therefore, preprocessing steps are needed to correct the bias field signal before applying image processing algorithms to MRI images [23]. Intensity normalisation is used to bring all images into a common scale of intensities which improves other image processing algorithms performance. The main types of normalisation are Whole-brain normalisation and White Stripe normalisation [24]. Skull stripping is another important pre-processing stage in which the effect of the skull is reduced to the minimum. The presence of non-brain tissues such as skin, fat, muscle, and eyeballs is an obstacle for automatic brain image segmentation and analysis techniques [25]. Several approaches were suggested to remove this no-brain information using various techniques such as histogram, texture, edges, and morphological operators [26], [27].

2.3.2 AI-based approaches

As shown in Fig. 3, automatic brain MRI analysis is divided mainly into AI-based and non-AI-based.  AI-based approaches utilise the principles of artificial intelligence and machine learning in identifying the abnormality in MR images. Different approaches have been suggested in this field using various AI techniques such as Fuzzy, neural networks, and deep learning. AI techniques have been used in different stages of MRI analysis such as pre-processing and clustering.  Fuzzy C-mean clustering (FMC) was used to cluster the pixels according to their features. The main advantage of this kind of clustering is that it allows the pixel to be a member of more than one cluster with a membership value [28]. This is useful when there are no crisp borders between the brain tissue and the tumour. More details about clustering are available in [29] and [1].

Machine learning techniques such as supervised and unsupervised learning were used in clustering and segmenting the images. Artificial neural nets, support vector machines, and deep learning algorithms are commonly used in such applications. To classify a set of images, a labelled dataset is required which includes a set of images with labels; these labels might be Yes/No or types of tumour. This dataset is used to train Convolutional Neural Network, Support Vector Machine [18], or Deep Learning algorithms [30], [12]. The algorithm is then used to predict the MR images status. The main challenge such applications may face is that they use different images to predict another image status such as in [17] and [31]. In other words, the model is trained using a certain set of images and the trained model is used to identify another new image. This may not give accurate results always due to the high variation of the shape of the brain and the tumour.

2.3.3 Non-AI-based approaches

This type of algorithms uses traditional image processing techniques to identify and segment the abnormality of the regions. Several techniques have been utilised such as thresholding, clustering, segmentation, and edge detection.  As shown in Fig. 3, non-AI-approaches are divided into pixel-based, edge-based, and region-based. In pixel-based, the algorithms use pixel features such as the intensity, colour-band value, and location in the analysis process. Thresholding is one of the commonly used approaches to split the contents of an image, in which a certain threshold value is selected either manually or automatically to separate pixels in the image into regions. Numerous thresholding techniques have been proposed since the early days of image processing such as bimodal and adaptive thresholding. Manual threshold value specification is used to create the ground truth dataset only and is not feasible to be used in other applications. Automatic threshold identification approaches may include statistical methods, bimodal histogram, Fuzzy histogram approximation [32].

Clustering, which is an unsupervised process, is another pixel-based approach in which the pixels with similar features such as location, intensity, and texture are grouped together to form groups. K-mean clustering and fuzzy c-mean algorithms are the most well-known clustering algorithms in which the user need to specify the number of clusters and the algorithm finds the related pixels based on the distance to the centre of the cluster. The centre of the cluster is then recalculated and moved to the new location and the process is repeated until the difference between the present location and the calculated one becomes insignificant.

The second type of non-AI approaches is the edge-based approaches. Edge is the sudden change of intensity of neighbouring pixels, this change can be detected using an edge detection algorithm such as Laplacian, Sobel, Canny and others [17].

Region-based approaches are the approaches that deal with a region rather than a pixel. The main region-based approaches are the split/merge approach and the region growing approach [33]. The split/merge approach works in a manner similar to k-mean clustering but here we use regions instead of pixel values. The split/merge process may split the object itself, the tumour in our case, into regions [1]. In the region growing approach, seed points are selected where the regions start growing from. For each seed, the distance to its neighbouring pixels is calculated and if it is found less than a predefined threshold, the point will be included in the region. Without a careful selection of the threshold values and seeds, these algorithms can cause separate regions to become connected. Besides, such algorithms need user interaction because they are not fully automatic as the user needs to select the seed points and the threshold values.

2.4 Saliency identification

Saliency identification is the process of highlighting the salient regions in an image based on how abrupt they are as compared to other parts of it. Detecting salient regions in an image is important for applications such as adaptive content delivery, image segmentation, and image and video compression [34]. Many approaches were proposed to extract the salient regions from an image in both spatial and frequency domains. Techniques such as Wavelet [35], [36], [37], [38], and [39], Geometric features such as corners [40], [41], Saliency map [42], [43], and Frequency domain [44], [45], [46], [47], [48], [49], [50] were widely used. The strengths and weaknesses of the mentioned methods are beyond the scope of this paper, a sufficient discussion is found in [51], and [52].

From studying the available approaches, we found that the saliency-based on irregularity can be used and can give excellent results. In this approach, the region is said to be salient if it differs significantly from the rest of the image in terms of intensity distribution, and this applies to any abnormality in the brain. Two statistical measures can be used to highlight the uniformity and the irregularity of a region, namely expected value and variation. In regular regions (the majority of the regions), the expected value is very close to the pixel’s value and the measure of variation is small, while in irregular regions, the difference between the intensity value of the pixels and the expected value is high and the variation measure is high also. Based on this, the measure of irregularity can be derived based on two measures namely, uniformity and variation; the mean or the median can be used as a measure of regularity or uniformity, while the variance can be used as a variation measure.

3 Saliency-Based Brain Abnormality Identification

3.1 Image Pre-processing

In our discussion, we will divide the image into four regions, which are the background region (BGR), the border region (BDR), the brain tissue region (BTR), and the abnormality region (ABR), and study the features and characteristics of each region to get a better understanding of the nature of each one of them and analyse it to get optimum results. Fig. 4 shows the four regions mentioned above.

 

Fig. 4. Regions of the brain MR image, (a) the original image, (b) the background region, (c) the border region, (d) the tissues region, and (e) the abnormality region.

 

Background region (BGR): As shown in Fig. 4 (b), this region does not usually contain any significant information and is of no importance in the analysis because it represents the background of the brain, but it may affect the calculations of other measures, such as mean and variance, due to the dominance of the black pixels.

Borderline region (BDR): This area may include eyeballs, skull, fat, and other unnecessary information. This area has little effect on calculating the mean and variance, however, it does affect the way abnormal areas in the image are highlighted when the saliency filter is applied. The BDR is shown in Fig. 4 (c).

Brain tissue region (BTR): The region shown in Fig. 4 (d) represents the brain's lobes and the meandering structures of the brain. This area is essential for calculating the mean and the variance that are used in the saliency filter.

Abnormal region (ABR): As shown in Fig. 4 (e), this area contains the abnormal part of the brain, whether it is a tumour or any other type of abnormality. This area is the part that the algorithms, including our proposed one, aim to identify and extract. This part usually differs from the brain tissue region in intensity and texture; therefore, it is supposed to be highlighted when the saliency filter is applied.

Based on the above discussion, we shall define and as the sets of unimportant and important pixels respectively so that the set of all pixels in the image is defined as given in equation (1).

(1)

The sets of all unimportant pixels and the set of the important pixels are defined in equation (2) as given below:

(2)

where and are the sets of pixels in the background and the border respectively, and and are the set of brain tissues pixels and the abnormality region respectively.

To remove the unimportant information from the image we shall define the mapping , which removes all the unnecessary information and keeps the important pixels only. The function can be implemented by masking the image with a mask derived from the image itself. The mask can be extracted by a thresholding process followed by morphological operations such as dilation and closing.

Fig. 5 shows the steps of removing the unimportant information. In this figure, (a) shows the original image, this image undergoes a binarisation process using Otsu thresholding technique. The image obtained from the binarisation process, which is shown in (b), contains some holes that affect the masking process. The morphological closing process is then applied to fill the gaps in the mask in (b). This process produces the mask shown in (c) which includes all the details of the brain including the borders.

As it was discussed earlier, the information contents of the borders are also part of the unimportant information. Therefore, the border should be removed as well using what is known as the skull stripping technique. After the skull stripping process, the mask shown in (e) is obtained which produces the important information of the brain given in (f).

 

Fig. 5. Implementation of by masking the image, (a) original image, (b) image after applying Otsu thresholding, (c) the mask after using morphological closing operation, (d) image after removing the background, (e) mask to remove the borders, (f) important information in BTR and ABR.

 

Procedure 1 summarises the process of extracting the important information from the image. The process starts with thresholding the image to extract the mask corresponds to the important information. This step is followed by a morphological closing filtering process to fill the gaps in the mask, then a blob extraction algorithm is applied to identify the largest blob which represents the border of the tissues (the skull) and remove it. Finally, the remaining blob is the brain tissues region which contains the abnormal part also.

Procedure 1 Extracting the important information from the image

Procedure Get_Important_info(image): Begin: # Apply Otsu thresholding Threshold = Otsu (image) # fill the gaps in the mask Mask = Morphological_Closing (Threshold) # get the connected regions Blobs [] = Get_Blobs (Mask) # sort the connected regions based on their size Blobs[] = Sort_descending (Blobs[]) # Remove the largest connected region which represents the border remove_largest_component(Blobs[]) # the second largest component is the mask corresponding to the important information Important_Mask = Blobs [1] # Extract the region which contains the important information important_Region = bitwise_AND (Import_Mask , image) End

 

The effect of the presence of the background and borders on the mean and variance values is shown in Table 1, which shows the same image with different extent of background and borders, and the corresponding mean, median, standard deviation, and variance values. In case No. (1), the value of the mean and median is much smaller than the rest of the cases. This is due to the effect of the background which consists of black pixels with intensity values of zero. In case No. 2, the effect of the background is slightly less than in case No. 1 as we removed most of the background, but even though most of the black pixels were removed, the remaining background still affects the mean and median measures. In case No. 3, the background effect was completely removed and only the brain image is considered. The mean and median values are higher than the previous cases and the deviation is lower as the variation is reduced when the black area is removed. Finally, in case No. 4, the mean and median values are lower than in case No. 3 as the border was represented in this image by light pixel.

Table 1. The mean and variance values corresponding to various cases of background and borders presence.

3.2 Abnormality identification

In this section, we shall discuss the process of applying the irregularity filter. As it was discussed earlier, the irregularity shall be measured in terms of uniformity which is represented by the expected value and variation which is represented by the variance or the standard deviation.  The irregularity measure can be expressed as given in equation (3) [52], [51]. The expected value (the mean) and the variation value (the variance or the standard deviation) or are calculated for the entire image, thus the irregularity mapping ( ) can be extracted as a function of the expected value and the variation value as given in equation (3). Since Standard deviation looks at how spread out a group of numbers is from the mean and the variance measures the average degree to which each point differs from the mean, the variance is used in identifying the variation as our interest focuses on the variation regardless of the distribution.

(3)

In the above equation, the mapping is used to highlight the value of the pixel intensity, is the value of the pixel in the location after applying the saliency filter.

To show that the value of is small in regular regions and large in irregular regions, we need first to discuss the values of the expected value and the variance. The expected value and the variation of continuous random variables are given in the following equations:

(4)

Since the digital image is discrete, then the above formulas are represented as follows:

(5)

Since the probability of occurrence of each pixel is , where , the mean is then equal to:

(6)

To prove that the value of is minimum in regular regions, we need first to find the minimum value of the mean square error which is represented as . The minimum value can be found by differentiating with respect to and equate it to zero.

(7)

Since is constant and we are applying the mean square error on a single pixel then the equation will be:

(8)

The maximum value of is 255 when all pixels are white and the maximum value of is always less than 255 when the mean is maximum, and the pixel intensity is 0. will not reach 255 as the presence of pixel with an intensity value of zero will never allow the mean to be 255.

In the same way, we can prove that the variance is minimum at the regular regions.

(9)

3.3 Proposed Algorithm

The proposed algorithm is summarised in the algorithm given in Procedure 2 and the diagram shown in Fig. 6. The following are the main steps of the algorithm:

1. Filtering: In this step, a low pass filter is applied to remove any outliers and noise; many filters can be applied here such as averaging, median, and bilateral filters. Bilateral filter is preferred because it does not degrade the edges much and effective in removing the noise and outliers.

2. Skull striping: In this step, the unimportant details such as skull and eyeballs are removed using the method described in Procedure 1.

3. Saliency filter application: To highlight the salient region of the image, which is mostly the abnormal region, a saliency filter is applied. The resulting image can then be easily thresholded and converted into a binary image.

4. Abnormality mask extraction: To improve the binary image, the application of some morphological operations such as closing, opening, and erosion was studied, and their effect on improving the results was discussed.

5. Masking: To extract the abnormal region, a bitwise ANDing process is applied between the extracted mask and the original image. The resulting image represents the abnormal region.

Procedure 2. Extracting the abnormal region from the image.

Procedure Extract_Abnormal_Region(Original_image): Begin: # Apply Smoothing Smoothed_image = Bilateral_Smoothing (Original_image) # Apply Skull Striping NSKL = Skull_Stripping (Smoothed_image) # Apply Saliency Filter Saliency = Apply_Saliency (NSKL) # Apply Binarisation Binary = Binarisation (Saliency) # Improve Binary Image using Morphological Filters Abnormality_mask = Morphological_Filter (Binary) # Extract the resultant abnormal Part Result = Bitwaise_And (Abnormality_mask, Original_image)         End

 

Fig. 6. Proposed algorithm diagram.

4 Results and Discussion

The proposed approach has been implemented and applied to a standard dataset containing 250 various brain images established by selecting a subset from “Brain MRI Images for Brain Tumour Detection” [14] and “Brain Tumour Classification (MRI)” [15].

In the discussion we shall consider the following cases which are shown in Fig. 7:

1. Whole image: In this case, the image as a whole is processed as shown in Fig. 7 (a).

2. Reduced Background (RBG): In this case, the background is reduced to the minimum as given in Fig. 7 (b).

3. No Background (NBG): In this case, only the brain tissues and the borders are considered as shown in Fig. 7 (c).

4. No Skull (NSKL): In this case only the brain tissues including abnormality, if any, are considered. This case is shown in Fig. 7 (d).

 

Fig. 7. The four cases that are used in our test, (a) whole image, (b) reduced background, (c) no background, (d) no skull.

4.1 Statistical Measures

The statistical measures and the effect of the presence of the background, borders, and skull have been studied carefully to select the appropriate measures which give the best result. The curves in Fig. 8 (a) show a comparison among the values of the mean in the four mentioned cases. From the figure, it is clear that the mean in the first case (whole image) is lower than in other cases. This is because of the effect of the background which contains mostly black pixels with low values. The same applies to the RBG curve where the effect of the rest of the background is still present. The NBG and NSKL curves show that the value is higher and are close to each other.

Fig. 8 (b) shows the curves of the variance for the four cases, from the curves one can notice that the variance values in the first two cases (whole and RGB) are high and this is due to the presence of the background. In the case of NBG, the background has no effect, but the presence of borders represented by the skull, eyeballs and other details made the variance value higher than in the last case, which is NSKL.

 

Fig. 8. Comparison of the mean and variance in the four cases, whole, RBG, NBG, and NSKL, (a) expected value curves, (b) variance curves.

 

The effects of the presence of unnecessary information such as the background of the image and the boundaries of the brain are also shown in Fig. 9 (a), where the averages of the mean and the median values are lower in the cases of the whole image and RBG and being higher in NBG and NSKL while the average value of the standard deviation decreases as the background and boundaries diminish. Finally, Fig. 9 (b) shows a comparison between the mean and the median values of the four cases. The values are close to each other as the background effect decreases until very close values are reached in the absence of the background.

 

Fig. 9. Comparison of the average of mean, median, and standard deviation in the four cases: whole, RBG, NBG, and NSKL, (a) mean, median, and standard deviation for each case, (b) mean and median values for each case.

 

Fig. 10 shows the results obtained from applying the algorithm to a sample image from the dataset. The figure shows that the original image undergoes a skull stripping process giving a skull stripped image. The resulting image then undergoes a saliency enhancement process to highlight the abnormal regions. The mask is then extracted to be used to limit the saliency algorithm from being applied to the background or boundaries. The selection of the threshold value to create the binary image is not a difficult process as the difference between the important region (abnormal) and other unimportant regions is high. The value of the threshold can be selected between 100 and 200 and in our experiments, we used 127 as this value represents the midpoint of the grey levels.

The obtained binary image contains the salient region, which is the region that contains the abnormal parts, in addition to some other small regions which were falsely detected. Therefore, a morphological operation is needed to improve the result, where several operations were tested to determine the operation that produces the best results. From the figure, it is evident that the morphological closing operation improved the abnormal area but increased the size of other undesirable and incorrectly identified regions while the erosion operation removed and reduced the size of the unwanted parts, but it also reduced the size of the abnormal area. Finally, the morphological opening filter removed the wrongly detected areas and kept the salient area unchanged.

In many cases, the edges of the tissue appeared as important regions and to reduce this effect, a refinement was made by shrinking the mask by five pixels or by no more than 3% of its size to overcome this effect. This amendment does not affect the results as its effect is limited to the cases where the abnormal area is located on the borders, which is rare, and this modification does not have a significant impact on the results obtained. This is illustrated in the third row of Fig. 10.

 

Fig. 10. Applying the proposed algorithm to a sample of the dataset.

4.2 Evaluation

The precision-recall measures given in equation (10) are widely used in comparing binary images with each other such as salient regions comparison with the ground truth data. The same can be applied to comparing the obtained abnormal regions masks with the ground truth masks.

(10)

The formulas of the precision and recall given in equation (10) are modified to make them suitable for comparing the extracted region with the ground truth images. The new formulas are given in equation (11).

(11)

where and G are the resulting image after identifying the abnormality and the ground truth image, respectively.

F-Measure is used to evaluate the overall performance and is defined as the weighted harmonic mean of precision and recall and is given by equation (12).

(12)

4.2.1 Precision, Recall, F-Measure Features

The main features of precision, recall, and F-Measures are listed below:

1. The precision measure is high when the intersection between the extracted region and the region in the ground truth image is large, but it is also high when the ground truth region is a subset of the extracted region, which is a drawback of this measurement as illustrated in cases 3, 4, and 7 in Table 2.

2. The recall measure is high when the intersection between the regions in the two images is high, but it is high also if the extracted region in the resulting image is a part of the region in the ground truth image as shown in case 6 in Table 2.

3. The F measure is extracted from both measures and it reduces the effects mentioned in the above points.

Table 2. Precision, Recall, and F-Measure for the various cases.

4.2.2 Cases considered in the evaluation

In the following discussion we shall consider the following cases:

1. B: The binary image obtained from applying the thresholding process.

2. C: The binary image after applying morphological closing.

3. E: The binary image after applying erosion.

4. O: The binary image after applying morphological opening

5. BC: The binary image obtained from applying the thresholding after the shrinking refinement.

6. CC: The binary image obtained after applying morphological closing to BC.

7. EC: The binary image obtained from applying morphological erosion to BC.

8. OC: The binary image obtained from applying morphological opening to BC.

Fig. 11 shows the precision, recall, and F-measure curves as well as the average of the three measures in all of the mentioned eight cases. In this figure, (a) shows the precision curves in which the best curves are associated with B and C cases and the worst cases are associated with erosion (E and EC). In Fig. 11 (b), which shows the recall curves, the best curves are associated with erosion and the ones associated with B and C cases are much less than others. This is because of the drawbacks mentioned above, as in the cases E and EC the resulting regions might be subsets the ground truth region as the erosion usually reduces the size of the resulting region and hence the recall value is high, while in case B, the value is smaller because of the extra undesired regions that were falsely extracted.

 

Fig. 11. Precision, Recall, and F-Measure Curves, (a) precision, (b) recall, (c) F-measure, (d) average measures.

 

The F-measure shown in Fig. 11 (c) is more accurate than the precision and recall measures separately as it is derived from both measures. According to the curves in Fig. 11 (c) and the averages given in Fig. 11 (d), it was found that the best results have been obtained from B, O, C, BC, OC, and CC.

To differentiate between the cases mentioned above, a qualitative comparison has been made as shown in Fig. 12, which shows samples of the results obtained. By combining both qualitative and quantitative evaluation, we concluded that C and CO are the cases where the best results can be obtained.

 

Fig. 12. Samples of the results obtained from applying the proposed algorithm.

4.2.3 Results Benchmarking

The obtained results were benchmarked with state-of-the-art research considering in the benchmarking the dataset and the approach used in addition to the accuracy. Unfortunately, there is no single approach to measure or evaluate the effectiveness of the segmentation process and even the accuracy which is adopted by many research cannot give accurate evaluation alone. The accuracy is usually calculated as a ratio of the correctly identified images to the total images in the dataset and ignores the quality of the extracted region.

The methods considered in the benchmarking are listed in Table 3 and a brief description is given below:

1- Anitha and Murugavalli (AM), in this approach, the authors used an adaptive K-means algorithm for segmenting and isolating the tumour from the rest of the brain tissues using three custom datasets including 40, 60 and 70 images respectively [53].  The main limitation of this approach is the small number of images in the dataset.

2- El-Dahshan et al. (ELD-1), where the authors used Pulse Coupled Neural Network (PCNN) for segmenting the images. They used a subset from the Harvard Medical School dataset (HMS) including 101 images divided into 14 normal and 87 abnormal brain images. The dataset was divided into 65 images for training and 36 for testing [54]. The authors reported an accuracy of 99% where the number of images in the test set is 36. If 35 images were predicted correctly that means the accuracy is 97.2%. The other limitation is the low number of images used in the training and the test.

3- Zöllner et al. (ZO), in this method, the author suggested the use of S upport vector machine (SVM) and applied their algorithm to 101 images from Gadovist, Bayer Schering Pharma dataset (BSP) [55]. Again, the main limitation is the small number of images in the dataset and the accuracy which was reported in the research is 85%.

4- El-Dahshan et al. (ELD-2), this research was used as well in the benchmarking process. This research uses feed-forward back- propagation artificial neural network (FP-ANN) and k-nearest neighbour (k-NN) classifiers to classify whether the brain is normal or abnormal. This research is not quite useful in our benchmarking as the proposed algorithm does not include a segmentation process. They applied their method to 70 images from HMS dataset [56]. The author reported an accuracy of 98%.

5- Gilanie et al. (GIL) approach, where the authors used Gabor texture features and SVM to classify the images into normal and abnormal. They used three subsets from HMS dataset with 101, 75 and 70 images respectively [57]. Although the authors have reported an accuracy of 100%, the small number used in the experiments is still a limitation.

6- Damodharan and Raghavan (DR) approach, this approach uses a neural network for classification and  WM, GM, CSF for segmentation [58]. There is no information available about the dataset and the number of images used. The authors reported an accuracy of 85%.

7- Zanaty  (ZN) approach, In which, the authors proposed a hybrid approach, combining FCM, seed region growing (SRG), and Jaccard similarity coefficient (JSC) algorithm for segmentation [59].  The accuracy was reported to be 90% with no information about the dataset.

8- Kumar and Vijayakumar (KV) approach, in this approach, the authors introduced a method that used principal component analysis (PCA) and radial basis function (RBF) kernel-based SVM for segmentation and classification [60]. The authors used a subset of HMS dataset and no information is available about the number of images. The authors reported an accuracy of 94%.

9- Cui et al. (CUI) approach, in which, the authors used a localised fuzzy clustering (LFC) with spatial information to form an objective of medical image segmentation and bias field estimation (BFE) for brain MR images [61]. The authors reported an accuracy of 83% to 94% and no clear information found about the dataset.

10- Sachdeva et al. (SAC) approach, where the authors proposed using ANN PCA-ANN to classify, segment, and extract features from the MR images [62]. The authors reported an accuracy of 77% to 91% using images for 55 patients.

From the above discussion, we can see that the proposed approach has produced a high accuracy value.

Table 3. Benchmarking with existing approaches

5 Conclusions

In this research, a new approach to extract the abnormal regions from the MRI images is presented. The new approach considered the saliency extraction algorithms in the identification process as it considered the similarity between the abnormality extraction in MRI and the saliency extraction definition. Irregularity-based saliency extraction approach was used as the abnormal region in the brain, which is probably a tumour, is smaller than other parts of the brain and differs in terms of luminance, colour, and texture. The algorithm was applied to a standard database consisting of 250 images and the obtained results were discussed thoroughly. Various cases have been considered to decide the optimum conditions which give the best results. The obtained results have been evaluated using a common evaluation approach which is the precision-recall measure in addition to the F-measure which is derived from these two measures. The obtained results showed a high level of accuracy that reached 96%. The proposed algorithm used statistical measures in deriving the irregularity identification function, nevertheless, other measures can be used and tested as well such as those that can be derived from the structure or the texture of the image.

References

1. M. S. H. Al-Tamimi and G. Sulong, “Tumor brain detection through MR images: A review of literature,” J. Theor. Appl. Inf. Technol., vol. 62, no. 2, pp. 387–403, 2014.

2. D. M. Kumar, D. Satyanarayana, and M. N. G. Prasad, “An improved Gabor wavelet transform and rough K-means clustering algorithm for MRI brain tumor image segmentation,” Multimed. Tools Appl., Oct. 2020, doi: 10.1007/s11042-020-09635-6.

3. S. Saritha and N. Amutha Prabha, “A comprehensive review: Segmentation of MRI images—brain tumor,” Int. J. Imaging Syst. Technol., vol. 26, no. 4, pp. 295–304, 2016, doi: 10.1002/ima.22201.

4. P. N. Kale and R. T. Vyavahare, “MRI Brain Tumor Segmentation Methods- A Review,” 1271| Int. J. Curr. Eng. Technol., vol. 6, no. 4, pp. 1271–1280, 2016, Accessed: Dec. 01, 2020. [Online]. Available: http://inpressco.com/category/ijcet.

5. S. R. Rajput and M. S. Raval, “A Review on End-To-End Methods for Brain Tumor Segmentation and Overall Survival Prediction,” arXiv, 2020, doi: 10.32010/26166127.

6. S. M. Kulkarni and G. Sundari, “A Review on Image Segmentation for Brain Tumor Detection,” in Proceedings of the 2nd International Conference on Electronics, Communication and Aerospace Technology, ICECA 2018, 2018, vol. 3, pp. 552–555, doi: 10.1109/ICECA.2018.8474893.

7. A. Wadhwa, A. Bhardwaj, and V. Singh Verma, “A review on brain tumor segmentation of MRI images,” Magnetic Resonance Imaging, vol. 61. pp. 247–259, 2019, doi: 10.1016/j.mri.2019.05.043.

8. A. Anand and H. Kaur, “Survey on Segmentation of Brain Tumor: A Review of Literature,” Ijarcce, vol. 5, no. 1, pp. 79–82, 2016, doi: 10.17148/ijarcce.2016.5118.

9. S. Tripathi, R. S. Anand, and E. Fernandez, “A Review of Brain MR Image Segmentation Techniques,” Int. J. Res. Anal. Rev., vol. 5, no. 2, p. 1295, 2018, [Online]. Available: http://ijrar.com/.

10. A. Işin, C. Direkoǧ lu, and M. Şah, “Review of MRI-based Brain Tumor Image Segmentation Using Deep Learning Methods,” in Procedia Computer Science, 2016, vol. 102, pp. 317–324, doi: 10.1016/j.procs.2016.09.407.

11. S. M. Anwar, M. Majid, A. Qayyum, M. Awais, M. Alnowami, and M. K. Khan, “Medical Image Analysis using Convolutional Neural Networks: A Review,” J. Med. Syst., vol. 42, no. 11, 2018, doi: 10.1007/s10916-018-1088-1.

12. L. Pei, L. Vidyaratne, M. M. Rahman, and K. M. Iftekharuddin, “Context aware deep learning for brain tumor segmentation, subtype classification, and survival prediction using radiology images,” Sci. Rep., vol. 10, no. 1, pp. 1–11, 2020, doi: 10.1038/s41598-020-74419-9.

13. N. B. Bahadure, A. K. Ray, and H. P. Thethi, “Comparative Approach of MRI-Based Brain Tumor Segmentation and Classification Using Genetic Algorithm,” J. Digit. Imaging, vol. 31, no. 4, pp. 477–489, Aug. 2018, doi: 10.1007/s10278-018-0050-6.

14. N. Chakrabarty, “Brain MRI Images for Brain Tumor Detection,” 2018. https://www.kaggle.com/navoneel/brain-mri-images-for-brain-tumor-detection (accessed Oct. 01, 2020).

15. Sartaj, “Brain Tumor Classification (MRI),” 2020. https://www.kaggle.com/sartajbhuvaji/brain-tumor-classification-mri (accessed Aug. 01, 2020).

16. E. S. H. Ibrahim and R. E. Gabr, “MRI basics,” in Heart Mechanics: Magnetic Resonance Imaging-Mathematical Modeling, Pulse Sequences, and Image Analysis, 2017, pp. 81–120.

17. Q. Li, Z. Yu, Y. Wang, and H. Zheng, “Tumorgan: A multi-modal data augmentation framework for brain tumor segmentation,” Sensors (Switzerland), vol. 20, no. 15. pp. 1–16, 2020, doi: 10.3390/s20154203.

18. G. Tomasila and A. W. Rahardjo Emanuel, “MRI image processing method on brain tumors: A review,” in AIP Conference Proceedings, 2020, vol. 2296, p. 20023, doi: 10.1063/5.0030978.

19. H. Peni Agustin Tjahyaningtijas, “Brain Tumor Image Segmentation in MRI Image,” IOP Conf. Ser. Mater. Sci. Eng., vol. 336, no. 1, 2018, doi: 10.1088/1757-899X/336/1/012012.

20. Z. Akkus, A. Galimzianova, A. Hoogi, D. L. Rubin, and B. J. Erickson, “Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions,” Journal of Digital Imaging, vol. 30, no. 4. Springer New York LLC, pp. 449–459, Aug. 01, 2017, doi: 10.1007/s10278-017-9983-4.

21. N. Kumar and M. Nachamai, “Noise Removal and Filtering Techniques used in Medical Images,” Orient. J. Comput. Sci. Technol., vol. 10, no. 1, pp. 103–113, Mar. 2017, doi: 10.13005/ojcst/10.01.14.

22. P. J. Kostelec and S. Periaswamy, “Image Registration for MRI,” Mod. Signal Process., vol. 46, pp. 161–184, 2003.

23. J. Juntu, J. Sijbers, D. Dyck, and J. Gielen, “Bias Field Correction for MRI Images,” Springer, Berlin, Heidelberg, 2008, pp. 543–551.

24. J. C. Reinhold, B. E. Dewey, A. Carass, and J. L. Prince, “Evaluating the Impact of Intensity Normalization on MR Image Synthesis,” arXiv. SPIE, p. 126, Mar. 15, 2018, doi: 10.1117/12.2513089.

25. P. Kalavathi and V. B. S. Prasath, “Methods on Skull Stripping of MRI Head Scan Images—a Review,” Journal of Digital Imaging, vol. 29, no. 3. pp. 365–379, Jun. 01, 2016, doi: 10.1007/s10278-015-9847-8.

26. J. Swiebocka-Wiek, “Skull stripping for MRI images using morphological operators,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 9842 LNCS, pp. 172–182, 2016, doi: 10.1007/978-3-319-45378-1_16.

27. S. M. Smith, “Fast robust automated brain extraction,” Hum. Brain Mapp., vol. 17, no. 3, pp. 143–155, Nov. 2002, doi: 10.1002/hbm.10062.

28. J. Chen, C. Yang, G. Xu, and L. Ning, “Image Segmentation Method Using Fuzzy C Mean Clustering Based on Multi-Objective Optimization,” J. Phys. Conf. Ser., vol. 1004, no. 1, p. 012035, Apr. 2018, doi: 10.1088/1742-6596/1004/1/012035.

29. M. Al-Azawi, Y. Yang, and H. Istance, “A new gaze points agglomerative clustering algorithm and its application in regions of interest extraction,” Souvenir 2014 IEEE Int. Adv. Comput. Conf. IACC 2014, no. 1, pp. 946–951, 2014, doi: 10.1109/IAdCC.2014.6779450.

30. W. Wu et al., “An Intelligent Diagnosis Method of Brain MRI Tumor Segmentation Using Deep Convolutional Neural Network and SVM Algorithm,” Comput. Math. Methods Med., vol. 2020, 2020, doi: 10.1155/2020/6789306.

31. C. C. Benson, V. Deepa, V. L. Lajish, and K. Rajamani, “Brain tumor segmentation from MR brain images using improved fuzzy c-means clustering and watershed algorithm,” in 2016 International Conference on Advances in Computing, Communications and Informatics, ICACCI 2016, Nov. 2016, pp. 187–192, doi: 10.1109/ICACCI.2016.7732045.

32. M. A. N. Al-azawi, “Image Thresholding using Histogram Fuzzy Approximation,” Int. J. Comput. Appl., vol. 83, no. 9, pp. 36–40, 2013, doi: 10.5120/14480-2781.

33. I. Zabir, S. Paul, M. A. Rayhan, T. Sarker, S. A. Fattah, and C. Shahnaz, “Automatic brain tumor detection and segmentation from multi-modal MRI images based on region growing and level set evolution,” in 2015 IEEE International WIE Conference on Electrical and Computer Engineering, WIECON-ECE 2015, Mar. 2016, pp. 503–506, doi: 10.1109/WIECON-ECE.2015.7443979.

34. M. Al-azawi, “Saliency Identification as a Computational Model of Human Visual Attention,” Int. J. Adv. Sci. Technol., vol. 29, no. 3, pp. 3348–3360, 2020, [Online]. Available: http://sersc.org/journals/index.php/IJAST/article/view/4780.

35. E. Loupias, N. Sebe, S. Bres, and J.-M. Jolion, “Wavelet-based salient points for image retrieval,” in International Conference on Image Processing, 2002, pp. 518–521 vol.2, doi: 10.1109/icip.2000.899469.

36. Q. Tian, N. Sebe, M. S. Lew, E. Loupias, and T. S. Huang, “Content-based image retrieval using wavelet-based salient points,” in Storage and Retrieval for Media Databases 2001, 2001, vol. 4315, pp. 425–436, doi: 10.1117/12.410953.

37. H. Song, B. Li, and L. Zhang, “Color salient points detection using wavelet,” Proc. World Congr. Intell. Control Autom., vol. 2, pp. 10298–10301, 2006, doi: 10.1109/WCICA.2006.1714018.

38. S.-H. Lin, Dong-Woei; Yang, “Wavelet-Based Salient Region Extraction,” in Advances in Multimedia Information Processing – PCM 2007. Vol. 4810, Hong Kong: Springer, 2007, pp. 389–392.

39. R. N. Arivazhagan, S.; Shebiah, S. Arivazhagan, and R. N. Shebiah, “Object Recognition using Wavelet Based Salient Points,” Open Signal Process. J., vol. 2, no. 2000, pp. 14–20, 2009.

40. P. Kapsalas, K. Rapantzikos, A. Sofou, and Y. Avrithis, “Regions of interest for accurate object detection,” in 2008 International Workshop on Content-Based Multimedia Indexing, CBMI 2008, Conference Proceedings, 2008, pp. 147–154, doi: 10.1109/CBMI.2008.4564940.

41. C. Schmid and R. Mohr, “Local grayvalue invariants for image retrieval,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 5, pp. 530–535, 1997, doi: 10.1109/34.589215.

42. C. Koch and S. Ullman, “Shifts in selective visual attention: Towards the underlying neural circuitry,” Hum. Neurobiol., vol. 4, no. 4, pp. 219–227, 1985, doi: 10.1007/978-94-009-3833-5_5.

43. M. B. Vinay and K. S. Rekha, “A model of saliency-based visual attention for rapid scene analysis,” Int. J. Recent Technol. Eng., vol. 7, no. 6, pp. 412–415, 2019.

44. N. D. B. Bruce, D. P. Loach, and J. K. Tsotsos, “VISUAL CORRELATES OF FIXATION SELECTION: A LOOK AT THE SPATIAL FREQUENCY DOMAIN Department of Computer Science and Centre for Vision Research 4700 Keele Street , Toronto , Ontario , Canada M3J 1P3,” Spectrum, pp. 289–292, 2007.

45. X. Hou and L. Zhang, “Saliency detection: A spectral residual approach,” 2007, doi: 10.1109/CVPR.2007.383267.

46. J. Li, M. D. Levine, X. An, X. Xu, and H. He, “Visual saliency based on scale-space analysis in the frequency domain,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 4, pp. 996–1010, 2013, doi: 10.1109/TPAMI.2012.147.

47. S. Liu and J. Hu, “Visual saliency based on frequency domain analysis and spatial information,” Multimed. Tools Appl., vol. 75, no. 23, pp. 16699–16711, 2016, doi: 10.1007/s11042-016-3903-3.

48. R. Achantay, S. Hemamiz, F. Estraday, and S. Süsstrunky, “Frequency-tuned salient region detection,” 2009 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work. CVPR Work. 2009, vol. 2009 IEEE, no. Ic, pp. 1597–1604, 2009, doi: 10.1109/CVPRW.2009.5206596.

49. L. Zhou, Bolei; Hou, Xiaodi; Zhang, “A phase discrepancy analysis of object motion,” 2010.

50. C.-W. Fang, Yuming; Lin, Weisi; Lee, Bu-Sung; Lau, Chiew-Tong; Chen, Zhenzhong; Lin et al., “Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum,” IEEE Trans. Multimed., vol. 14, no. 1, pp. 187–198, 2012, doi: 10.1109/TMM.2011.2169775.

51. M. Al-Azawi, Y. Yang, and H. Istance, “Irregularity-based saliency identification and evaluation,” in 2013 IEEE International Conference on Computational Intelligence and Computing Research, Dec. 2013, pp. 1–6, doi: 10.1109/ICCIC.2013.6724128.

52. M. A. N. Al-Azawi, “Saliency-Based Image Retrieval as a Refinement to Content-Based Image Retrieval,” ELCVIA Electron. Lett. Comput. Vis. Image Anal., vol. 20, no. 1, p. 1, Jan. 2021, doi: 10.5565/rev/elcvia.1325.

53. V. Anitha and S. Murugavalli, “Brain tumour classification using two-tier classifier with adaptive segmentation technique,” IET Comput. Vis., vol. 10, no. 1, pp. 9–17, 2016, doi: 10.1049/iet-cvi.2014.0193.

54. E. A. S. El-Dahshan, H. M. Mohsen, K. Revett, and A. B. M. Salem, “Computer-aided diagnosis of human brain tumor through MRI: A survey and a new algorithm,” Expert Syst. Appl., vol. 41, no. 11, pp. 5526–5545, 2014, doi: 10.1016/j.eswa.2014.01.021.

55. F. G. Zöllner, K. E. Emblem, and L. R. Schad, “SVM-based glioma grading: Optimization by feature reduction analysis,” Z. Med. Phys., vol. 22, no. 3, pp. 205–214, Sep. 2012, doi: 10.1016/j.zemedi.2012.03.007.

56. E. S. A. El-Dahshan, T. Hosny, and A. B. M. Salem, “Hybrid intelligent techniques for MRI brain images classification,” Digit. Signal Process. A Rev. J., vol. 20, no. 2, pp. 433–441, 2010, doi: 10.1016/j.dsp.2009.07.002.

57. G. Gilanie, U. I. Bajwa, M. M. Waraich, Z. Habib, H. Ullah, and M. Nasir, “Classification of normal and abnormal brain MRI slices using Gabor texture and support vector machines,” Signal, Image Video Process., vol. 12, no. 3, pp. 479–487, 2018, doi: 10.1007/s11760-017-1182-8.

58. S. Damodharan and D. Raghavan, “Combining tissue segmentation and neural network for brain tumor detection,” Int. Arab J. Inf. Technol., vol. 12, no. 1, pp. 42–52, 2015.

59. EA Zanaty, “Determination of Gray Matter (GM) and White Matter (WM) Volume in Brain Magnetic Resonance Images (MRI),” Int. J. Comput. Appl., vol. 45, no. 3, pp. 16–22, 2012, [Online]. Available: https://pdfs.semanticscholar.org/1757/d8a0e6292835eb598bc9c40fac7b71029822.pdf.

60. P. Kumar and B. Vijayakumar, “Brain Tumour Mr Image Segmentation and Classification Using by PCA and RBF Kernel Based Support Vector Machine,” Middle-East J. Sci. Res., vol. 23, no. 9, pp. 2106–2116, 2015, doi: 10.5829/idosi.mejsr.2015.23.09.22458.

61. W. Cui, Y. Wang, Y. Fan, Y. Feng, and T. Lei, “Localized FCM clustering with spatial information for medical image segmentation and bias field estimation,” Int. J. Biomed. Imaging, vol. 2013, 2013, doi: 10.1155/2013/930301.

62. J. Sachdeva, V. Kumar, I. Gupta, N. Khandelwal, and C. K. Ahuja, “Segmentation, feature extraction, and multiclass brain tumor classification,” J. Digit. Imaging, vol. 26, no. 6, pp. 1141–1150, 2013, doi: 10.1007/s10278-013-9600-0.