Classification of Red Blood Cell Abnormality in Thin Blood Smear Images using Convolutional Neural Networks

One of the most common morphological red blood cell abnormalities encountered during routine thin blood smear microscopy for the detection of malaria parasite is the rouleaux formation, which is the stacking together of red blood cells to form a chain. Rouleaux formation signifies an underlying infection and as such microscopists are mandated to report its presence. A lot of work has been done in automating malaria diagnosis using deep learning, but no model has been developed which is capable of detecting rouleaux formation in malaria infected red blood cells. Thus, this study collected 231 peripheral blood smear (PBS) images of normal red blood cell morphology and 231 PBS images with rouleaux morphology. The images were pre-processed and segmented into equal instances of 3044 coloured images of size 750×750 pixels. Two convolutional neural network (CNN) models were developed and trained to classify the images into normal red blood cell morphology or rouleaux morphology. The CNN models were trained on two different image sizes: 300×300 and 500×500. The first CNN model achieved validation accuracy/loss values of 87.91%/0.8177 and 56.58%/1.4090 when trained on images of sizes 300×300 and 500×500 respectively. In the second CNN model, the CNN layers of the first model were replaced with depthwise separable CNN layers, it was also trained on images of sizes 300×300 and 500×500 achieving validation accuracy/loss values of 90.95%/0.2804 and 87.75%/0.5904 respectively. This study demonstrates the capability of CNN models in detecting red blood cell morphology abnormality in thin smear images at an optimal image size of 300×300.


Introduction
According to the 2021 world malaria report; Nearly half of the world population live in areas at risk of malaria transmission in 87 countries and territories and the direct cost associated with the disease such as treatment and premature death has been estimated at 12 billion dollars annually, the cost in lost economic growth is estimated at a much higher value in 2020 [1].Digital thick and thin blood smear microscopy is still the most preferred method of malaria diagnosis globally.It entails the visual inspection of Giemsa or field-stained thin blood smear slides, for parasite identification, specie inspection and life stage assessment of the identified specie while the thick blood smear slide is inspected for parasite detection and quantification.Visual microscopy is tedious, time consuming, error prone and highly subjective as the accuracy of such a technique depends on the expertise of the microscopist which is highly lacking in malaria endemic regions [2].Thus, to standardize thick and thin smear microscopy result and tackle the dearth of skilled personnel in malaria endemic regions, research in the field of automating the detection of malaria parasites in light microscopy images using deep learning has been an active area of research for the past decade [2,3,4].
In a real laboratory setting, examining thin blood smear slides goes beyond malaria parasite identification, other abnormalities detected in the slides are also reported by the microscopists in charge.One such abnormality commonly reported is the rouleaux formation depicted in figure 1.This abnormality is a structural deformity caused by the sticking together or linking of red blood cells into chains like stacks of coins.It occurs due to an increase in abnormal proteins in the plasma such as immunoglobulins or excess fibrinogen, which is usually seen in inflammatory diseases [5,6].It is an indication of an underlying infection such as myeloma, macroglobulinemia, chronic liver disease, amyloidosis, or connective tissue disease [6].
Furthermore, identifying parasites when there is a rouleaux formation poses difficulty, as the cells are stacked together.When such a formation is encountered the slide must be washed again for proper viewing of the cells.To develop a truly robust and automated system of diagnosing malaria, such abnormalities should be taken into consideration, and a means of notifying the operator of such occurrence developed such that when it is encountered, instead of giving a false negative, the slide could be prepared again to have a better visualization and the abnormality reported; as a rouleaux formation as per the standard International Council for Standardization of Haematology (ICSH) protocol [7].
To the knowledge of the author no dataset has been developed, or model trained which contains or identifies rouleaux formation.To tackle these drawbacks, this study developed a dataset of thin blood smear images containing normal red blood cell morphology and rouleaux morphology.The study developed and compared two different deep learning models which were trained and tested on the dataset to differentiate between the two classes of red blood cell morphologies.The dataset developed will be made available to the research community to further aid research in automating malaria diagnosis.The model developed will also serve as the first stage of a truly robust and automated malaria diagnostic system capable of in cooperating capabilities of reporting not just malaria parasite, but other deformities reported by microscopists when carrying out routine blood smear microscopy in low resource settings.

Relevant literature
The process of automating red blood cell detection or malaria detection in red blood cells entails segmenting the red blood cells, pre-processing the segmented images, and then classifying the segmented red blood cells into malaria infected or healthy.The work by Delgado-Ortet et al developed a three-stage pipeline that involves segmentation, cropping and masking and the binary classification of malaria infected and healthy cells.The final classification stage a CNN with 13 layers achieved a validation accuracy of 95% and a test accuracy of 75% [8].Another study by Angel Molina et al developed a dataset containing malaria infected red blood cells and other inclusion such as Howell jolly bodies, pappenheimer bodies, basophilic stippling, and platelets on the red blood cell.Images of thin blood smear were segmented into individual red blood cells.A VGG-16 pre-trained model was trained on the dataset of single red blood cell images achieving a classification accuracy of 99.5% [9].Zhao et al, compared the performance of a 17-layer deep CNN model and a an AlexNet mode pretrained on CIFAR for the classification of malaria parasite from thin blood smear images.The trained CNN model had better performance with an accuracy of 93.7% compared to the transfer learning model which achieved an of accuracy of 91.99% [10].Rajaraman et al Developed a CNN model to classify RBCs into malaria infected and non-infected.The model was compared with ALEXNET, RESNET50, VGG-16, XCEPTION and DenseNet pre-trained on ImageNet.RESNET-50 recorded the highest accuracy of 95.7% [11].Asma et al used image augmentation techniques to improve their 5layer CNN model for the binary classification of malaria infected and non-infected red blood cells, the model reported an accuracy of 96.82% [12].

Data Set
3.1 Data Acquisition 462 field of views of Giemsa(dye)-stained and field(dye)-stained slides were acquired using an iPhone 10 mobile phone with a 12MP camera.The phone was attached to an Olympus microscope with 1000× objective lens.Half of the acquired images were red blood cells with a normal morphology and the other half had a rouleaux formation morphology.Out of the Rouleaux images, 51 images were acquired from Giemsa-stained slides at Murtala Muhammad Specialist hospital, Kano state, Nigeria while 180 images were acquired from field-stained slides at Isyaka Rabiu paediatrics hospital, Kano state, Nigeria.The 231 normal cell morphology images were acquired from Murtala Muhammad specialist hospital from slides stained with Giemsa.The slides used consists of cells of varying morphology such as crenated cells, microcytes, ovalocytes, normalocytes and sickle cells.The size of the images was in the range of 4032×3024 pixels.Figure 1 and Figure 2 depict original captured images of rouleaux cell morphology and normal cell morphology.

Data pre-processing
The image was separated into a binary (black and white) image using binary thresholding technique.The threshold that separates the foreground and background of the image was automatically set using the Otsu thresholding technique.The contours around the image were then cropped out.After cropping, the sizes of the images were reduced to 2500×2500, as depicted in Figure 3 and Figure 4, these were then segmented into smaller sized images of sizes 750×750 pixels as depicted in Figure 5 and Figure 6.

Depthwise Separable CNN architecture
The convolutional layers in the CNN architecture were replaced with depthwise separable convolutional layers.This layer performs a spatial convolution on each channel of its input, independently after which the output channels are mixed using pointwise or 1×1 convolution.Hence spatial features and channel-wise features are learnt separately thereby reducing the number of parameters and computations leading to an increased model speed and accuracy.

Network Training
The images were resized from 750×750 to 300×300 and 500×500 image sizes, they were normalized by scaling the data from the 0-255 range to the 0-1 range.The images pass through each convolution layer and convolution operation depicted in Equation 1 is performed on it, the output is passed through a non-linearity, the ReLu function depicted in Equation 2, the output of which is a feature map, the feature map is passed through the MaxPooling layer where the size of the map is reduced by half.After feature extraction in the convolution layers, the feature map is converted to a single vector and fed into dense layers where classification occurs.The final prediction is compared with the actual output and the difference between these two is the error or loss function, where for this work, a binary cross entropy loss function was used as depicted in Equation 3. The network is trained over many epochs to try and minimize the network loss function by updating the weights of the network and propagating it back through the network using stochastic gradient descent, as depicted in Equation 4, until a suitable result is achieved.The network was then validated on a new data without labels.

Results and Discussion
Each network was trained for 25 epochs with a batch size of 32.From Table 1 and Figure 8, the 300×300 input size separable CNN was the fastest network to train with a better accuracy and loss values for both training and validation (accuracy: 99.38%, loss: 0.0538, validation accuracy: 0.9391%, validation loss: 0.2884) while the separable CNN trained on 500×500 image size reported lesser values for both training and validation as depicted in Figure 9 (accuracy: 98.81%, loss: 0.0336, validation accuracy: 87.75%, validation loss: 0.5904).The CNN model trained on 300×300 input size reported evaluation metrics, depicted in Figure 10 (accuracy: 98.87%, loss: 0.0311, validation accuracy: 87.91%, validation loss: 0.8177) higher than those trained on input size of 500×500, depicted in Figure 11

Conclusion
This work developed a thin blood smear data set that contains 231 images of normal cell morphology and 231 images of rouleaux cell morphology with sizes of 4032×3024 pixels.Data pre-processing techniques were applied to crop and slice the images to give a dataset containing 6088 images of 750×750 pixels with equal instances of normal cell morphology and rouleaux cell morphology.Two custom made deep learning techniques were developed and evaluated for the binary classification of the dataset and evaluating the image size that gives the best performance.A 5 layer deep convolutional neural network was developed, and a separable convolutional neural network was developed by replacing the CNN layers of the first network with separable CNN layers while maintaining the same dense layers.Both networks were trained and validated for 25 epochs.Separable CNN model reported a higher train and validation accuracy than the CNN model while training the network on images of size 300×300 gave better performance in both CNN and separable CNN models than images of size 500×500 pixels.This work has shown the capability of CNN models in diagnosing haematological abnormalities in malaria infected red blood cells.Future work will focus on increasing the size of the dataset, using different network architectures and regularization techniques to improve model performance.

Figure 3 :
Figure 3: Image of rouleaux formation morphology cropped to a size of 2500x2500 using binary and Otsu thresholding.

Figure 4 :
Figure 4: Image of normal red blood cell morphology cropped to a size of 2500x2500 using binary and Otsu thresholding.

Figure 5 :
Figure 5: Images of rouleaux formation morphology after segmenting whole slide images of 2500x2500 pixels into smaller images of size 750x750 pixels.

4. CNN Model development 4 . 1
CNN ArchitectureA sequential convolutional neural network model architecture was developed for identifying images of normal cell morphology and rouleaux cell morphology.The model was developed using Keras and TensorFlow in python programming language.The model is made up of five convolutional layers, two fully connected layer and an output layer.The input or first convolutional layer has 16 filters of size 3×3 and a ReLu activation function.A MaxPooling layer with a window size of 2×2 was added to down sample the size of the feature map and a Batch Normalization layer was added followed by a dropout of 10%.The second, third, Fourth and fifth convolutional layers had the same arrangement of: convolutional layer, ReLu activation function, Batch normalization layer and a Dropout layer in a sequential manner.The second convolutional layer had 32 filters of size 3×3 and a 10% dropout rate.The third convolutional layer had 64 filters of size 3×3 and a 20% dropout rate.The fourth and fifth convolutional layers had 128 and 256 filters each of size 3×3 respectively, both with dropout rates of 20%.The output of the fifth convolutional layer is connected to a flatten layer.Two Dense layers are connected after the flatten layer.The first dense layer has 512 filters with a ReLu activation function, followed by a Batch normalization layer.The second dense layer has 256 filters activated by a ReLu function, followed by a Batch normalisation layer and a dropout of 50% added to it.The output of the second dense layer is connected to the output layer with a single neuron and activated by a sigmoid function.

Figure 7 :Figure 6 :
Figure 7: CNN Architecture depicting the arrangement of CNN and MaxPooling layers and flow of data from first CNN layer (input) to the last Dense layer (output)

Figure 8 :
Figure 8: Accuracy and loss metrics for separable CNN with 300x300 input size

Figure 9 :
Figure 9: Accuracy and loss metrics for separable CNN with 500x500 input size

Figure 10 :Figure 11 :
Figure 10: Accuracy and loss metrics for CNN with 300x300 input size

Table 1 :
(accuracy: 96.72%, loss: 0.1161, Val accuracy: 56.58%, val_loss:1.4090).From the results obtained, depthwise separable CNN architecture has better performance than CNN architecture while models with smaller input size (300×300) had better performance metrics than models trained on larger image size (500×500) in both depthwise separable CNN and CNN architecture.In both models the difference between train and validation accuracy is large, 8.43% for the best performing Separable CNN model and 10.89% for the best performing CNN model, this shows that the network has overfit the training data.Summary of Result

Table 2
depicts the results obtained from other works in the literature.The results presented are based on the binary classification of malaria infected red blood cells and non-infected red blood cells, the images used are single images of red blood cells segmented from whole slide images, while the images used in the proposed work are of size 750×750 and each image contains no less than 15 red blood cells.The depth of the CNN models reported in the literature are all deeper than the proposed work, hence their higher level of accuracies, while the work by Asma et al has the same number of CNN layers as the proposed work, but the dataset was larger, containing 27,578 images compared to the proposed work which has 6,088 images, furthermore the work by Asma et al further increased the size of the dataset by using image augmentation thus achieving a high level of accuracy with fewer CNN layers.No other work has been done on the binary classification of red blood cell morphology that involves normal cell morphology and rouleaux morphology and no other dataset has been created with such images, hence the proposed work cannot be compared on an equal basis with other works in the existing literature.

Table 2 :
Performance comparison of proposed method with other CNN architectures