Comparative analysis of multiple machine learning algorithms for epileptic seizure prediction

Epilepsy is a common neurological disease that affects more than 2 percent of the population globally. An imbalance in brain electrical activities causes unpredictable seizures, which eventually leads to epilepsy. Neurostimulators have the power to intervene in advance and avoid the occurrence of seizures. Its efficiency can be increased with the help of heuristics like advanced seizure prediction. Early identification of preictal state will help easy activation of neurostimulator on time. This research concentrates on the performance analysis of various machine learning algorithms on recorded EEG data. Through this study, we aim to find the best model, which can be used to create an ensemble model for better learning. This involves modeling and simulation of classical machine learning technique like Logistic regression, Naive Bayes model, K nearest neighbors Random Forest, and deep learning techniques like an Artificial neural network, Convolutional neural networks, Long short term memory, and Autoencoders. In this analysis, Random Forest and Long Short-Term Memory performed well among all models in terms of sensitivity and specificity.


Introduction
Over 50 million people around the world suffer because of epilepsy. Elliptical seizures create changes in behavior, muscle tone, or movements. This eventually leads to vigorous shaking, breathing problems, loss of consciousness, and so on. These seizures occur without any warning and thus leaving no time for patients to prepare. The chance of untimely death of people with epilepsy is nearly three times greater than healthy people. Patients suffering from epilepsy avoid many social activities of their life due to fear of stigma and discrimination. Even with the present advancement in the medical field, we could be able to control only 70% of seizure cases using medicines [1]. The unpredictable nature of this disease causes anxiety and sometimes even physical injuries in patients. With the accurate prediction of upcoming seizures, the anxiety in patients can be reduced. This also provides them with adequate time to put themselves in a AICECS 2021 Journal of Physics: Conference Series 2161 (2022) 012055 IOP Publishing doi: 10.1088/1742-6596/2161/1/012055 2 safe position to avoid any injury. Neurostimulators can intervene in advance to avoid the occurrence of seizures [2]. With a proper seizure prediction algorithm, one can easily activate neurostimulators before the seizure. This early prediction will increase the efficiency of the neurostimulator.
Nowadays, Machine learning techniques have been used extensively in many health care applications like anomaly detection in Phonocardiogram [3], Hepatitis disease prediction [4], Breast Cancer Diagnosis [5], and others. A seizure can be detected at the early stages with classical machine learning techniques and deep learning models. The primary method of seizure detection is by analyzing patterns in electroencephalogram (EEG) [6]. EEG data has application in a wide variety of problems like emotion detection [7], EEG-based brain control [8] and so on. The machine learning approach looks through the recordings of EEG and identify the brain activity patterns during seizures. The seizure prediction task makes use of classical machine learning methods like the random forest [9], SVM [10], and so on. But these classical machine learning algorithms are less efficient in high volume EEG data. Variability in data also affects the performance of algorithms. Availability of voluminous data and high-end processors lead to advancement in deep learning techniques. This eventually eased the challenges in seizure predictions.
In this research, we focus on the performance of various machine learning techniques (Logistic regression, Naive Bayes, Random Forest, and K-Nearest neighbor, Artificial neural network, Convolutional neural network, Long short term Memory technique and Auto Encoder) on EEG data to find the best machine learning algorithms that perform well on our datasets. From our analysis, we understand that among classical machine learning techniques, the Random Forest classifier has performed remarkably well in terms of specificity and sensitivity. Among deep learning algorithms, Long Short Term Memory (LSTM) is the best performing model.

Literature Survey
Epilepsy detection has been studied extensively since the 1970s. Early, statistical methods and classical machine learning techniques were used for Epilepsy classification. But due to the data complexity, the researchers started applying deep learning to this problem. Since 2016, ample research has been done on it to identify epilepsy using deep learning models. Many deep learning models such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Deep Belief Networks (DBN), Autoencoders (AE), Long short-term memory (LSTM), and generative adversarial networks (GAN) have been used [6]. Feature extraction is a key step in the epilepsy detection technique. Various techniques have been used to get relevant features from the EEG. This helps in getting better accuracy and faster computation. For feature extraction, many techniques have been used in recent years which include discrete wavelet transform (DWT) [7] techniques based on Huffman coding [8], Hilbert Transform [9], spectral analysis, and Wavelet Transform [10]. By the 20th century, deep learning models took off. Artificial neural networks along with the sliding windows are utilized for pattern recognition [11]. This was done on the Bonn data set [12]. An accuracy of 93.7% accuracy in detecting normal signals and 95.06% accuracy in detecting seizure signals were observed. Later many new approaches were proposed that make use of more deep learning algorithms. One such approach was to use Convolutional Neural Networks for interictal epileptiform discharge [13]. This is a common symptom of partial epilepsy. This is the first paper that put forward the idea of using automatic feature extraction techniques. But they were focused on the detection of interictal epileptiform discharge rather than predicting seizure. More complex neural networks like Recurrent neural networks were introduced after that. The time-series nature of EEG signals . Another approach makes use of a residual network for epilepsy seizure prediction [15]. This model uses ResNet-50 for classifying EEG into 3 classes (ictal, neonatal, and pre-ictal). This classification technique gives an accuracy of 94:98%, but the main drawback was, this model only takes 2D EEG images as input. So, one had to convert 1D EEG data to two 2D EEGs. This incorporated additional computational cost and complexity in the model. Various comparative studies were held to identify the best model. One such research tried to make a comparative analysis of different networks available for seizure detection [16]. This paper covers a comparative study of three different deep learning models i.e., CNN, RNN, and CNN. Training and the test data set in this paper are obtained from totally separate groups of mice. Later, these classifiers were applied to human encephalogram recordings to validate the results. They concluded that CNN is most suitable when using raw EEG waveforms images for automated seizure detection. In 2019 IBM come forward with SeizureNet [17]. It was able to classify seizures at 98.4% accuracy. SeizureNet's architecture is divided into two sub-networks i.e., a deep convolutional network and a classification network. The dataset used in this paper is TUH EEG Seizure Corpus (v1.4.0).

Proposed model
Through this study, we are mainly focusing on performing a comparative analysis of different machine learning algorithms including deep learning techniques. To satisfy our requirement, we have created 8 models using different techniques and trained & tested them on two different EEG datasets-the Bonn dataset [11], and the CHB-MIT dataset [12]. EEG signals are highly unpredictable data with a massive amount of variability on it. It shows different potential at different frequencies, so the best way to analyze an EEG signal is to convert it from the timeamplitude domain into the frequency-time domain. For conversion and feature extraction we have used Wavelet transform (WT). Wavelet transforms outputs both time and frequency of a signal as a result we can use these extracted signals for studying underlying hidden correlations between features. All these steps are executed with the help of mne package available in python. mne is a python tool kit that is exclusively dedicated to medical data analysis. It supports functions like wavelet transforms, down sampling, filtering, etc. To start with, we have used functions like mne. preprocessing and mne. filter for preprocessing our EEG signals. With the help of mne.time-frequency sub-module we have created a time-frequency representation of data. Later with the mne features. feature extraction library, we have extracted prominent features from the data.
Our proposed method includes usage of 8 models, where four of them make use of classical machine learning algorithms like Logistic regression, Naive Bayes, Random Forest, KNN, while the remaining four uses deep learning techniques like an artificial neural network, Convolutional neural network, Long short term memory (LSTM) and Autoencoder. An overview of our study is in figure 1 shown below.

Classical machine learning models
We have used four supervised machine learning algorithms for the classification task.

Logistic Regression.
Logistic regression classifies the data by learning weights(w) and bias term(b) from the training data. Each input feature (x) is associated with a weight, which is a real number. The weight defines how important an input feature is for the classification decision. After learning the weights and bias from the training data we make use of equation

Random Forest.
A random forest is an ensemble of decision trees, which are machine learning algorithms able to perform both classification and regression. This is a part of decision algorithms which make use of tree representations.

Deep learning models
Deep learning techniques, make use of neural network algorithms that work on the principles of forwarding propagation and backpropagation [13]. Initially, information is passed into the input layer, it is then transferred into the hidden layers. Interconnections between input layers and hidden layers assign a weight to each input. This weight is multiplied with every input. After that, a bias term is added to this. This is then passed to an activation function. Activation determines the limit of the output range, and it decides which features should be selected for appropriate output. Outputs from hidden layers are aggregated in the output layer and it is passed to an activation function to produce output. Later this output is tuned with the help of backpropagation. Neural networks can be again divided into multiple classes based on the methodology they adopt for solving applications.

Artificial Neural Network.
One of the models we had used for our comparative study is a basic artificial neural network that makes use of 12 fully connected layers. This model is a multi-class classification model that works on ReLu and SoftMax activation functions. This model is tested on two datasets to study the performance of the model on different amounts of data.

Long term short Memory (LSTM). Long-term short memory (LSTM) is an upgraded version
of RNN with memory elements. RNN suffers with long term dependency problem, this is overcoming in LSTM with the help of cell state. As a result, LSTM is suitable for time series analysis of larger datasets. O u r network architecture makes use of two LSTM layers and two fully connected layers. A dropout of 0.3 is induced in the network to avoid overfitting. We have used tanh, sigmoid, and SoftMax activations in our model.

Autoencoder.
Autoencoder is an unsupervised machine learning algorithm that works based on an encoder-decoder circuit [15]. Autoencoder is commonly used for representation learning and it works on the principle of dimensionality reduction. We have implemented a denoising autoencoder by adding noise into our original data, then we have induced a gaussian dropout of 0.3 in our model to avoid overfitting and to induce faster computation. The proposed model makes use of 6 fully connected layers that uses tanh as the activation. Mean Square Error is used as a metric for calculating the efficiency of the model.

Bonn Dataset (primary dataset)
Bonn university dataset [11] contains EEG signals sampled at 173.61 Hz. It is recorded using a 128-channel amplifier system It is mainly used for seizure event detection. EEG recordings were done using standard electrode placement which contains 10-20 electrodes. This dataset contains five sets of data from 100 channels. This is named from A TO E.

CHB-MIT
The CHB-MIT dataset is a publicly available epilepsy data set. It contains 686 scalp encephalogram recordings of 23 patients treated at the Boston Children's Hospital with intractable seizures. The dataset contains 2 recordings of each patient 1.5 years apart. This data set contains 969 hours of EEG recordings with 173 seizures. Subjects include males, females, and children aged between 10-22.

Experiment Results
We have created 8 machine learning models which include 4 classical machine learning techniques and 4 deep learning models. We have tested all these models on 2 different datasets which include the Bonn dataset and CHB-MIT for early prediction of epileptic seizures. To find the best model, we have performed 3 types of analysis. In the first phase, we had chosen the Bonn dataset as our primary dataset and applied all 8 algorithms on it to find the best performing classical machine learning algorithm and deep learning algorithm, later to validate the results we have applied the same models on 1 more dataset and made a comparative study to analyze which algorithm is performing better among all models. In the third phase, we had made a comparative analysis of the best performing classical machine learning and deep learning models. To frame our deep learning models, we have used Keras from the tensor flow. It incorporates data generators as root class. For data preprocessing and data extraction, we have used mne package. It has specialized modules for analyzing medical data. Another package we have used is scikit learn. It offers many functionalities like data splitting, in-built modules for computation, etc. Then we had made use of basic python packages like NumPy, pandas, matplot library and so on for analyzing our data sets. We had used precision, recall, sensitivity and specificity, and accuracy as evaluating metrics for our models.

Prediction on Bonn dataset
We had considered the Bonn data set as our primary dataset. Our first aim was to find the best classical machine learning algorithm for EEG data. Results obtained for 4 classical models are shown in table 1. For a better understanding of performance, we have used the ROC curve (figure 2) to analyze our models.

Deep Learning Algorithms.
We have developed 4 neutral networks for analyzing the Bonn dataset, for a better understanding of model performance, we have created a graph that showcases changes in loss and accuracy with respect to variation of epochs.

Artificial Neural Network
The graph in figure 3 shows the variation in accuracy with respect to increase in epochs. From graph, we can understand that accuracy for training data set is increasing constantly, while the accuracy of validation set is initially increasing and later it is remaining constant. This is due to overfitting.     Loss curve of LSTM model in figure 8 showcase a decrease in loss with respect to an increase in epochs. From that, we can conclude LSTM model is performing well on datasets.  Results obtained from the above 4 deep learning models are presented in table 2. From values we can understand both Convolutional Neural Network and Long Short Term was equally good for analyzing our Bonn EGG data.

Comparison of Random Forest with LSTM
From the above results can conclude Random Forest is the best classical machine learning to analyze EEG data. Similarly, Long Short-Term Memory (LSTM) is the best deep learning algorithm for EEG datasets. Random Forest is less computationally expensive when compare with the Long Short-Term Memory (LSTM) model. Long Short-Term Memory (LSTM) model performance can be effectively increased with the help of GPU acceleration. Random Forest supports variability of data, while LSTM is more suitable for time-series data. Long Short-Term Memory (LSTM) model has a provision for avoiding long dependencies this makes our predictions more accurate when compared with other models. The main problem with Long Short-Term Memory (LSTM) model is, it requires a large amount of data for training purposes.

Conclusion
We have conducted a comparative study of different machine learning techniques to identify the best machine learning algorithm identifying best performing epileptic seizures prediction method. From our analysis of 4 classical machine algorithms -Logistic Regression, Random Forest, and KNN we concluded that Random Forest is the most cost-effective method to analyze EEG data in case of limited training data. Random Forest supports variability in data. But is Random Forest is not sufficient for analyzing big data, In that case, we have to use neural networks. From our analysis, we had concluded that LSTM is the best deep neural network among Artificial Neural Network, Convolutional Neural Network, and Autoencoders. LSTM has a memory cell, and it avoids long-term dependency problems. LSTM is the best model for seizure prediction of big volume time-series big data. It can achieve high sensitivity and specificity when compared with other models. However, there is still an opportunity to get better in numerous perspectives. In the future, we can incorporate ensemble learning in seizure prediction by combining LSTM with other efficient models. Other than that future research can be done to reduce the number of parameters learned in Deep Learning Models.