Robust ball tracking scheme for soccer robot under various illuminations and occlusions

The ball tracking method for soccer robot usually done using simple color thresholding and shape detection. The method is designed solely to track specific type of ball determined by the soccer robot regulation. However, the method should be robust and not to be limited to the regulation. In this research, a robust ball tracking method for soccer robot that can afford to track various types of ball under various illuminations and occlusions is developed. The proposed method uses combination of Histogram of Oriented Gradients (HOG) and Linear Support Vector Machine (Linear-SVM) to detect the ball and Optical Flow to track the ball. In order to reduce the computation, the method only scans the whole scene when the ball is not detected. Once the ball is detected, the method only scans the area around previously detected ball. HOG itself has its own mechanism to handle various illuminations and SVM as machine learning method is tolerance to occlusion. The experiment shows that the proposed scheme is accurate and can perform real time.


Introduction
Robotics has been widely used not only in industrial but also in educational field. Various competitions are held every year to increase the interest for robotics. One of them is Indonesian Robotics Contest which held on national scale. There are some categories that are included in the competition. One of them is the new category, wheeled soccer robot that first held in 2017.
The wheeled soccer robot uses ball tracking and goal detection to perform actions such as dribbling and kicking the ball towards the opposing goal. The movement of the ball influences the calculation of motor rotation. The motor rotation will determine the movement of wheeled soccer robot while following the ball. In other words, the ball tracking method determines how quick the wheeled soccer robot can find and follow the direction of the ball.
Most ball tracking systems are limited to the soccer robot regulations, for example a regulation that set the ball color to be orange, the goal to be white, and so on. This regulation causes most of the ball tracking systems for wheeled soccer robot use color detection and shape detection to detect the ball. If the regulation changes, the method for ball tracking also need to be adjusted. The adjustment will take time to do research, experiment, and performance test in order to get the suitable method for the competition. In this research, authors want to keep the adjustment to minimum and save the time to do improvement.
This research proposes a robust ball tracking scheme for soccer robot under various illuminations and occlusions. In summary, the contribution of this work is given as follow:  The proposed scheme uses texture information to detect the ball instead of color because texture information will need a little adjustment to fit the regulation.  The ball detection uses machine learning method to handle the occlusion and only scans the whole scene if the ball is not tracked in the previous scene otherwise only scans the surrounding area of previously tracked ball. This scheme can localize the detection area and reduce the computation for real time application.  Finally, the ball is tracked using optical flow to estimate the ball direction and movement.
The rest of this paper is organized as Section 2 presents the related works, Section 3 presents the proposed ball tracking scheme, Section 4 presents the results and discussion, and the conclusion of this work is described in Section 5.

Related works
Ball tracking method for soccer robot has been developed by many researches. Most of them [1][2][3][4] use combination of color detection and shape detection. As in [1], the authors aim to implement a real time object detection and tracking for soccer robot. The method uses color based segmentation in HSV (Hue-Saturation-Value) color space to differentiate the ball and the goal. Hough transform is used to detect the goal and ellipse detection is used to detect the ball. The result shows that the method can perform well although there are some errors in the detection.
Color based detection method also used in [2]. The authors use omnidirectional camera which has 360 degrees of view in order to increase robot field of vision. The benefit using omnidirectional camera is the distance calculation become accurate. However, the computation become more complex and sensitive to the surrounding illumination.
Detection of ball color and ball position have been done by [3]. The method is implemented for goalkeeper humanoid soccer robot. Their research proposes a method to track ball position using HSL (Hue-Saturation-Lightness) color space, perform blob detection, and using nine cells coordinate model to define the ball position. The result shows that the system is able to detect the ball although the accuracy of ball detection needs to be improved, especially under different illumination.
The environment of the robot tournament is unpredictable where the ball color is not guaranteed to be the same every year. Most of the participants use color detection to detect the ball. The research of aerial ball detection has been done by [4] to predict the direction of ball while in the air. The method is implemented in goalkeeper robot for real-time application. It uses blob detection with Kinect sensor and color filtering using depth information to discard the color of the object that found farther than certain distance.
In this research, a robust ball tracking scheme for soccer robot is proposed. The ball tracking scheme uses combination of Histogram of Oriented Gradients (HOG) and Support Vector Machine (SVM) to detect the ball and Optical Flow to track the ball. In order to reduce the computation, the method only perform tracking on the whole scene if the ball is not detected otherwise only on surrounding area of the previously tracked ball. The proposed scheme uses ball texture information rather than color and the detector uses various types of ball in the training process. This scheme can track the ball with good accuracy and perform real time. Moreover, the proposed scheme will adapt the regulation easily.

Methodology
The general procedure of robust ball tracking for soccer robot under various illuminations and occlusions is shown in figure 1. The procedure of the proposed ball tracking scheme consists of three steps: training the ball detector, perform ball detection, and tracking the ball.

Training the ball detector
At the first step, before performing ball detection, the ball detector needs to be trained. The detector is trained using SVM from extracted HOG descriptors. HOG descriptors can be extracted using the following steps [5]:  Convert image to grayscale.  Normalize gamma from input image then calculate the gradient. SVM is a classifier that determined by separation which called hyperplane. Hyperplane can be calculated by maximizing margin or distance from two set of object from two different class. Classification with SVM consist of training and classification. In the training step [6], if there are given training data ∈ , = 1, … in two classes and label ∈ such that ∈ {1, −1}, it can be solved using equation (1).
where is weight vector, is bias, is slack variables, ( ) maps into higher dimensional space and C > 0 is the regularization parameter and the decision function is shown in equation (2).
where ( , ) is the kernel function. After training process, parameter ∀ , , label names, support vectors, and kernel parameter saved as output model from training SVM.
Optimal parameter is selected using k-fold cross validation which is a method to do cross validation by dividing training data into k set which has (k-1) as training data and the rest will be the test data. After training process, we will get variable w, x, and b for each class.

Ball detection
At the second step, the ball is detected using previously trained ball detector. HOG has own mechanism to handle various illumination by applying contrast normalization. SVM is a machine learning method that tolerance to occlusion if proper trained. In the classification process, voting strategy is performed for each data which will be designated to be in a class with the maximum votes [6]. The classification process can be done with the following steps.
 Calculate linear SVM kernel.  Calculate decision function with (2).  Repeat the first step and second step for other classes.  Determine the class by function which gives the most maximum result. Because the heavy computation of HOG and SVM, authors set a scheme for the detection as shown in figure 3. The method only detects the ball in the whole scene if the ball is not tracked from the previous iteration otherwise the method will detect the ball only in the surrounding area of previously tracked ball. This scheme can minimize the area of detection and reduce the computation time.

Ball tracking
At the final step, the detected ball is tracked using Optical Flow. Optical Flow assumes that the intensity from the same object is not changing from sequences of frame. The neighborhood pixel and object pixel will move with the same movement.
If a pixel in the first frame ( , , ) and moves by distance ( , ) in the next frame after time, we will have equation (3).
( , , ) = ( + , + , + ) According to Taylor series approximation of right-hand side then remove the common terms and divide by , we will have equation (4).
where = and = are the image gradients, is the gradient along time and , can be solved using Lucas-Kanade method [7] as shown in equation (5) Optical Flow will match the features from current and previous ball area to find the distance and direction of ball. The distance and direction are used to predict the movement of the ball.

Performance evaluation
The performance evaluation procedure uses confusion matrix then calculates the accuracy, specificity, sensitivity, and precision using equation (6), (7), (8), and (9) respectively where TP is true positive, FP is false positive, TN is true negative, and FN is false negative.

Result and discussion
The proposed ball tracking scheme runs in a simulation written in C++ and additional OpenCV library on Laptop with Intel i5 processor and 8GB of RAM. Authors analyze the fps generated in the result. The test data are two videos with ten seconds duration collected from soccer robot point of view. The test data contain change of illumination and occlusion. Authors use approximately 1,000 positive data and 20,000 negative data to train the ball detector.

Ball detection result
The proposed ball detection is tested under sudden change of illumination and occlusion. The performance evaluation of proposed ball detection is shown in table 1. From table 1, the accuracy and the precision of proposed ball detection is high which has average 92.42% and 98.13% respectively. The high accuracy and precision mean the proposed method can detect ball under sudden illumination change and occlusion well. The low sensitivity means that the method is less sensitive when detecting the ball. The large number of negative data when training the ball detector affect in less false positive but more false negative.  accepted because once the ball is tracked the robot can quickly follow the ball because the detection is real time. The red rectangles on figure 3 and figure 4 show the surrounding area of previously tracked ball.
The drawback using combination of HOG and linear SVM is the detector need to be large enough to produce better feature descriptors. The yellow rectangles on figure 3 and figure 4 show the size of the detector. Large size of detector means more computation to perform that is why author minimize the detection area by using the mentioned scheme. The range of detection also depend on the scale and pyramid level used in the detector.

Ball tracking result
The ball is tracked using optical flow. Figure 5 and figure 6 show the result of proposed ball tracking scheme on sequences of frame. From figure 5 and figure 6, blue rectangle is the detected ball, red rectangle is the surrounding area of previously tracked ball, yellow rectangle is the detector size, yellow snapshot with arrowed line is the distance and direction from robot's center to the tracked ball, green line is the ball trails, and the magenta text is the information including the direction of ball movement. Figure 5 and figure 6 show that the proposed method can tracked the ball and predict the movement. Optical flow will produce arrows for each matched features from previous and current detected ball that represent the direction and distance. The average of distance and angle are used to estimate the speed and direction. By using the ball trails which marked by green line, the proposed method can track the history of ball's movement if going out of scene.

Conclusion
In this research, authors proposed robust ball tracking scheme for soccer robot under various illuminations and occlusion. The proposed ball tracking scheme can perform well which proven by the high accuracy and precision from the result of performance evaluation. The proposed scheme also can handle sudden illumination change and occlusion in the sequences of frame. The fps is acceptable despite of the heavy computation of combination of HOG and SVM and with the proposed scheme, the fps can be further increase to real time if the ball is already tracked in the previous scene. By using this scheme, the adjustment for robot soccer competition is only adding more training data and size of the detector. For future work, the proposed scheme can be improve using more training data and applied in the real hardware of robot soccer.