Review of automatic detection of pig behaviours by using image analysis

Automatic detection of lying, moving, feeding, drinking, and aggressive behaviours of pigs by means of image analysis can save observation input by staff. It would help staff make early detection of diseases or injuries of pigs during breeding and improve management efficiency of swine industry. This study describes the progress of pig behaviour detection based on image analysis and advancement in image segmentation of pig body, segmentation of pig adhesion and extraction of pig behaviour characteristic parameters. Challenges for achieving automatic detection of pig behaviours were summarized.


Introduction
China is a country with mass production and large amount of consumption of pork. The livestock and poultry production was 84.54 million tons in 2015, in which, the pork production was 54.87 million tons, accounting for 64.9% of the meat production. The swine production plays an important role in national economic development. At present, the swine production is developing rapidly from extensive breeding mode of small scale and mixed farming to large scale and intensive breeding mode. The appropriate scale farming regarding the family as the unit with the slaughter of 300 to 3000 pigs per year has become the main part of pork supply in China. The development of large-scale farming has brought new challenges to breeding managers. On the one hand, as the farming density increases, the influence of the piggery environment on the pork production has increasingly become prominent. The temperature, humidity, lighting and concentration of harmful gases and other environmental factors in the piggery will affect the growth, development, reproduction and final production performance of pigs [1]. Increased number of pigs leads to a shorter time of observation for each pig from managers. Abnormal behaviours of pigs, which could be found during the inspection tour on site previously, are becoming more and more difficult to be distinguished. Therefore, automatic detection of pig behaviours is becoming more and more important and has become an important problem in the breeding of pigs.
The existing livestock breeding environment monitoring and control systems are mostly based on data collected by sensors [2]. However, because the following factors are not considered, such as radiation, floor type, growth stage, physical condition, nutritional and health status and other comprehensive factors, the regulation result does not meet the actual needs of pigs for calories, and cannot achieve optimal regulation of the breeding environment. The indicator of the best breeding environment comfort is the pig itself. Pigs integrate environment factors, body conditions and other Through the analysis of the behaviour of pigs by image processing, the behaviour information of pigs can be recorded in a non-invasive way without disturbing their normal activities. This study analysed the research progress and application limitations of pig image segmentation, separation of pig adhesion, extraction of pig herd behaviour feature parameters and other image processing technologies and summarized the existing problems.

Pig image segmentation
The image segmentation and splitting of adhesive pigs' images in the piggery environment is the premise and basis of pig behaviour analysis. The ability identifying the pig contour correctly can directly affect the accuracy of the pig herd behaviour analysis. Image segmentation and extraction of ROI (Region of Interest) have been the difficulty of image processing technology. Fluctuation of lighting condition, uneven distribution of light, and low contrast of ROI and background in the piggery increase the difficulty of pig image segmentation [3]. Ji Bin and others [4] used ALLA (adaptive local lightness adjusting) method to carry out nonlinear brightness adjustment for bright and dark regions of images in different piggery scenes to reduce the influence of lighting conditions on image segmentation. This method achieved better performance than that by the global and local histogram equalization method without affecting image quality. The subsequent image segmentation of the research has not been reported. 16 images were used. The effectiveness of this method in image segmentation needs to be verified. Guo and others [5] proposed a multi-object extraction method from topview group-housed pig images based on adaptive partitioning and multilevel thresholding segmentation. Firstly, the maximum entropy threshold segmentation algorithm was used to carry out the first segmentation of the whole image. The effective region according to segmentation results was extracted. Secondly, after morphological processing, the image was divide into sub images. Lastly, each sub image was segmented respectively for the second time. The correct detection rate was 92.5%, but this study did not discuss the separation of adhered pig bodies. The main problems faced by pig segmentation in the video include sudden changes in lighting condition in piggery caused by natural light changes or turning on/off lights, long resting time of foreground object pigs and dynamic changes in the video background caused by straw, water and excretion on the ground in the piggery [6]. In order to solve these problems, Tu and others [6] put forward a foreground detection method for overlooking of group-housed pigs based on cyclic reliability propagation algorithm. Compared with the common algorithm of foreground detection-Gaussian mixture model (GMM) algorithm, the results showed that the effect of the segmentation method was better, but the computational complexity was high. The computational efficiency needs to be improved. Guo and others [7] improved the Gaussian mixture model algorithm and proposed a prediction Gaussian mixture model algorithm, which integrated the processed results with those of maximum entropy threshold segmentation algorithm based on foreground color information and solved the empty problem of foreground objects. 92% of the foreground segmentation rate was achieved. However, this method cannot segment pig body contour accurately when the pig is stable under dark environment.

Segmentation of adhesive pig images
Segmentation of adhesive pig images is essential to the next extraction of pig herd behaviour characteristic parameters. The adhesion segmentation methods mainly include ellipse fitting, watershed transformation and concave point analysis. Kashiha [8] , Nasirahmadi and others [9] used the least square method to conduct ellipse fitting for pigs, and then separated the adhered pig bodies according to the length of major and minor axis and the position of the centre point, but they did not evaluate the performance of ellipse fitting method when several pigs are adhered. Lu and others [10]  Firstly, the contour line of adhered part was extracted. Then, the contour line according to the concave point was segmented. The ellipse fitting for the contour lines was carry out after segmentation. Finally, five ellipse screening rules were developed. The ellipse which did not comply with the rules were integrated. The recognition accuracy of this method for pigs was over 86%. However, the severe adhesion was not analysed in this study.

Extraction of pig behavioural feature parameters
The fluctuation in piggery environment conditions can cause behavioural changes in rest, excretion, drinking water, walking and others of pig herds, which can be shown through three behaviour characteristics, as follows: closeness, activity area and activity level of pig herd.

Resting behaviour
The piggery temperature will directly affect the resting behaviour of pigs. In a comfortable temperature range, the pig mainly lies on its side. When the temperature in the piggery is too high, in order to dissipate heat, the pig will change supine position and stretch its legs, while avoiding contact with other pigs. In the piggery, herds are generally scattered. When the temperature in the piggery is lower, pigs will use the sternum lying posture, and huddled together in order to reduce the surface area of exposed skin, thus reducing heat loss [11]. Wouters and others [12] used the image analysis method to extract the temperature regulating behaviour characteristic parameters of piglets and generated parameters for temperature control in the piggery. Through opening operation, thresholding, statistical number of pixels and other image processing operations, two parameters for temperature control were extracted. One of them is the instant space occupation index of the pig body area in the resting zone. The other one is the proportion of pig pixel in the specified window outside the designated resting zone. The method measured the comfort of animals by means of their reaction to the environment. For the first time, environment control based on pig's behavioural change was realized. Shao and others [13] proposed a method of evaluation and control of temperature comfort in the piggery based on computer vision. The invariant moment of image, background foreground pixel conversion frequency, occupation ratio of pigs, closeness of pig herds and other characteristics vectors were extracted by image analysis. The minimum Euclidean distance was used to distinguish whether pigs are in cold or comfortable state. Blob analysis was used to identify whether the pig herds are in warm or hot state.
The experimental results showed that the system could effectively detect the movement of pigs and correctly divided the behaviour of pig herds into cold, comfortable or hot state. Nasirahmadi and others [9] explored the method for analyzing the change in the rest behaviour mode of pig herd by means of the machine vision. By installing the camera on top of the piggery to acquire the top view images of pig herd, the least square ellipse fitting method was used to segment pig images. The centre of fitted eclipse was extracted. The unique Delaunay triangulation of the ellipse center point set was calculated. The result can reflect position and distance between pigs. The average value of triangulation perimeter (MVP) can be used for representing the closeness of pig herd distribution. The analysis results after 15-day experiment showed that the closeness of distribution of pig herds had a significant correlation with the ambient temperature. However, the closeness index MVP of this research was related to the camera type, installation height, angle and other conditions, which did not have universal applicability. This study did not investigate the relationship between the distribution model of pig herds and the comfort of piggery environment.

Defecation and drinking behaviour
The defecation habit of the pig is the best among livestock. Under normal circumstances, pigs have strong regional sense, even in a very limited area, they still set aside resting area and dunging area, and try to keep the dunging area away from the feeding area and resting area in order to keep them dry [14]. The research shows that when the temperature rises, pigs prefer to stay in dunging area or resting area. Pigs do not have sweat glands, and they cannot regulate their own temperature by sweating. When the ambient temperature is above 20℃, pigs accelerate their respiratory rate to promote water in the body to evaporate and release excess heat. At the same time, with the consumption of the water in the body, the pigs must drink a lot of water to maintain water balance in the body. The pig herd would spend more time to drink and play in water. Therefore, if pigs frequently visit the drinking and dunging area, it indicates that the current ambient temperature is too high in the piggery. Kashiha and others [8] marked different symbols on the backs of pigs They achieved the automatic identification of individual pigs by calculating the Euclidean distance of Fourier descriptor. The recognition accuracy reached 88.7%. While the piggery was divided into zones of feeding, drinking, resting and defecation, and the time of visiting various zones by pigs was automatically recorded, which provided a convenient way for exploring the behaviour of pigs. By using the image segmentation method and analysis technology of machine learning, Nilsson and others [15] calculated the proportion of pigs in the dunging area and resting area to indicate the regulatory status of the livestock breeding environment. Normally, pigs do not rest in the dunging area, the increased number of pigs in the dunging area means the failure of livestock breeding environment control. Zhu and others [16] developed the embedded video monitoring equipment to monitor the defecation of group-housed pigs for 24 hours. Statistical analysis was conducted on the number of times when pigs visiting dunging area within a certain time interval to detect the abnormal behaviour of pigs. The correct recognition rate of sick pigs reached 78%. Duan and others [17] proposed an optimized compressive sensing tracking algorithm, which could get accurate and stable tracking result for pigs under the conditions of complex background, dramatic changes in lighting conditions.

Activity level
The activity level can indicate health of pig herd. Costa and others [18] extracted the activity level and occupation indices of pig herd by means of image processing software Eyenamic. The relationship between these motion feature indices and environmental parameters were investigated. The activity level and occupation index could be used as an indicator of physiological state of animals and could reflect the piggery environment, e.g. changes in temperature, relative humidity and air flow. In view of the responsive behaviour of animals in accordance with changes in the microenvironment, Bloemen and others [19] provided a quantitative method based on machine vision, which could measure the animal activity index simply and accurately. Ott and others [20] recorded the activity level of pigs by video analysis and compared it with the results of manual recording. The correlation coefficient of video analysis with manual observation results reached 0.92, which proved the effectiveness of video analysis when calculating the activity level of pig herds. The experiment used self-adaptive histogram equalization method for the image pre-processing. The background subtraction method was adopted to calculate the activity level of pigs.

Abnormal behaviour
The intensive livestock breeding environment easily causes negative stimulus to pigs for a long time. Pigs often have abnormal behaviours, for example, biting tail, biting ear, fighting, mounting and other attacks. The abnormal behaviours seriously affect the health and production performance of pigs. Viazzi and others [21] achieved automatic continuous monitoring of aggressive behaviours among pigs by using the image processing method. Viazzi and others got the information about the activities of pigs by means of the motion history image (MHI) method and extracted the average motion intensity and occupation index from the motion information. Linear discriminant analysis (LDA) was implemented by using the mean activity and occupation index to detect aggressive behaviours among pigs. The analysis results showed that the accuracy of the method for the determination of aggressive behaviours reached 89%. Nasirahmadi and others [22] used the ellipse fitting method to locate pigs, considering the major axis of the fitting ellipse as the head or tail, the minor axis as the side of the pig, by calculating the Euclidean distance of the head, tail and side of the pig. The mounting behaviour was detected. The recognition rate reached 94.5%. Disease can lead to abnormal behaviour of pigs. Tachypnea is a common symptom of disease. Pigs often show sluggish action and tachypnea. Ji Bin and others [23]  image pre-processing, background subtraction and other image processing methods. The recognition precision of fluctuating ridge-abdomen was higher than 85%, which indicated the well potential of this method for early-warning of tachypnea among pigs.

Conclusion
Recently, many studies have been focused on pig image segmentation, segmentation of adhesive pig images and extraction of herd behaviour feature parameters in piggery. New progresses have made, but there are still some problems need to be solved, mainly in the following aspects: (1) Lighting condition, ground debris, and colour of pig body affect the result of image segmentation greatly. A robust pig body segmentation method is desired. (2) The calculation efficiency of pig adhesion segmentation is low and its accuracy is not satisfied. The segmentation performance of crowded pig adhesion is not ideal. (3) The pig behaviour is complex. Behaviours with potential diagnostic value need to be reviewed. Specific feature parameters need to be studied according to the detection requirements of the particular behaviour. Currently, the threshold for feature extraction needs to be set manually. The self-adaptive extraction method has not been studied yet.