The task adaptation method for determining the optical flow problem of interactive objects recognition in real time

In this paper, we solved the problem of determining the optical flow as a stationary image and a real-time video stream. Unlike conventional methods of calculation, the problem of the module integrated vision determines the optical flow in Unity3D interactive environment to obtain real time data. The carried out testing and approbation of data presented an image processing algorithm, an experiment, the results obtained by the optical flow.


Introduction
The obtained image can be captured in real time; thus, it makes it possible to obtain useful exhaustive information: the movement of objects in the scene and the camera motion parameters. Light Change (brightness) of the image was observed in the movement of the camera relative to the depicted objects called an optical flow [14].
The optical flow provides information even if there is deformation of the image of the observed surface. The motion of objects in front of the camera during shooting or camera movement relatively the fixed stage leads to corresponding changes in the image. These changes can be used to restore the relative motion in the image frames on the basis of changes in time. Ideally, the optical flow corresponds to the field of motion (vector field), with the proviso that when shooting, illumination of objects does not change [13].
The relevance of the work consists in adapting the method for determining the optical flow to the problem of recognition of objects in the interactive mode in real time.
In this paper, to accelerate computation and optimization of image processing, we used the EmguCV library. The program is written on the basis of the C # language in the Unity3D environment.
In general, the term "Emgu CV" means a cross-platform "shell" for the OpenCV image processing libraries (the main library for working with technical vision). Emgu CV is also called a machine vision library.
EmguCV supports a variety of programming languages: C #, Visual, C ++, IronPython and several other languages. It is used to solve various problems associated with 2D and 3D graphics, identifying persons and objects in the photo and video.
Project Emgu CV can be compiled in Mono Development in the Unity3D medium, provided that work can be adapted under Linux, Windows, Mac OS X and popular mobile platforms, such as Android, iOS. OpenCV, developed by Intel, is a library of programming functions. It uses a real-time computer vision. Developers are free to use the library in accordance with the license, an open-source. It is written in optimized C and C ++ languages and contains more than 500 functions that cover different areas of computer vision. Moreover, the library may use multicore processors to retain computational efficiency.

Methods
Sequences of images in the stream allow estimating the velocity or movement of the image changes, such as a discrete shift of these images. Methods are based on the optical flow calculated motion between two frames, taken at two points in the previous time frame (T1) and the subsequent frame for them (T2), for each pixel [28].
Studies of the optical flow is widely carried out in various fields of image processing, including video compression and motion analysis. Optical Flow algorithms determine not only the motion of the vector field, but also the use of the optical flow in the analysis of 3D-scenes, as well as the movement of objects and the observer relative to the scene [26] The problems of determining the optical flow are used in robotics when recognizing objects, object tracking, determining movement and robot navigation [23,25]. In addition, the optical flow is used to study the structure of objects [7], since the definition of the movement and the creation of the scene surrounding the depths of the environment are applied in the areas of simulation of human vision.
In this problem, the formation of the optical flow consists in determining the magnitude and direction of the displacement of individual image points in two frames selected from the resulting video sequence from a connected camera in real time [1,2].
The following actions are performed in the beginning: 1. Determine the presence of the camera on the computer, when the camera is not found, an error message appears.
2. If the camera is found: 2.1. Sets story frames (array retention) 2.2. Runs a frame capture process (process frame) 2.3. Capture a frame and display it on the screen (Figure 2  The implemented algorithm used the C# programming language based on Unity3D v.5.3.5 environment. (64 bit). For the calculations, a PC was used with the following specifications: Intel Core i5 3.20 GHz, RAM 8 GB, OS Windows 7 Pro 64bit. The test was carried out on the Sven IC-990HD camera with 1920x1080 resolution and a frame rate of up to 30fps / sec. To speed up the computation, the original image was converted to black and white. As a result of the work, the following drawings were obtained (Figure 2, 3, 4).

Conclusion
In this article, the problem of determining the motion parameters for the optical flow based on the C # language was considered. Unlike the traditional methods of calculation, the problem of the module integrated machine vision was used interactively to determine the optical flow in the Unity3D environment to obtain real-time data. The carried out testing and approbation of data, resulting in the advantage of working with this method, significantly increase the speed of image processing by an average of 4.86% in contrast to conventional methods. During the experiment, it was found that the accuracy of a sketch of the contours of objects depends on the illumination of the object, the better the lighting, the better the image quality.
For any movement relative to the camera and the object, we can determine the appropriate optical flow. The optical flow is dependent on many parameters, such as the brightness of the subject, and the camera features. The results of this work can be implemented in the design of robotic systems, used in various technical fields of application, including the analysis of images obtained during astronomical studies.