Augmented Reality Techniques in Industrial Warehouse Logistics in Mining Industry

Visualization techniques for augmented reality (AR) and mixed reality (MR) are developed well enough to display any data, if need be. This paper deals with a new data visualization technique for efficient product storage and use. The solution proposed herein applies AR visualization. The paper describes an application that displays all necessary data on the screen using AR-markers, and has an option of making them markers complicated and adding three-dimensional models of an object.


Introduction
3D computer graphics and constantly advancing hardware and software for augmented reality (AR) and mixed reality (MR) (i.e., GoogleGlass [1], MicrosoftHololens [2]) make researches in various fields pay more and more attention to them. These technologies can be applied for different purposes, since they create specific visualizations in a certain area and enhance objects of a real-world environment with computer-generated images on the screen.
At the majority of mining and ore-dressing enterprises there is a problem of organizing logistics of a quarry warehouse for storing raw materials and / or finished products. For effective storage of a large volume and a wide range of products of mining and processing enterprises, quick access to information about the product, its condition and quantity is required. This article presents a new method of data visualization for the quality use of raw materials and storage of finished products. No instant access to information about condition of materials makes an obstacle to their efficient use. This paper presents a new method of data visualization for efficient use of raw materials and storage of finished products. The solution proposed in this article uses augmented reality visualization techniques.
Augmented reality (AR) is a subset of mixed reality (MR), that expands our physical world, adding layers of digital information onto it.
Human-centric AR applications have been studied in many research projects. Nowadays, we can find training applications for employees of industrial automation industry or applications for car driving that use AR and Haptic technologies [3].
Augmented reality provides a whole range of options for control and management of production processes. Visualization can help to improve understanding of the process and, as a result, performance of the employees.

Augmented reality in industrial systems
Due to easily accessible real-time information and control over many parameters through computer interfaces, automation ensures simultaneous control of several machines, thus, reducing the number of personnel. However, many critical operations cannot be fully automated, and operators may need to monitor numerous rapidly changing parameters at a time, simultaneously checking and interactively controlling components of the current process.
Augmented reality (AR) is a computer technology that visually alters real environment by using virtual elements [4][5][6]. Key features of AR: • There is a tracking system to track the view and location of the user; • Virtual objects are displayed in the real-world field of view; • Virtual objects displayed in the real-world environment have a correct position, orientation and scale in each frame; • It is possible to display both virtual and real objects in real time.
Most production processes require that an operator should monitor them behind a protective glass and simultaneously operate the computer to control the process and receive automation feedback. Such workflow poses a problem, since the operator's attention is split between monitoring of a computer screen to track process parameters and visual control of the process itself. Placing displays next to equipment can solve this problem to a certain extent, but, essentially, it separates the process from its data [7].
In order to effectively combine computer graphics with a real environment, AR should be trouble-free and use no additional tools, which require complex tracking [4]. A good modern example of it is Oculus Rift [7]. Recently, video viewing systems have grown in popularity thanks to such software libraries and products, as ARToolkit [8] (cross-platform SDK for developing augmented reality applications) and more advanced Vuforia [9], that make it possible to build and develop prototypes quickly and easily. Spatial AR systems have long been studied, i.e., the ASTOR system [10]. Nowadays, most AR systems have turned into marker applications, which means, that they use AR markers to display virtual information and track real objects.

AR Markers
A QR code can encode up to 7089 digital and 4296 alphanumeric characters, and can be read by a simple application for a smart-phone or personal computer with a camera, thus, giving a great advantage over other 2D barcodes that require special scanners for decoding. A QR code can be generated by numerous online tools or the Python QR Code library [11].
Other, much more complex markers are those available in the Vuforia library [9]. Vuforia has several types of markers, mainly, frame markers, image targets and Vumarks [12].
The most common marker is ImageTarget. This marker uses images pre-processed on the Vuforia website to identify special features of an image. On the Vuforia website, you can make your own list of images and create a filter that prohibits recognition of certain images [13].
Depending on its tracking performance, each image uploaded into Vuforia is scored from 0 to 5, where 0 is the minimum ability to detect this image, 5 is the maximum.
Another type of markers is Multi-Target. This marker is composite, since it contains several ImageTargets placed side-by-side [13]. Location of each marker in Multi-Target is set in a coordinate system, originating in the volume center of Multi-Target.
All components of Multi-Target can be tracked simultaneously, since, knowing location of one ImageTarget and relative position of other ImageTargets, we can calculate location of all markers in this group. Therefore, if at least one marker is detected, the screen displays all objects related to markers of this group.
When it is necessary to detect and track images rolled into cylindrical and conical shapes, cylindrical marks can be used [15]. Vuforia app can recognize words on a guide word list, uppercase/lowercase Latin letters, hyphens, apostrophes and spaces. SDK provides access to a list of 100,000 most common English words. It supports most fonts; and the text can be either in italic or bold.
Another feature of Vuforia app is recognition of 3D objects, which allows detecting and tracking 3D real-world objects, including complex ones, with intricate non-primitive geometric shapes [16].
Object Targets are a digital representation of the features and geometry of a physical object. An Object Target is created by scanning a physical object using the Vuforia Object Scanner. [17].
VuMarks are advanced ImageTargets built in vector graphics designer tools. Vumark can encode unique identifiers, strings and bytes, thus, being more flexible than a QR code. A disadvantage of Vumark is that it encodes 100 characters maximum, which is much less than QR code.
As soon as the Vuforia library detects the marker, it decodes the ID inside the marker processing the same frame in which the marker was detected. High-quality cameras enable detection under very unfavourable conditions, for example, a distance of a couple of meters from the marker or a high angle in regard to the marker -30 and even fewer degrees). That means that decoding is instantaneous and this fact sets Vumarks as the most superior AR markers as long as the code is of smaller length (ASCII 100 or fewer characters, 1000 numbers or 100 bytes).
Various auxiliary visual effects, for instance, using charts next to data, can greatly reduce the time required to monitor important data and its changes. This approach provides additional visual feedback.

Questionnaire for AR application
In order to ensure efficient organization of warehouse logistics and implement the method proposed in this paper, it is necessary to keep in mind characteristics of production. We selected some criteria that can be considered crucial for choosing implementation methods and the structure of the AR application: • Way of storage: objects to be safely stored can be kept either in special boxes accommodating several pieces or in individual packages. It affects the number of markers (codes) to be created; • Access priority of warehouse employees to the corporate database; • Confidentiality regime: whether the information provided by the application is confidential or can be disclosed in the customer company. It affects the way of storing markers and database files; • Number of markers to be read simultaneously.

Ways to store markers and database files Vuforia Web Services API
The Vuforia Web Service API is a cloud-based recognition system designed to identify objects stored in the user's cloud database. Developers can add up to 1 MB of base 64-encoded content as metadata attached to the target's search result. Examples of this content could be images or mobileoptimized 3D models.
For managing cloud DB targets using the Vuforia Web Service API and performing an Image Recognition Query, you will need to use the server access keys associated with your cloud DB [18].
Using the Vuforia Web Services API is only possible if the information in the AR application is not confidential, for example, when the warehouse stores finished goods to be sent to the client, and the data displayed in the application refers to a certain object and must be made available to the buyer.

Reading files in an AR application
Vuforia allows you to download datasets from different sources. If the data to be used in the application is stored in a file or a corporate database (.XML and .DAT files), it can be read from the external storage (SD card) of the device with the application.

Number of markers read
The number of markers read simultaneously depends on the number of types of goods, methods of storage and scanning purposes.
For example, the company that stores a large number of diverse, small-sized goods and uses a piece-pick warehouse method, does not need advanced scanning and search options of the system. Such company only needs simultaneous receipt of information by one object.
In case it is necessary to select one product from a number of similar ones, there is an option of tracking several targets simultaneously.
The maximum number of targets that can be tracked at the same time in the AR application project is determined by two factors: • The number of targets that can be detected in the device camera's field of view; • The computational performance of the device. The first factor relates to a target's physical size and the maximum distance from which a set of targets can be detected to maintain tracking. The greater the number of targets, the farther the camera needs to be from them to capture them all. Eventually the necessary distance results in a loss of tracking because the features of the targets are not longer detectable.
The second factor is device-dependent. Target tracking is a computationally intensive process. Each device has an effective limit on the number of targets that it can track simultaneously while maintaining satisfactory performance of other tasks like screen rendering. Trying to track too many targets simultaneously will result in a poor user experience, due to a lowered rendering frame rate [19].
To avoid a loss of tracking, Vuforia limits the maximum number of simultaneously tracked image-based targets to 5.

Assignment
An application was developed for organizing the logistics of the warehouse of finished products of a mining and processing enterprise for storing dimensional products of the same type. And it was proposed in this work as an example of using of this method.
To show how the method proposed herein works, we developed an application for organizing industrial logistics at a company warehouse that stores medium-sized same-type products. Finished goods arrive at the warehouse in batches in special boxes, and then are delivered to retail outlets for sale. Selection criterion of products delivery is the date of manufacture. Questionnaire: • Objects to be safely stored are delivered to the warehouse in special boxes with a fixed amount of pieces in each; • Warehouse employees do not have access to the SCADA database; • The application is required for the staff in charge of organizing storage directly at the warehouse. The information provided by the application is not confidential; • The required number of simultaneously detectable markers is one.

Description of the proposed solution
Suppose the warehouse receives a batch of several boxes of goods with a certain date of manufacture. Each box has a label, one for the whole batch.
The warehouse classifies boxes by the date of batch manufacture. Each box is registered separately, without specifying the batch size, since the size of delivery to the store may differ from the batch size.
The application reads data from a text file, which has a list of batches and all information necessary for the application to work. See The application is installed on the electronic device of the warehouse employee, for example, a mobile phone or tablet; and after label scanning the screen displays the information about the product. Apart from the application, the device must also contain the file to read information from.
Minimum system requirements for the device with the application (Fig. 1).

Fig. 1. Minimum system requirements for the application
To demonstrate work of the application, we created a label as a QR code for one of the batches (Fig. 2). After label scanning, the screen displays the batch number, date of its manufacture and number of boxes currently in stock (Fig. 3).
The dialog box of the application has two active buttons that help to track movement of goods: "Append" button increases the number of boxes in the application database when a new batch of goods arrives; while "Select" button decreases this number when a box is sent to the store (Fig. 3).
In order to integrate the application developed under the above-proposed method, we must connect it to a corporate database and, if necessary, change the size of batches.

Outcome
This paper presents a new method of organizing warehouse logistics, which helps to classify and track the quantity of goods stored in the warehouse. The application described herein can visualize automation data, by extracting information from the database in real time and displaying it on the gadget using augmented reality. The objects identifying technology will increase efficient use of the warehouse and reduce delivery processing time.

Conclusion
As a rule, the main challenge of corporate warehouse logistics is search and selection of products, which leads to inefficient use of time and resources and, as a result, to decreasing performance.
Each warehouse is unique, therefore, recommendations for automation of warehouse logistics by using the hardware and software described in this paper, can be used with due amendments at many companies involved in mining industries.