Understanding of Virtual Reality with Visual Sensor Networks

With the development of computers and information technology, people are looking for new types of Human-Computer Interaction technologies to achieve a better experience. In recent years, virtual reality technology has become one of the major hotspots. Virtual reality technology provides a virtual scene for the users and allows them to generate virtual behaviors or control the elements in the virtual scene through limb movements. And the research on virtual scene modeling can enable users to obtain a more realistic experience. This article mainly discusses the virtual reality technology from the perspective of visual sensor networks, including the common scene modeling methods, the characteristics of visual sensor networks, and some related applications.


Introduction
In the past 30 years, the emergence of the Internet and the World Wide Web has had a revolutionary impact on the development and application of information technologies, especially e-collaboration [1,2]. Among many fields of research, research on Human-Computer Interaction (HCI) has been a hot topic in recent years. In fact, interaction design is a cross-discipline [3,4] involving computers, images, cognitive sciences, psychology and other disciplines. While investigating the principles of interaction, researchers are also paying close attention to new forms of interaction, from the most primitive keyboard and mouse input, to touch-screen mobile tablet computers, to virtual reality technology (VR), and to new interactive technologies as long as the innovations in computer technology and contributions from each generation of researchers. Recently, a popular movie Ready Player One tells us exactly what people imagined in the VR world. On the one hand, it shows people's imagination, on the other hand, it also reveals that VR has received a high degree of attention throughout the world. At the same time, related scholars have conducted a lot of researches and discussions on VR, including lens coverage, transmission network construction, data transmission, visual data processing, and modeling. So how to use the existing technology to improve the availability of VR products and expand the application scenarios, and how to improve the existing VR core technology have a strong research significance and practical value.
In recent years, wireless sensor networks (WSN) have attracted widespread attention in the academic community [5][6][7]. WSN is made up of many small devices, each of which is a node, and each node contains sensing, computing, communications, and power supply systems. Distributed networks facilitate the detection of the physical world and collect relevant data, which are then aggregated over wireless transmissions. With the increasing demand for technology, visual sensor networks (VSN) equipped with miniature cameras have gradually become a research hotspot. In addition to the basic functions of the traditional WSN, the VSN can sample visual images, which means that VSN not only enhances basic applications such as traditional tracking and environmental monitoring, but also expands new application areas. Besides, as foundation, VSN has helped many emerging technologies be better developed and applied. For example, in virtual reality, programmers can obtain more accurate parameters and visual information based on the VSN to build a more realistic scene model or directly achieve the effect of remote access scenarios with the help of the remote VSN.
The establishment of virtual scenes is a top priority for VR products. This article will mainly focus on the modeling of VR from the perspective of VSN, including the development history of VR, the basic structure of VSN, and the application of VSN in VR and some related advantages and challenges.

Virtual Reality
Like many human-computer interaction technologies, virtual reality is a multidisciplinary discipline that involves a number of key technologies such as computer graphics, sensor technology, real-time computing technology, artificial intelligence, and human behavioral research and at the same time, virtual reality is also affiliated with one of the new multimedia technologies. And like most of the ultimedia technology, there is no well-defined exact definition for VR. Some scholars will give their own understanding of the definition of VR. For example, Steuer [8] thinks that VR is a kind of medium for transmitting information similar to telephone or television. But unlike traditional media, VR requires specific hardware conditions, such as computers, head-mounted displays, headphones, motion-sensing gloves, etc.
As early as 1965, Sutherland firstly proposed that the computer can sense the muscle activity of the human body, and can provide a virtual interface. And people can control this virtual interface through their own activities, and the virtual interface can also give as many Feedback information [9]. By 1984, NASA's Ames Lab began developing virtual visual environment display projects for virtual detection and virtual environment management. In the 21st century, the improvement of hardware technology has greatly increased the application of VR. Nowadays, VR has been widely used in entertainment, education, medical and other industries.

Common modeling methods
The characteristics of virtual reality are mainly manifested as immersion, interactivity and imagination. Immersion allows users to put themselves into the virtual space; interactivity provides users with the opportunity to control virtual behavior in virtual space and imagination is mainly about people's ability to create virtual space. The core of the technology that achieves the above three characteristics is to create a realistic virtual environment that includes elements such as scenes, interactions, and sounds. Among them, the amount of information obtained through visual is the largest, accounting for about 70%. Similarly, the visual feedback information page is the most intuitive and realistic. Therefore, visual modeling has become one of the most important aspects of virtual reality technology. The basic modeling process is divided into two parts, scene modeling and model optimization.

Scene modeling
The traditional method of scene modeling is mainly based on 3D geometric modeling of computer graphics. And developers can use the computer graphics language represented by OpenGL as the model. With the related graphic functions, it can realize simple 3D model building and 3D real-time interaction. OpenGL has mature API for developers to use, and it also provides graphical transformation functions that can help developers build 3D graphics easily.
In addition, development tools such as VRML that specializes in virtual reality modeling or visualization are also alternatives, such as Maya, AutoCAD, 3D Max, etc. Compared with traditional programming languages such as OpenGL, VRML has higher integration and more powerful modeling capabilities, which is the main method of scene modeling at present.

Model optimization
Compared to scene modeling, the optimization of the model can lead to a better user experience. But there is a common problem that the picture is not that real and smooth. The unreality of the picture is mainly limited by the modeling material and the specific method of building the model. The picture, which is not smooth, is not only related to the method of building the model, but also had the connection with the speed of computer operations. The graphic display needs to change the real scene according to the user's operation or activity. Thus, in order to achieve a smooth and accurate picture, the computer needs to generate an image of at least 30 frames per second, and the fineness of the image will influence the refresh time of the amount of computation required to some extent. The current common model optimization direction is as follows.

Duplicated objects.
There is no need to remodel the recurrent objects multiple times. The object-oriented programming and geometric transformations can be used to control node and position changes.

Texture projection.
The texture project refers to changing the size and orientation of the geometry by using different texture maps to make its visual effects more realistic. What's more, texture project can improve the realism of objects and refine granularity without increasing the amount of modeling operations.

Blanking.
Whether it is a real or a virtual scene, the user's visual field of vision is limited. When some scenes are occluded by others, the complexity of the scene needs to be reduced by limiting the observer's field of view and perspective, thereby reducing the amount of scene modeling operations.

Visual sensor networks in VR modeling
The usage of visual sensor networks for automated assembly modeling is one of the traditional methods of building geometric models. Firstly, it makes use of conventional visual sensors to ingest image from the pending model entity or a model. Then it processes and analyses the captured images, and extracts useful 3D information and mop structure from the model. And finally, geometric model modeling method is automatic generated based on the information.
In terms of image information capture, the camera sensors can acquire more data than ordinary scalar sensors, and generally perform multi-angle shooting for the same object. The acquisition of visual images mainly depends on the lens, while the acquisition of image information requires subsequent algorithms to support. At the same time, camera coverage has become a topic worthy of study. Coverage optimization algorithms attempt to assign the working status of each node's visual sensor through calculation. For example, if the visual range of multiple visual sensors is the same, one or more visual sensors can be put to sleep. On the one hand, energy consumption can be saved, and on the other hand, the amount of information in the picture information transmission stream can be reduced. Meanwhile, since the viewed objects are mostly three-dimensional images, the camera coverage becomes pretty complicated. Thus, to solve this problem, many related scholars have proposed to solve the 3D coverage problem by using polynomial time [10]. In addition, the depth of field also affects the VSN to acquire the image information.
Common VSN fall into two main categories: homogeneous visual sensor networks and heterogeneous visual sensor networks. Homogeneous visual sensor networks mean that the camera sensors of each node in the network have the same or similar capabilities, thereby they can reduce the complexity of the network in a large VSN. Camera sensors at each node are different of heterogeneous visual sensor networks, and may even include other types of sensors such as temperature, humidity, or vibration sensors. This kind of VSN is more flexible and can be applied to more scenarios. At the same time, in order to cope with the increasing complexity of the network, the multi-tier architecture is emerging. For example, Figure 1 shows surveillance applications where camera nodes are triggered by a vibration or sound sensor and then collaboratively start sending data to the aggregation node.

Figure 1.
Example of data processing tasks in visual sensor networks [11] Nowadays, image information processing and analysis methods mainly rely on computer visionbased information technology. Some related algorithms of computer vision have been fully developed in recent years. And the image feature extraction and analysis technology has been well-developed from the principal component analysis (PCA) to deep learning (DL). Especially, the introduction of artificial neural networks greatly speeds up the processing of picture information, and with the help of deep learning, the computer can recognize the features of the picture better and faster to create a more vivid model.

Applications
As the related technology turns to be mature, VR quickly penetrates into all walks of life, such as entertainment, education, and medical care.

Entertainment
Entertainment applications have been one of the important applications of VR all the time. The main reason is that it can directly bring considerable income and profits to VR equipment and content producers as well as game industry [12]. Through somatosensory interaction technology, 3D tactile technology and 3D phenomenon technology, VR games can offer players a new type of operating experience, and also give players a more intuitive experience from visual aspects. Since the birth of video games, people have longed for scenes in the game that can become more realistic and realistic. That's why inchoate video games workers make their best to optimize game interfaces and devices. However, regardless of the degree of optimization, the experience of the computer screen to the player is still limited, but VR technology breaks through this layer of interaction bottlenecks, allowing players to obtain a more realistic experience.
Besides video games, VR technology is also valuable in movies [13] or even travel [14] in the entertainment industry. Compared with mainstream 3D movies, VR movies are a new form of expression and research direction. At present, VR movies are not yet available technically and videos based on VR technology still need more time. Virtual tourism is also a hot research topic at present. It will bring a brand-new viewing perspective for tourists, and breaks through the limitations of geographical location and climatic conditions. With the enhancement of related technologies, VR tourism may become one of the entertainment methods in the future.

Education
At present, in the education field, there are two goals that can be achieved with the help of VR technology: virtual learning environment and assistive skills training. Jarmon et al. [15] first used VR devices to virtualize a non-existent conference room environment in 2009. In this environment, graduate students from different locations use VR devices to conduct interdisciplinary learning exchanges and name this "Second Life", which has been widely praised. In 2013, Luo et al. [16] designed a set of scenario generation system, producing scenarios that can meet different objectives and at the same time be customized for individuals. This scenario generation framework is for mission-based virtual training, which aims to generate scenarios from both trainer and trainee's perspective. On the other hand, the use of VR technology for simulation training, compared with practical training, it has the advantage of avoiding security risks, saving school costs, and can even inspire students' interest. In 2014, Palter and Grantcharov [17] compared the effectiveness of teaching laparoscopic cholecystectomy between the VR and ordinary two-dimensional video. It was found that the time and error rate used by the students who received the VR teaching were significantly lower than those of the ordinary two-dimensional video teaching group.

Therapy
The introduction of virtual reality technology into rehabilitation training is a new era of intelligent rehabilitation. Through immersive 3D visual display devices such as head-mounted stereoscopic displays, data gloves, motion capture devices, force feedback devices and other interactive devices, immersive rehabilitation treatment solutions are provided for patients. VR technology has broad application prospects in the medical field, it mainly reflected in three aspects: First, medical teaching, it is helpful for theoretical explanations, anatomical exercises and surgical training; Second, clinical diagnosis and treatment, VR technology can enhance remote clinics, build lesion models and improve rehabilitation effect and realization of anesthesia; Third, medical intervention, it is mainly for the treatment of mental illness. As we all know that the growth of doctors and the innovation of treatment methods are inseparable from a great deal of practice, meanwhile, the complexity of the human body also increases the difficulty of being replaced. So the ability of VR technology to simulate real-world scenarios will effectively save resources and shorten training time, while the realistic degree of simulation will also have the effect to the future of VR technology.

Conclusion
Virtual reality technology has always been a hot topic these years. And some relevant research has emerged in an endless stream, no matter from its core technologies on software and from hardware to application areas. In terms of software, the core issue now is the VR modeling. So how to build a realistic and low-volume visual world is a topic that people are always concerned about. This paper mainly discusses VR modeling from the perspective of visual sensor networks. VSN can mainly provide visual information support for VR system, besides, it can also supply concise and effective image information to support the establishment of digital models. At the same time, a wide range of applications give the VR industry enough confidence to create more value. While the related VR software technologies are rapidly developing, we should also pay enough attention to the innovation of VR hardware. A comfortable and well-worn interaction mode can greatly improve the user experience and thus make VR products even better.