AR Heart: A Development of Healthcare Informative Application using Augmented Reality

The utilization of Augmented Reality (AR) in industrial transformation is very well aligned with the fourth industrial revolution’s values. AR innovation can improve the perceptual knowledge of humans and enable us to communicate with a large amount of data. Therefore, this project takes the opportunity to create an AR healthcare app related to the human heart. The scopes are focusing on the human heart and provide information about the heart to the public besides improving their knowledge. Society nowadays is less concerned about heart health. There is a lot of information about the heart in newspapers, televisions and social media but most of the public still ignore it. Therefore, the information needs to deliver attractively. AR technology will be one of the interesting ways to spread information. The main goal is to provide information about the human heart in an AR environment. The other goals are to provide exposure for the users on how the insides of the human heart work and to make the users familiarize themselves with AR technologies. By using Unity and Vuforia Software Development Kit (SDK), this application lets users easily scan the provided marker anywhere and anytime using their smartphone to see the virtual heart.


Introduction
In the digital world, the advancement and revolution of gadgets are already widely utilized by users nowadays to have a more advanced and quality life. Presently, the gadgets became reduced in size and improved the power. It also allows the users to access internet services conveniently anytime and anywhere since they bring gadgets together. AR is already considered as information processing that is applied in a variety of latest apps. AR is a pop-up style of adventure and exploration with the combination of the real and virtual world generated by a computer [1]. In simple words, AR merges with the virtual and real-world in a real-environment and real-time interaction and it is will appear as a three-dimensional (3D) view. Many AR applications are emerging based on features of combined real and virtual scenes, such as in the fields of entertainment, educational and medical care [2]. This paper will concentrate on the healthcare field by using the marker-based AR technique because this app used a picture as a marker to make it easy for the users to use it.
Marker-based AR has various uses conforming to market purposes. Once open the application using this technique, the object in front of the camera will detect and the object's detail will appear on the device interface. 3D version replaced the corresponding object with appearing the virtual visualization on the marker. Therefore, by using their device, the user can experience the virtual object in more detail from various angles and also can rotate the 3D imagery. The unique picture or Quick Response (QR) code can be the image target to process the output after the marker is detected by a sensor. Simple patterns but have distinct style is suitable to make it as a marker because it is easy to recognize and does not need much processing ability to detect it [3,5]. The basic structure created with several black boxes and white background is an example of simple marker-based AR [4].
This project use android smartphone that have ARCore supported as the platform to perform the AR application since android is a widely used device. Microsoft Visual Studio, Blender, Vuforia, and Unity are the software used for AR creation in this project design and development. Users need to scan the prepared marker with their android smartphone and then the model of the human heart will pop-up straight away on their device's screen. After that, the users can explore all the information on every part of the heart interestingly by clicking buttons via smartphone screens [6].
There are several sections involved in this paper including literature review, methodology, result and discussion as well as conclusion of this project. The literature review will focus on the systems of AR in medical or healthcare and scopes that used augmented reality technology in their daily life. The research on the existing system is conducted to find out the advantages, limitations and also methods or tools used. By comparing them, it helps to give the overviews and ideas on the suitable AR devices, technique and software to be used in this system. The methodology and result section aims are to show and explain in detail how the process of creating AR is planned, designed, develop and implemented.

Types of augmented reality
AR has become a major issue in media usage, business, educational, healthcare, sports, safety and various fields including the evolution of mobile computing like portable devices and wearable devices [7]. In the healthcare field, AR has been used in a variety of aspects and there is a lot of application available in handheld devices that already developed using this technology. There are two famous ways to create AR. The first way is using a marker, the second way is using marker-less. Table 1 shows the difference between marker-based AR and marker-less AR. Table 1. Difference between marker-based and marker-less AR.

Marker-based AR
Marker-less AR -It use a marker that might be a 3D element, picture, QR code or human face -It use surface detection or Global Positioning System (GPS) to record the device position and display information -The virtual object can be embedded on the target and viewed after the AR device detects it -It is more reliant on the device's capabilities

Existing Projects
In the healthcare field, the amount of usage on AR for operation, medical and recovery is increased so this is great demand that can upgrade current medical practice [8]. Therefore, this situation offers to the designer, computer programmer, engineer and users the outline of the possibility of AR development in cultivating the designing of helpful applications with computer-generated elements. Currently, the medical and healthcare fields will use AR on a large scale. Nowadays most doctors and  [9]. Table 2 shows the comparison between the existing projects.  [18] relocation in orthognathic surgery become easy system LCD screen with digital camera Based on comparison Table 2, most of the researchers used smartphones as a tool to display AR. The majority of the researcher also used Unity and Vuforia as the software to design and develop the creation of AR applications. In terms of the purpose of the research, most of them want to introduce AR technology to their users. They want to make their field to become more interesting and enjoyable by using AR technology. From the observation, the marker-based technique has been a favorite choice among the researcher because it is cost-effectively.
From the overview of the research and state of the arts that had been made, there are a lot of advantages and several disadvantages of different approaches and methods used to develop an AR application. Several factors need to take action to design and develop this application system. The main key that needs to decide is the type of AR technique that will be used in this system. The other factor that needs to identify who will be used this system and this system needs to be planned in an organized and orderly manner. From the observation of the existing project, it is difficult to find an AR application that has a lot of information about the human heart. The existing project mostly shows the AR 3D model only without more additional information. Therefore, this system planned to overcome the lack of that part.

Methodology
Before the development of the application system start, deep research and some drafting about how the application will perform had been sketched. The storyboard was created and it will have multiple scenes for the application development. Figure 1 shows some of the scenes involved in the application. One of the benefits of drafting the storyboard is it will give a clear view of how the whole application will flow and unfold in 2D animation. Storyboarding is a common technique in Human-Computer Interaction (HCI) and design for demonstrating the contexts of use and the system interfaces. Therefore, it can help to make the process of application development much easier [19]. Figure 2 shows the general block diagram of the project. Unity Technologies has created a free platform game engine programme called Unity. In this project, Unity Software will be the main software used to design AR Heart applications and the android smartphone with ARCore supported will be the main hardware to display the AR virtual object. Unity can create three-dimensional, twodimensional, augmented reality and other experiences. Google's AR services which is ARCore, helps users to develop their own AR experiences. ARCore enable the smartphone to sense the surroundings, perceive the environment, and communicate with information using different Application Programming Interface (API) [20].
In this project, a marker-based technique will be used. Marker-based AR is a technique that allows the mobile device application to scan physical images, render a 3D model and allow the user to interact with it using the device. This project will be used a picture of the heart as a marker and upload it at Vuforia SDK to see the quality of the marker. The function of Vuforia SDK in this project is to create and appear the AR in virtual viewing. After that, edit the marker and 3D model in Unity Software. Microsoft Visual Studio with C# programming language is also used to write the program and to control the functionality of the application. After the design is completed, install it on an android phone then the AR Heart Application will available to use.   Once the users open the application, there will be an interface that shows the main menu of the application with several optional which is start application, KPJ Penang Specialist Hospital information and quit. If the users choose the start application option, it will be automatically activated the Vuforia's Camera on the smartphone.
After the camera is ready, the user needs to scan the provided marker or also known as an image target to see the augmented reality. Therefore, the virtual object will pop out on that marker. The user can view the AR in terms of virtual 3D model, text, voice, etc. The AR is all about the heart such asthe functionality of the chambers of the heart, the overview of the human heart and so on. Users also can choose a lot of optional in the application likes videos, self-check quiz, etc. If the users already finish exploring the application, they will quit the application, but if they want to explore more, it will go back to the main menu options.

Designing the 3D Model
A marker or picture target is a unique image that can be scanned by the device's sensor. Since it has sufficiently distinct visual points, a marker could be anything, including a picture or a threedimensional view. It's best to use a marker with a lot of corners and edges. Then Vuforia will do its job which is to trace the quality and features at the marker while analyzed it with the data that has already been set inside the Unity Software. The marker is traced if parts of the marker are in the device camera's view.
The identification of a marker depends on its position, scale, and rotation as the marker can move freely and the object will appear depending on the user's standpoint. This indicates that the 3D object will appear depends on the user's position when they scanned the marker. The Vuforia will determine and evaluate the marker by the ranked of the stars. The larger score of stars will make the detection of camera devices easier and more exact. There are two main models, the first is a model of the whole heart and the second is the inside heart model as shown in Figure 4. The function of these models is to show the overview of the human heart to the users. Other than AR heart models, the AR texts, audios, videos and animations are also added.

Application in Android Phone
After all the adjustments and design are completed, the application is ready to test. If everything is in a good position, it is ready to export to an android phone. The Android Package Kit (APK) file is created. Install that software on any android phone. Figure 5 shows the AR Heart application after installing it. Therefore, it is available for users to use it. If that software is clicked, it will be automatically on the Vuforia Camera in that phone.  Figure 5. AR Heart application in an android phone.

Result and Discussion
There are a lot of scenes that had been created. All of these scenes have audio that explains the current scenes. Figure 6 shows the first interface which is the main menu when the user opens this application. On this opening screen, the user can select three options. Once the user has selected Start Application options, the scanner interface will pop up. The user can scan the marker at the other device or directly scan it on a piece of paper. The user needs to scan the provided marker as shown in Figure 7 to see the Augmented Reality 3D model of AR Heart.   After the user scanned the marker, the augmented reality will come out. Figure 8 shows the AR whole heart model while Figure 9 shows AR inside the heart model. Each chamber inside the heart is labelled and when the user clicks on it, it moves to a different scene which explains the function of each chamber of the heart respective to the label clicked on as shown in Figure 10. In the same scenes, there are also videos link to YouTube that tells stories about how the chambers work. In this application, the blood flow is also shown in augmented reality and there are a lot of optional that users can choose in this application. Figure 11 shows the blood flow at the right chamber of the heart while Figure 12 shows the blood flow at the left chamber of the heart. The arrow in the model represents the blood flow. There is an option in the top right corner of the screen to provide an audio explanation.      Figure 11. Blood flow at right chamber. The feature of additional options also added as shown in Figure 13 which is includes more information in the form of informative videos that have no copyright claims. However, since such videos are quite hard to come by, the links that redirect to YouTube videos, for the time being, are used. This feature requires an internet connection on the user's device. More videos are also prepared for the users who do not have an internet connection. Therefore, they can still watch videos related to the human heart. The main menu of this application also has the KPJ Info button. That means if the user clicked that button, the information about KPJ Penang Specialist Hospital will pop up. The information included the KPJ official website, the KPJ location that linked directly to GPS, social media like Facebook and Instagram as shown in Figure 14.   Figure 14. KPJ information.

Conclusion
Let's invite more people toward modernity, where everything we can access using the Internet. With the availability of the Internet, everybody can search large amounts of knowledge. By developing this AR Heart app, hope it can help to provide useful information to the users. Since there is no waste of non-renewable energy, AR Heart is proven to become cost-effective. The purpose is to provide information about the human heart in an Augmented Reality environment, to provide exposure for the users on how the insides of the human heart work and to make the users familiarize themselves with AR technologies. By using software development such as Unity, Vuforia, Microsoft Visual Studio and programming C#, the aim of the project was successfully accomplished. Using ARCore, this application lets users easily scan the provided marker anywhere and anytime using their mobile phone to see the virtual heart. Furthermore, with this application, the users can see the heart in three dimensions. Some issues had been discovered during implementing the AR Heart application. Amongst the most serious problems is the animated visualization of blood flow inside the chambers of the heart not appropriate. Sometimes, the AR is not in the right position at the user's view. The method used for developing this app was Rapid Application Development (RAD), this way will be able to check and verify the app and make sure the app is function properly as planned. Although we achieved the target, there still have more things that could be include in this application. For example, this app can create a way to detect the user if they exhibit any symptoms of a heart attack occurring. As of now, the AR Heart app that is made is concentrated on providing information, which mainly revolved around the heart itself. The improvement is planned for this application with more features in the future that will be more cantered on heart attack disease and precautionary steps that can be taken for a healthier lifestyle.