Improving teleoperation robots performance by eliminating view limit using 360 camera and enhancing the immersive experience utilizing VR headset

A teleoperated robotic system offers a rapid deployment for performing tasks in dangerous environments. An effective teleoperation requires a proper feedback system such as a vision system using cameras. Despite its effectiveness of using camera, the limited field of view (FoV) become one of the main burdens of the operation. To deal with this problem, some researchers used wider-view camera or multiple cameras. In contrast, this research proposed to achieve a better teleoperation by eliminating FoV limit by using 360 camera. This research also aimed to improve the immersive experience by utilizing virtual reality (VR) headset. The proposed framework then developed to accommodate all the requirements and then evaluated by some numerous trials by some participants. An AprilTag marker was placed on the robot and tracked by a ceiling mounted camera to record the trajectory data. The obtained trajectories then measured its performance by using a standard performance metrics. The experiments demonstrated that the proposed method achieved lower collision area, closer to desired ideal path and smoother than using a standard camera as the feedback system. However, as the 360-video streaming induced higher latency, the proposed approach was slower. The measurements justified that the proposed approach achieved a better teleoperation especially for mission critical tasks.