Application of simple kinematic model from flexion movement of upper-limb with RGB-D camera perspective

This study describes the use of an RGB-D camera for the assessment of the upper-limb movement for stroke rehabilitation patients. The assessment process is carried out by making comparisons between patient movements and simulated movements. The motion simulation is modelled by the kinematics model of the 6 DoF arm with extended flexion motion. Tests were carried out on 13 normal patients with movement schemes that often appear in the rehabilitation process. The results show that the use of the 6 DoF model results in better accuracy and calculation time than using the 8 DoF model.


Introduction
The telerehabilitation system for partial stroke paralysis is the important equipment for this pandemic situation. Based on data [1], only 39.4% of patients who undergo therapy routinely until they recover, the others do not return to treatment after treatment and therapy with an uncertain schedule. Uneven health facilities are one of them, because there is still high awareness of patients about stroke disease outside of economic factors. In designing a telerehabilitation system, a guide robot is needed as previously designed [2] [3]. These robots are divided into two groups, namely exoskeleton robots, and end-effectors. The end-effector robot allows free movement of objects with a greater degree of freedom than the exoskeleton robots. The end-effector robot system that has been built [2] is not equipped with a feedback system so that the movement of the robot cannot adjust to the object movement. The monitor system required is at least able to represent all the movements of the upper-limb in three axes. During its development, this monitoring system has been carried out by various methods. Du et al (2018) used the IMU sensor to track upper-limb movements [4]. Meanwhile Ref [5] uses an EMG sensor to assess the strength of the arm and will take action if the natural object is fatigued and unable to perform a movement. The system using EMG is generally used on a single axis system, so it is quite easy to represent muscle strength in one axis. A monitoring system using a camera has also been used [6]. This monitoring uses a positioned camera to assess hand movements on the table. Monitoring systems using RGB-D cameras have also been used [7] [8]. In implementations, [9] uses a comparison with the kinematic model of upper-limb as a comparison of the motion captured by the RGB-D camera.
The problem that has not been resolved in the system [9] is that there are still several factors that have not been modeled such as the tilt of the camera towards the object, curvature of the elbow in a straight position, and the position when the wrist, elbow, and shoulder are located in a straight line parallel to the camera. This problem causes the reading from the camera to have an error in that case. So that in this study it is necessary to add compensation to overcome the problems from the previous paper. Then the modeling in previous research also includes from the shoulder to the fingertips, which makes the assessment results take time to process these commands. This paper will discuss the simplification of the model into six degrees of freedom with the end effector being the wrist. This simplification of the model is intended to reduce the calculation time required for the assessment. This paper is composed of an introduction containing the background, problems, and objectives to be achieved, then a method containing system design and model simplification. For testing, a comparison with the previous model will be discussed regarding the resulting accuracy after adding some compensation and a comparison of the calculation time of the two designed models. In closing, this paper ends with conclusions and possible future research to be developed.

Methods
The system designed is a development of Ref [9]. This system still uses an RGB-D camera with Kinect v2 type to measure the coordinates of the object's upper-limb motion when performing a flexionextension. Extension-flexion is a movement by moving the upper and lower arms in the sagittal plane.

Design System
The system design consists of an RGB-D camera and a computer as a signal processor. The camera system used is the Kinnect v2 camera, which is the latest version of Kinnect. In the system design, the model is used as a reference. The flowchart of the proposed system can be seen in Figure 1. Starting with a camera set up to calibrate the system against the patient's physical location and condition. From these two parameters, the camera is placed at the optimal point of the observation. After obtaining the optimal distance, the camera reads the coordinates of each joint to obtain the length parameter of each segment as input to the model. In the assessment process, the camera is only tasked with reading the position of the coordinates of each joint when performing the flexion motion which is used to describe the trajectory of the object's upper-limb motion. The resulting path is then compared with the path of the simulation result. The assessment is based on the deviation from the actual and simulated paths. The greater the deviation, the smaller the value to be obtained at each joint.

Upper-limb Kinematic modelling.
The purpose of modeling is to assess the flexion motion of the object. An illustration of the flexion and path movements resulting from the reading of these movements can be seen in Figure 2. There are 3 paths that must be observed to assess that the flexion movements performed are correct, namely wrist, elbows, and shoulders path. This path is represented on three axes to accommodate the movement of objects in three axes (X, Y, Z). The green path is a path of the wrist, the red one is elbow and the blue one is the shoulder.
To adopt this flexion motion, it is actually enough to use one degree of freedom located on the shoulder, the angle of which is changed increment in value to get the coordinates of the end-effector. However, this model cannot accommodate the differences in hand structure in each object. This difference in structure is caused by the historical conditions of each object. This different pattern causes the model of the hand of each object to not be completely straight. Where h1 is the chest coordinate height and is parallel to the camera height. a1 is the distance between the camera and the center of the chest. a2 is the length of the shoulder, which is the distance between the center of the chest and the shoulder. a3 is the length of the upper arm or the distance between the shoulder and the elbow and the last, a4 is the lower arm, which is the distance between the elbow and the wrist. From the model in Figure 3, the Denavit-Hartenberg parameter (DH Parameter) is obtained which can be seen in Table 1. The parameters of the six joints are used to get the coordinates of the wrist, elbow, and shoulder by applying the forward kinematics transformation method. The transformation convert s the change in angel θ1 -θ6 to the change in the coordinates of the end-effector. The coordinate position of the wrist can be calculated by multiplying the transformation matrix for each joint. Where θi is a parameter that represents the angle of the connection formed from the xi-1 axis to the zi axis with the right-hand rule. The parameter di represents the distance from the center point of the coordinate frame between the zi and the xi axis along zi. The ai parameter represents the distance from the intersection point between the zi axis and the zi+1 axis on the xi axis. And the αi parameter represents the angle between zi and zi+1 axis on the xi axis. The First joint is used to tolerate reading errors due to the angle formed between the camera and the chest plane. This angle is formed because in its placement to facilitate system implementation, the camera is only positioned at an optimal distance from the object. This optimal distance is sought by calculating the minimum error value from the measurement results with the camera's RGB-D with the actual value. The camera height is set parallel to the center of the chest using a tripod. The system set up is done by displaying the measurement results on the monitor according to predetermined parameters.  The calculation of this value is based on calculating the deviation between the simulation and the value that was read. The greater the deviation value, the smaller the final value. The deviation calculation using point to line distance measurement as in the equation (1).  FS is the final score which represents the value of the movement made by the object. The value of FS is in the range 0-100. The greater the value indicates the movement is approaching the normal upper-limb movement.

Result and Discussion
Several tests were carried out on the proposed model. Tests were carried out on 13 objects in healthy condition, consisting of 2 women and 11 men. The object age ranges from 17 to 24 years, with ideal physical conditions. Figure 4 is the result of running the program on an object with normal movement. It can be seen that for normal movement, the overall score is 72.22, the shoulder score 58.15, the elbow score is 79.98 and the wrist score is 78.54 on a scale of 0-100. From this value will be determined the minimum threshold of a normal upper-limb movement. This value is used as a threshold indicating that if the score obtained lower than threshold, the result of the movement is acted as an abnormal movement. And vice versa, movements that have a higher score than threshold can be classified as appropriate (back to normal). In Figure 4, it also can be seen that the farthest deviation from the model occurs at the position when the end-effector is marked with a red node. From this node, it is hoped that the therapist will know which part of the movement needs to be improved. The second test is done by comparing 2 kinematic models, namely a model with 8 degrees of freedom with a model with 6 degrees of freedom. The test results can be seen in Table 2. It can be seen from the average model with 6 DoF that it requires relatively less time than the 8 DoF which is 2.35 s. Although some data showing that the time required for 6 DoF is greater, such as in the 5 th object data. This is because, in the data, the number of points tends to be more than the others. This more point is obtained because the object moves the hand too slowly. In Table 3, can be seen the comparison of the scores obtained from the same 13 data with different models. It can be seen that the overall average with the addition of this compensation shows that the score on the 6 DoF model is better than the core at 8 DoF, which is 74.41, while the 8 DoF

Conclusion
From several tests it can be seen that the proposed model has better accuracy and time values than the previous model. Although it does not apply to all data, the overall results can represent the entire data taken. So that for the sake of it, this model can be used for the telerehabilitation system to be built. And this model is also possible to be combined with a wireless IMU sensor to get a better assessment value.