Robot sensing based on electrical capacitance tomography sensor with rotation

The existing sensing technology for industrial robot hands cannot sense inside of an object. To recognise the material and internal distribution of an object when it is grasped, electrical capacitance tomography (ECT) may be used if the object is non-conductive, as ECT can measure the permittivity distribution inside an object from external capacitance measurements. An ECT sensor normally has 8 or 12 electrodes equally distributed around an object. This paper introduces the first attempt to make use of ECT with robot hands. Because the number of electrodes is limited, it is necessary to rotate the object to be imaged mechanically, so that sufficient measurements can be taken. Four aspects of the ECT sensor are discussed based on simulation: (1) the structure of electrodes; (2) the number of rotation times; (3) the starting position of rotation; and (4) the diameter of rotation. Two image reconstruction algorithms are used: the linear-back projection and Landweber iteration with three typical distributions. The results show that the rotational three-electrode ECT structure has good ability to produce useful images, showing the internal distribution of unknown objects.


Introduction
Robot sensing is a popular research topic, which endows robot hands and fingers with sight, hearing, and touch. Robots can perform some tedious or dangerous production activities [1]. It is expected that forthcoming robot hands will be able to execute various complicated tasks, very like human hands and even have abilities surpassing human. Tactile exploration is an * Author to whom any correspondence should be addressed.
Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. important ability for both human and robot hands to recognise and process unfamiliar objects. Different from human, who can handle complete mixture of haptic features over time and space, robot hands are constrained in contact interaction tasks [2,3].
Okamura et al developed a sensor array applied to robot fingers for rolling/sliding a sensor over an unknown object surface to determine surface properties. The object is grasped and held between one finger and the palm, then the other finger begins to roll and/or slide. However, to maintain the grasp stability, a limitation is the workspace for the rolling finger, which affects the measurement results [4]. Romano et al used two 5 × 3 capacitive tactile sensors integrated with the PR2 two-finger robot gripper. Different control states are triggered by tactile signals. The detection of slip information is successful but is only used in simple pick-and-place tasks [5]. Following Romano's work, many researchers have worked on robot hands integrated with different tactile sensors. Soh and Demiris presented an online discriminative and generative classifier based on the iCub humanoid robotic platform, with a five-finger robot hand and 12 capacitive pressure sensors on each fingertip. Their results showed that tactile signals can be used to classify objects [6]. Chebotar et al proposed a threefinger hand installed with biomimetic tactile sensors for effective re-grasp. The spatio-temporal characteristics derived from biomimetic tactile sensors were used to adjust the re-grasping, and the success rate of the grasp was improved [7]. Spiers et al achieved high categorisation and the capacity to retrieve aspects of an object's dimensions during a single grasp [8]. A three-finger robot hand with an optical tactile sensor was designed by Pestell et al. The data extracted from the fingertips can be used for grasping control [9]. An object's properties contribute to manipulating actions, while an object class can execute an object-specific strategy or plan [10]. Robotic tactile sensing has recently been employed with electrical impedance tomography (EIT) [11,12]. It is challenging to use EIT to recognise conductive objects, because the currents pass through the object without creating a voltage difference.
Electrical capacitance tomography (ECT) may be used for robot hand sensing. A typical ECT sensor has 8 or 12 electrodes to obtain the internal cross-sectional material distribution by measuring the capacitance between all different electrode pairs. It has advantages of low cost, high speed, non-intrusiveness, non-invasiveness, and without radiation. Over the past decades, ECT has been used extensively for multiphase flow measurements in pipelines and circulating fluidised beds, and many other fields [13][14][15][16].
Industrial robot hands typically have two or three fingers. Therefore, the number of electrodes is restricted to three or fewer, which is insufficient for reconstructing images. One possibility to increase the number of measurements is to rotate the robot hand mechanically [17]. In this paper, a threeelectrode ECT sensor is used with rotation for robot hand sensing. It is expected that it can recognise internal distribution of objects, identify the constituent materials, and provide information to improve grasp stabilisation without relying on palpatory motion and dexterity of robot hands.

Sensor design
A three-electrode ECT sensor gives limited information for image reconstruction. If the three-electrode ECT sensor can be rotated around an object, the number of independent measurements can be increased. For one angle measurement, one of the electrodes is used for excitation and the other two electrodes for detection. The total number of independent capacitance measurements with this strategy is where T and N are the number of rotations and the number of electrodes, respectively.

Sensor model
A 2D model of the three-electrode sensor structure used for the simulation is shown in figure 1. Its inner and outer diameter is 18 mm and 20 mm, respectively. Each electrode is 3 mm in length and 0.55 mm in width. The low and high permittivity values are 1.0 and 2.6. The finite element method software package COMSOL is used. For image reconstruction, linear back projection (LBP) and Landweber algorithm with 100 and 400 iterations are used to analyse the effect of the number of iterations on the reconstructed images. When a three-electrode ECT sensor is used with mechanical rotation, some capacitances may be measured more than once in different directions, because when they are rotated to a certain angle, the positions of the two electrodes may overlap. Therefore, the electrode structure affects the number of independent measurements.
To investigate the impact of electrode structure on the number of independent measurements and eventually the quality of reconstructed images, three structures (1-5-7, 1-3-9, and 1-7-11) are selected based on a conventional ECT sensor with 16 electrodes as shown in figure 2. Several rotations are conducted with each structure to collect data for one image. As a three-electrode ECT sensor with eight rotations, 24 independent capacitance measurements can be obtained.

Electric field distribution
The electric field and the potential distribution with electrode 1 for excitation and the other two electrodes grounded are shown in figure 3. The potential contours of a conventional ECT sensor converge to the excitation electrode, but the electrical field contours of a three-electrode ECT sensor are expanded outside the image area, because the sensing area of the three-electrode structure is open. The performance of a threeelectrode ECT sensor is affected by three factors: (1) the electrode structure, (2) the rotation times and (3) the starting point of the orientation.

Sensitivity map
A three-electrode ECT sensor without rotation has three essential sensitivity maps. By rotating and mirroring, other sensitivity maps can be generated. For the three-electrode ECT with T  rotation times, there are 3T sensitivity maps in total for image reconstruction.
It is necessary to obtain the sensitivity maps for each sensor structure. Figure 4 shows three typical sensitivity maps of structure 1-5-7, 1-3-9 and 1-7-11. The quality of the image reconstruction is directly affected by the sensitivity maps. As shown in figure 4, the sensitivity is higher in the proximity of the excitation electrode, whereas the sensitivity decreases towards the central region.

Image reconstruction
The ability to identify object characteristics or categorise them using both visual and tactile input is an innate sensory skill possessed by humans and animals. Vision-based approaches have been investigated to extract object properties in robots, but there are limitations in terms of surrounding environment and reliance on multiple touches. Moreover, robotic tactile sensors have been used for surface material verification only. For industrial pick and place task, some items may be nested from more than one material. If ECT can be applied with robot hand fingers, internal materials and their distribution can be reconstructed. Then, a grasping action can apply an appropriate force according to the information from the reconstructed image, improving grasp stability. 360 • rotation can be achieved by a joint between a robot hand and an arm.
To compare the effects of the three-electrode structures on the reconstructed images, three typical distributions are used, as shown in figure 5. These distributions contain high permittivity of 2.6 in red and low permittivity of 1 in blue.
To evaluate the ability of the three electrode structures in image reconstruction, the correlation coefficient (CC) between the typical distributions and reconstructed images is used [18] where g is the actual permittivity distribution of the object under measurements;ĝ is reconstructed permittivity distribution;ḡ andḡ are the mean values of g andĝ, respectively. The higher CC, the higher the image quality. Figure 6 shows the images reconstructed using the LBP algorithm and their CC. To visualise the reconstruction results, different scales of the reconstructed images are selected for different distributions. Only the central core distribution is discernible with the structure 1-5-7. With the structure 1-7-11, the central core and the edges of the other two distributions are blurred. Only the structure 1-3-9 generates good images in all three distributions. A higher CC value means that the reconstructed image is more similar to the original distribution. The CCs shown in figure 6 reflect that three-electrode ECT sensor used with rotation has ability to generate recognisable images.

Selection of electrode structure
From the results above, the structure 1-7-11 with electrodes symmetrically placed, and the structure 1-5-7 with three electrodes on one side are not good options. Some scenarios of industrial process need robot hands to manipulate unknown objects. To grasp an object stably, the distribution of the three electrodes cannot be on one side. Therefore, the angle between any two of the three electrodes and should be, as shown in figure 7.
To reconstruct high-quality images from a three-electrode structure with an open sensing region, it is essential to choose an optimal electrode structure. Figure 8 shows that there are eight different structures that may be used to investigate the effect of the electrode structure. From previous work in [17], the symmetrical structures are proved to be defective, and so these structures are not chosen in this research. The structures that are symmetrical with structures in figure 8 about the diameter and the centre of the circle are excluded. They are classified as the question about different start point for orientation discussed in section 3.3.

Different electrode structures
In this section, the LBP and Landweber algorithm with 100 iterations are used to reconstruct the images. Based on eight different structures as shown in figure 8, the reconstructed images of these structures are shown in figure 9. From the reconstructed images, it can be seen that only structure 1-7-9 with the LBP algorithm can generate satisfactory results for the central core. Images of other structures become visible when the Landweber algorithm is used. For two objects distribution, all images obtained by the LBP are unrecognisable. However, a noticeable improvement is achieved with Landweber algorithm. As for the stratified distribution, all structures with the LBP can generate recognisable images despite the blurring edges. With the Landweber algorithm, the edges become distinct but distorted. Image relative error (RE) is used as a quantitative assessment for evaluating the reconstructed images. A smaller RE means higher image quality A comparison of the RE for the central core distribution among the eight structures is shown in figure 10(a). It can be seen that the structure 1-7-9 gives the lowest RE about 0.75 with LBP, and RE close to 1 with other structures. After 100 iterations, the smallest RE with the structure 1-7-9 is between  0.75 and 0.37. A significant improvement of the image quality can be seen with the structures 1-5-10 and 1-6-10. Figure 10(b) shows a comparison of the RE for the distribution of two objects. All structures with LBP generate images with similar RE close to 1. The image quality is obviously improved with the Landweber algorithm. Moreover, the structure 1-7-9 generates the smallest RE.
RE of the stratified distribution is shown in figure 10(c). Compared with RE of the previous two distributions, a low RE can be seen with LBP of approximately 0.75 for all structures. RE are decreased by the Landweber algorithm after 100 iterations, and the structure 1-8-9 gives the smallest RE of approximately 0.58.
Because the sensing area of the three-electrode ECT sensor is open, the structure of electrodes is one of the key factors affecting the reconstructed image quality. It can be seen in figures 9 and 10 that the structure 1-7-9 has obvious advantages over other structures. Only structure 1-7-9 can reconstruct the central core distribution clearly without iterations and generate two objects with the lowest image error. That is, the structure 1-7-9 can obtain more valid measurement data than other structures to reconstruct images during the rotation.

Number of rotations
Once the optimal electrode structure is determined, the next crucial step is to consider the number of rotations. This refers to the number of angular subdivisions. It is assumed that increasing the number of rotations would enhance the image quality, because of more independent measurements. In this section, the same algorithms are used as before, but 400 iterations are added for comparison. Five kinds of angular subdivision based on the structure 1-7-9 is chosen as shown in figure 11. And figure 12 shows comparison of the RE of the reconstructed images with different numbers of rotations with three distributions.
Interestingly, there is no correlation between the number of rotations and the improvement of image quality. For the central core distribution, 8 and 16 rotations have advantage over other numbers of rotations. The RE of 8 rotations is slightly lower than that of 16 rotations. For two objects and stratified distributions, the performances of different rotations are similar to each other, but eight-rotations always gives the lowest RE.

Same structure with different start orientation
In section 2.5, the same structures symmetrical about the diameter and about the centre of the circle are excluded.
To verify the influence of different start orientation of the same structure on reconstructed images, three models are selected for comparison with the structure 1-7-9 as shown in figure 13 The structures 1-3-9 and 1-9-11 are symmetrical to the structure 1-7-9 about the diameter, and the structure 1-9-15 is central symmetrical to the structure 1-7-9. Figure 14 shows the RE of the structure 1-7-9 and its symmetrical structures for the three distributions. For the central core distribution, the structure 1-7-9 gives the lowest RE with LBP about 0.76, and the RE decreases from 0.76 to 0.43 after 100 iterations. Although the RE of the structure 1-9-11 is higher than that of the structure 1-7-9, it drops significantly to 0.36 after 100 iterations, becoming the lowest among the four structures. The four structures give almost the same results for the two objects and stratified distributions. It can be seen that only the central object distribution is affected by the start orientation.    Because the central core distribution is symmetrical in all direction of the sensing region. Figure 15 shows the isogram of electrical filed lines with electrode 1 for excitation, and the combined distribution after rotation with different starting point for orientation. It can be seen that the distribution of electric field lines with different starting points has big influence on the central region after rotations. That is why only the central object distribution is affected by the start point for rotation.

Diameter of rotation
Modifying the rotation diameter would affect the capacitance between the electrode pairs and, consequently, the distribution of sensitivity. The simulation is performed using six models with diameters of 12, 16, 20, 24, 28 and 32 mm, respectively. For the structure 1-7-9, the diameter of the rotation, which is also the distance between the electrode 1 and electrode 9. Figure 16 shows that with a smaller rotation diameter, the two electrodes at both ends of the diameter provide relatively high sensitivity in the central area and low sensitivity close to the electrodes. As the diameter rising, the negative sensitivity around the electrode is increased. When the fixeddiameter circular item is positioned at the centre, the image contrast has negative correlation with the rotation diameter. By comparing the reconstructed images generated with two algorithms, the LBP is unable to generate useful images. Yet, the image contrast is enhanced and become discernible with the Landweber algorithm with 100 iterations.
The RE with different rotation diameters are compared in figure 17. A small diameter of rotation gives a low starting RE, and RE decreases and becomes stable in a few iterations. For a large diameter like 32 mm, the initial RE without iterations is high. As the number of iterations increases, RE decreases slowly and stops decreasing after 50 iterations. Whereas, the RE of 16 mm diameter drops sharply after ten iterations and exhibits a great improvement between the initial value and the stable value after iterations.
In general, a closer distance between electrode pairs results in an increase in sensitivity the central area. Because the increase in diameter causes a decrease in sensitivity in the central area, a fixed-diameter item positioned at the centre of a larger rotating diameter produces a lower contrast. At shorter distances, fewer iterations are required to stabilise the RE.

Length of electrodes
The length of electrodes is another key issue in ECT sensor design, here it depends on the robot fingers. Simulation is carried out using a 60 mm length columnar model of 20 mm in diameter with low permittivity of 1, and a columnar of 5 mm diameter with permittivity of 2.6 is set in the central of the model. In [19], it is mentioned that when the length of the electrodes is cut down to a certain point, further cutting would not narrow the sensing zone. Simulation is performed with six different electrode length of 10, 12, 14, 16, 18 and 20 mm, which are smaller than or equal to the diameter of the model. Figure 18 shows the images of three-electrode ECT sensor with six different electrode length reconstructed by Landweber algorithm with 100 iterations.
For a traditional ECT sensor, the length of electrodes is usually larger than its diameter [19]. Simulation is performed for six other electrode lengths larger than the diameter, which are 25, 30, 35, 40, 45 and 50 mm. The reconstructed images are shown in figure 19.     electrode assembled on the robot finger is about 0.7 times to the diameter of the target object, which is conducive to image reconstruction.

Conclusions
This paper presents three-electrode ECT sensors with mechanical rotation for use with a robot hand. Although this type of ECT sensor consists of only three electrodes, it can provide sufficient independent measurements and generate images similar to the traditional ECT sensors with 8 or 12 electrodes. The three-electrode ECT sensors are studied with different electrode structures, different rotations, different start orientations and different diameters of rotation. Their effect on the image quality is also investigated. The conclusions are as follow.
• According to the initial simulation results, a symmetric structure and three electrodes on one side of the diameter are not recommended. The results of the different structures show that the structure 1-7-9 has superior performance for the three typical distributions. The structure 1-7-9 is combined with two of the electrodes distributed at both ends of the diameter of the rotating circle and the last electrode located at a 45 • angle to that diameter. • The image quality may benefit from a greater number of independent measurements. However, the reconstructed image quality is not improved with the increase in the number of rotations. Superior results are obtained with eight rotations, not ten or more. • Only central object distribution is affected by the starting position. • Theoretically, the sensitivity is affected by the change in the distance between the electrodes, and a small distance can enhance the sensitivity in the central area. A smaller distance between the electrodes means a smaller rotation diameter. • A large rotational diameter causes reduction of image contrast for the central object with a fixed diameter. The RE becomes stable quickly by reducing the rotation diameter. • For a three-electrode ECT of 20 mm diameter, 14 mm length of electrodes makes superior results.
This work is a preliminary simulation study and verification of whether such three-electrode ECT sensors can be applied to robot hand sensing. The promising results shows that the threeelectrode ECT sensors rotated with a robot hand is worthy of further research. This can provide useful information for object classification and grasp adjustment.

Data availability statement
All data that support the findings of this study are included within the article (and any supplementary files).