The Critical Study of Mutual Coherence Properties on Compressive Sensing Framework for Sparse Reconstruction Performance: Compression vs Measurement System

Compressive Sensing (CS) framework becomes well known since its ability to recover signal only by using less sampling required by Shanon-Nyquist theorem. The lack of required sampling is no longer constraint for having good reconstruction performance. The load is shifted to the reconstruction procedure instead of the sampling acquisition process. As long as the signal can be guaranteed sparse, the CS based method is able to provide high reconstruction accuracy. One of the CS principle is incoherence property, which can be represented by mutual coherence value. It represents the coherence between the sensing matrix and the sparse base dictionary. The theory said the less coherence between those two parameters, the more precise the reconstruction is. In fact, it is not consistently applied. The research presented on this paper find that, the theory is consistent for reconstruction on compression system, while it is not applied on the reconstruction of measurement system. Other properties are found to be more representative on assigning necessary condition for reconstruction performance on measurement system.


Introduction
Sparse reconstruction becomes challenging and developing research issue since Donoho and Micahel Elad put fundamental mathematical background and start to introduce it as Compressive Sensing framework [1] [2]. The Compressive Sensing framework is a revolution on the signal recovery solution. It enable signal recovery by having lower dimension of sampling compared to the Shanon-Nyquist theorem requirement [3]. This framework has been widely utilized on solving some reconstruction problem [3][4][5][6].
Mathematically, the compressive sensing approach can be used to solve under-determined linear system or inverse problem with ill posed condition. This phenomenon is frequently happened on image reconstruction from a certain sensor measurement [7].
Given a discrete-time signal and Mdimension of measurement value [3], The original signal is often reformulated as a linear combination of a small number of signals taken from a "resource database" determined as dictionary ϵ . = s Now s is the sparse representation signal of x. The main idea of CS system is projection of to a low dimensional measurement vector y by sensing matrixΦ, which is completely has no relation to the sparse base (dictionary) [5].
Compressive Sensing framework has two main principles need to be concerned. These principles are used to make sure the compressive sensing is working to solve the reconstruction problem. Sparsity is the first principle that is important on the Compressive Sensing framework. Sparsity in the CS framework means that most of the coefficient vector s is zero. Thus, most of the signal component can be ignored without eliminating the important information. The reconstruction by less sampling dimension is feasibly done by CS framework as long as the reconstructed signal is sparse in the original domain or in its transformed domain [1,3,7]. Thus, to make sure that the reconstructed signal is sparse, the original signal is often reformulated by certain dictionary as mentioned on Equation (3).
The second principle to guarantee the reconstruction performance by CS framework is the mutual coherence. The idea of compressive sensing is actually shifting the sampling load into the reconstruction method load. The reconstruction accuracy can be elevated by maintaining low mutual coherence. Low mutual coherence indicates that the sensing matrix and the dictionary is uncorrelated. In the established CS framework, the recovery or reconstruction process is successful as long as the mutual coherence between the sensing matrix and the dictionary is low [3,7]. In fact, it is not consistently applied. The two properties of the Compressive Sensing framework lead to the requirement on designing the sensing matrix and the dictionary.
The research presented on this paper evaluate the correlation of mutual coherence as the independent variable into reconstruction performance both on the compression system and on the measurement system as the dependent variable. X-ray image reconstruction and Electrical Capacitance Volume Tomography (ECVT) image reconstruction is used as the case study to evaluate the significance of the mutual coherence into the reconstruction performance on compression system and the measurement system respectively. In addition, it also evaluates the other properties that are potentially required as necessary condition for achieving better reconstruction performance on the measurement system.

Overview
CS framework is said to be enable to reconstruct a certain naturally sparse or transformed sparse signal by utilizing less number of sampling data compared to the Shannon-Nyquist theorem [3,7] Equation 4, it can be seen that important parameter on Compressive sensing framework is: the sensing matrix ( ) , the dictionary ( ) and the sparse representation signal (s). Thus, in general, the Compressive Sensing framework can be mapping into three main concerns:  Sensing matrix design which is enable to characterize the system and satisfy the mutual coherence properties  Dictionary setting to determine appropriate sparse signal that represent the original signal or image.  The reconstruction algorithm to reconstruct the sparse representation signal or image

Compressive Sensing Principles
Some Fundamental principles on Compressive sensing are aiming to guide the compressive sensing objective function determination, hence the optimal performance of Compressive Sensing can be achieved [8]. Sparsity and incoherence are two important principles on Compressive Sensing [8].

Sparsity
Sparsity means that the signal has low information rate which most of the coefficient vector is zero. Thus, most of the signal component can be ignored without eliminating the important information. In another words, it is sparse in the original or other transformed domain. To make sure it is sparse, normally the sensed signal is transformed into certain domain by certain sparse base function that is called as dictionary [7]. This property is needed to gain accurate signal recovery [3,9]. Some dictionaries commonly used are cosine base, sine base, wavelet base, chirplet base, curvelet base, etc [7].

Incoherence
The second property to guarantee the reconstruction performance by CS framework is the mutual coherence. The idea of compressive sensing is actually shift the sampling load into the reconstruction method load. The reconstruction accuracy can be elevated by maintaining low mutual coherence. Low mutual coherence indicates that the sensing matrix and the dictionary is uncorrelated. One of parameter than can be used to measure the incoherence property is mutual coherence.

 Mutual Coherence
Mutual coherence of = Φ , denoted as ( ), determines the worst-case coherence between any two columns (atoms) of A [2].

Definition 1. Mutual Coherence
For a given matrix = Φ , the mutual coherence of A { ( ) is defined as the largest absolute and normalized inner product between the two different columns in A, formulated as [10]: The strongest similarity between the different columns of matrix A can be evaluated by their mutual coherence. The measurement can reflect the weakness of the matrix. A close relationship between two columns may confuse any greedy pursuit algorithm, which can lead to inappropriate reconstruction [10].
The Gram matrix is considered another way to measure the mutual coherence.

Definition 2. Gram Matrix
For a given matrix = Φ , the Gram matrix is defined as [9]: the (i,j)-th element of the Gram matrix of A is defined as [9]: The Gram matrix is normalized such that = 1, ∀ = . The mutual coherence of A is determined by the maximum value of the off-diagonal elements of G.
The values of the mutual coherence are bounded in the interval of ≤ ( ) ≤ 1, with low bound defined as [9]: Instead of the maximum value of the off-diagonal elements of G, the former simulations show that the average of the mutual coherence is more related to the performance of the CS system. Thus, the other measurement, called average mutual coherence, is drawn as follow [9] ̅ ( ) = ∑ ∀( , ), ℎ ≠ with off-diagonal elements. In this paper, the distribution of the off-diagonal entries of normalized Gram matrix G is presented to give supporting reasons for the resulted simulation performance.

Case Study: Compression vs Measurement System
The intention of the presenting research is to evaluate and to analyse the impact of the mutual coherence into the reconstruction performance on both compression system and measurement system. It will be investigated whether the mutual coherence is consistently applied as necessary condition for both system. Reconstruction of Compression system means to reconstruct the complete signal or image from only few sampling of data or it is determined as compressed data. While, reconstruction on measurement system is reconstruction of the signal or image from set of measurement data which its dimension is normally much lower compared to the reconstructed one. A set of simulation had been set up as follow: On Figure 1 explains that compressed X-ray image reconstruction is used as the case study on compression system, while Electrical Capacitance Volume Tomography (ECVT) image reconstruction is used for case study on measurement system. The mutual coherence property between sensing matrix and dictionary is evaluated on its impact into reconstruction performance on both systems. Electrical Capacitance Volume Tomography (ECVT) is one of the alternative tomography modality which possess low energy, non-invasive, and portability characteristics [11]. It has been developing and used for some application in both industry and medical purposes [11][12][13]. One of development effort to elevate ECVT performance is on its reconstruction method [5,11,[14][15][16]. Refer to Equation (1), y on ECVT system is the capacitance measurement around the sensor's boundary, while x is the predicted perturbation inside the sensing domain, which is represented by permittivity value distribution. For sensing matrix Φ, for ECVT is defined deterministic as: where (= −∇ ) is the electrical field distribution vector when the source electrode in the ith pair is activated with voltage while the rest of the electrodes are grounded is the electrical field distribution vector when the detector electrode in the i-th pair is activated with voltage while the rest of the electrodes are grounded. 0 is the volume of the j-th voxel.

The Critical Study
In CS framework, the dimension of the measurement value is enable to be much lower compared to the dimension of the projected value as long as the projected value is sparse or nearly sparse in the original domain or in the transformed domain. In addition, the recovery or reconstruction process is feasible as long as the mutual coherence between the sensing matrix and the dictionary is low [3,7]. Random design of sensing matrix is normally used as its low mutual coherence properties subject to some dictionaries (sinus, cosines, curvelet based function) [7]. Compressive Sensing framework is utilized to reconstruct a signal or image from either the compression system or the measurement system.
Analysis presented on this section discuss the critical study of the mutual properties into the reconstruction performance. The analysis on the compression system is presented first and being followed by the analysis on the ECVT imaging performance as one of example on reconstruction from a measurement system. On Figure 2 below, the CS based reconstruction method is applied to conduct the intended study

Analysis and Evaluation of Mutual Coherence Properties on The Compression System
X-ray image database is taken as a case study to evaluate the movement of the mutual coherence properties into image reconstruction accuracy. The images are sampled into 50%, 10% of the data. The compressive sensing approach as proposed on Figure 2 is applied and the mutual coherence along with the PSNR is recorded as presented on Table 1. The qualitative measurement is presented on the image appearance on Figure 3. Table 1 presents the head-to-head mapping of Mutual coherence and PSNR. It intends to see the correlation between the mutual coherence and the reconstruction accuracy which represented by PSNR. Some random sensing design and dictionaries are applied. Some optimization approaches are also applied by using ETF (Equiangular Tight Frame) for the sensing matrix and KSVD for the dictionary. This optimization intends to decrease the mutual coherence value between the sensing matrix and dictionary. Statistic correlation test is conducted as presented on Table 2, to see the correlation between mutual coherence as independent variable into PSNR as the dependent variable.   that the smaller mutual coherence, the higher PSNR can be achieved. Higher PSNR indicates better reconstruction performance. Thus, the theory of the mutual coherence properties on sparse recovery [3,7] can be proven for compression system by this simulation result. The random design on the sensing matrix with some dictionary setting is working and supporting the established theory about the sensing matrix and dictionary design on sparse reconstruction. Section 4.2 evaluates the mutual coherence properties' movement subject to the reconstruction accuracy on measurement system. In addition, it also looks deeper on some aspects that affect the reconstruction performance besides the mutual coherence properties on solving reconstruction problem on measurement system.

Analysis and Evaluation of Mutual Coherence Properties on the Measurement System
Image Reconstruction on Electrical Capacitance Volume Tomography (ECVT) is used as an example of image reconstruction from a measurement system. It is sparse at least on the transform domain [5,8]. The ECVT image reconstruction problem forms under-determined linear system, which is indicated by much lower dimension of measurement value compared to the dimension of the projected value. Mathematically, the under-determined linear system can be solved by Compressive Sensing Approach.
The simulation is set up for measuring and evaluating the mutual coherence movement into the ECVT image reconstruction accuracy, which is represented by voxel coefficient of correlation (R). The ECVT deterministic sensing matrix design as formulated on Equation.10 is applied by their corresponding dictionaries. Table 3 below presents the value of mutual coherence and R for each of sensing matrix and dictionary assignment respectively. The correlation of the mutual coherence as the independent variable and the R as the dependent variable is evaluated using Pearson Correlation test, which is presented on Table 4. Based on presenting result on Table 3 and Table 4, it can be seen that there is no linear correlation pattern describing the movement of the mutual coherence, with its corresponding accuracy performance (R). The Pearson correlation value is -0.151, which closer to 0 rather than 1. It indicates the movement of mutual coherence does not affect significantly the accuracy of the ECVT Image reconstruction.
Mutual coherence properties is no longer necessary condition for having good sparse reconstruction performance on measurement system. However, every successful reconstruction always have small IOP Conf. Series: Journal of Physics: Conf. Series 1196 (2019) 012074 IOP Publishing doi:10.1088/1742-6596/1196/1/012074 8 mutual coherence. It means that, mutual coherence is sufficient condition for having good sparse reconstruction on measurement system.
Since the mutual coherence is no longer necessary condition for having sparse reconstruction performance on measurement system, there should be another parameter that can be used to characterize the necessary condition to be applied on the reconstruction's objective function. Below, some deeper analysis and evaluation on the sparse ECVT reconstruction is presented to find some properties that need to be concerned to achieve good sparse reconstruction on measurement system.

Deeper Analysis on the Sparse ECVT Reconstruction
A simulation is set up to evaluate the other potential property that influence the reconstruction performance. The method is designed as presented on Figure 4 below. Deterministic and Random based Sensing matrix are used as the sensing matrix design, DCT is assigned as the dictionary and OMP is assigned on the sparse reconstruction algorithm. Besides mutual coherence and R, we also evaluate the sparse error which represented by ‖Φ ‖ 2 , with E is formulated as Besides, the pattern on the multiplication of the sensing matrix and mutual coherence is also evaluated to see its correlation into the successful reconstruction performance The evaluation is presented on Table 5 for various number of dielectric contrast. Figure 5 up to Figure 7 present the image reconstruction appearance and the pattern of the multiplication between sensing matrix and dictionary for three (3) different sensing matrix design. The result presented on Table 5 shows that sparse error is more persistent to indicate the trend of the image reconstruction performance for all contrast dielectric. Lower sparse error will lead to higher reconstruction accuracy. In addition, the pattern of the multiplication between the sensing matrix and the dictionary is also observed as presented on

Conclusion
Critical evaluation and analysis on mutual coherence property of Compressive sensing framework into sparse reconstruction performance has been presented on this research paper. The evaluation is comprehensively done for both compression system and measurement system. X-ray compressed image reconstruction and ECVT image reconstruction are used as the cases study to represent both of the system respectively. The average mutual coherence is used to represent the parameter that describe the incoherence principle between sensing matrix and dictionary. While PSNR and R (Coefficient of Correlation) are used as the reconstruction performance parameter.
Simulation results show that the mutual coherence is proven to be a necessary condition for having good reconstruction performance on compression system. It can be shown by the value of Pearson correlation, -0.881, which is indicated strong negative linear relationship. It means that, the lower the mutual coherence value, the higher the reconstruction accuracy.
In another side, mutual coherence as necessary condition for having good reconstruction is not proven on measurement system. The results show that the mutual coherence as independent variable has no any linear relation with the reconstruction accuracy. It is indicated by the very low Pearson correlation value which is closed to zero, -0.15. However, it can be shown and guaranteed that for every successful reconstruction on measurement system, it always has low mutual coherence value. Thus, in measurement system, mutual coherence is a sufficient condition for having good reconstruction accuracy.
On the deeper analysis show that, sparse error and the pattern of the multiplication between sensing matrix and dictionary are more relevant to indicate the movement of the reconstruction accuracy. The two parameters should be considered to be attached on the objective function when Compressive Sensing model is adopted to solve reconstruction problem on a measurement system. In addition, sensing matrix design for reconstruction problem on measurement system, cannot be just simply random, but need to be deterministic which characterize the sensing process.