Information theoretic bounds for compressed sensing in SAR imaging

Compressed sensing (CS) is a new framework for sampling and reconstructing sparse signals from measurements significantly fewer than those prescribed by Nyquist rate in the Shannon sampling theorem. This new strategy, applied in various application areas including synthetic aperture radar (SAR), relies on two principles: sparsity, which is related to the signals of interest, and incoherence, which refers to the sensing modality. An important question in CS-based SAR system design concerns sampling rate necessary and sufficient for exact or approximate recovery of sparse signals. In the literature, bounds of measurements (or sampling rate) in CS have been proposed from the perspective of information theory. However, these information-theoretic bounds need to be reviewed and, if necessary, validated for CS-based SAR imaging, as there are various assumptions made in the derivations of lower and upper bounds on sub-Nyquist sampling rates, which may not hold true in CS-based SAR imaging. In this paper, information-theoretic bounds of sampling rate will be analyzed. For this, the SAR measurement system is modeled as an information channel, with channel capacity and rate-distortion characteristics evaluated to enable the determination of sampling rates required for recovery of sparse scenes. Experiments based on simulated data will be undertaken to test the theoretic bounds against empirical results about sampling rates required to achieve certain detection error probabilities.


Introduction
The Nyquist-Shannon sampling theorem states that perfect reconstruction of a signal is possible when the sampling frequency is greater than twice the maximum frequency of the signal being sampled. If sub-Nyquist sampling rates are used, the original signal's information may not be completely recoverable from the sampled signal [1]. Under certain conditions, compressed sensing [2][3][4][5] (or compressive sampling, CS) states that certain signals and images may be recovered from far fewer samples or measurements than that required by the Nyquist-Shannon sampling theorem.
According to the theory of CS, with an incoherent linear projection acquiring an efficient representation of a sparse or compressible signal, the signal can be reconstructed by solving an inverse problem either through linear programming or greedy pursuit [2]. Suppose that a vector x of length n is known to have a small number k of nonzero entries, but the values and locations of the nonzero entries are unknown and must be estimated from a set of m noisy linear projections (or samples) given by the vector Y : (1) where A is a known mn  measurement matrix and b is additive white Gaussian noise (CH1 in [6]).
The problem of determining which entries in the unknown signal x are nonzero is known as support recovery (sparsity pattern recovery) (CH1 in [6]). With the development of CS theory, a new synthetic aperture radar (SAR) imaging modality [7][8][9][10][11] has been proposed that can obtain accurate representation of signals with much fewer samples or measurements than traditional matched filter (MF) methods. An important question in CS-based SAR system designs concerns sampling rates necessary and sufficient for exact or approximate recovery of sparse signals.
Much previous work ( [1], [13][14][15][16][17][18][19][20][21][22] ) has focused on necessary and sufficient conditions for exact or approximate recovery of the sparsity pattern. In the literature, bounds of measurements (or sampling rate) in CS have been proposed from perspectives including information theory. However, these information-theoretic bounds need to be reviewed and, if necessary, validated for CS-based SAR imaging, as there are various assumptions made in the derivations of lower and upper bounds on sub-Nyquist sampling rates, which may not hold true in CS-based SAR imaging.
This paper seeks to provide an overview of theoretic fundamentals that can be used to provide anchor points for determining CS sampling rates. For this, CS theorems and classic information theoretic concepts and quantities including channel capacity and Fano's inequality are described with respect to the bounds on sampling rates necessary and/or sufficient for sparsity pattern recovery. Thus, a SAR measurement system can be modeled as an information channel, with channel capacity and rate-distortion characteristics evaluated to enable the determination of sampling rates required for recovery of sparse scenes. SAR imaging is briefly described to furnish its implementation in the language of CS. This is followed by experiments based on simulated data, which were undertaken to test the theoretic bounds on sampling rates required to achieve certain detection error probabilities. Finally, some concluding remarks are given.

CS and Restricted isometry property
x is an object we wish to reconstruct, Y are available measurements, b is noise, and A is a known mn  matrix with mn  . We define this kind of systems as underdetermined linear system, since the system has more unknowns than equations, and thus it has either no solution if b is not in the span of the columns of the matrix A , or infinitely many solutions [23]. Here, we are interested in this underdetermined case with fewer equations than unknowns ( mn  ), and ask whether it is possible to reconstruct x with good accuracy. As such, the problem is of illposed but suppose now that x is known to be sparse or nearly sparse in the sense that it depends on a smaller number of unknown parameters. This premise radically changes the problem, making the search for solutions feasible [14]. In fact, it has been shown that solving the convex problem P2: (3) will recover an unknown sparse object with an error at most proportional to the noise level provided that 1) x is sufficiently sparse and 2) the matrix A obeys a condition known as the restricted isometry property[14] introduced in Equation(4). Theorem 1.1 (Theorem 1.1 of [16]) Define the S-restricted isometry constant [15] S  of A as the smallest quantity such that  are restricted isometry constants of order 3S and 4S , respectively), then for any signal 0 x supported on If considering how many measurements are necessary to achieve the RIP, we can establish a simple lower bound if we ignore the impact of  (restricted isometry constant) and focus only on the dimensions of the problem (n, m, and k) ([17], Section A.1). Theorem 1.2 (Theorem of 3.5, Lemma 1.

of [24]) Let
A be an mn  matrix that satisfies the RIP of order 2k with constant

Information theoretic bounds 2.2.1. Rate distortion function and channel capacity
When an unknown signal is measured by a noisy linear measurements system as displayed in Equation (2), approximate rather than exact recovery is often what is possible as the measurements are imperfect. The rate distortion function [25]     I RD is the lower bound for information transmission rate, and also the least average information content needed to recover the source signal within the given distortion.
In order to determine how many measurements is necessary to recover the unknown signals, Sarvotham, Baron, and Baraniuk [20] applied information theory, by which a linear measurement system can be seen as a combination of encoder and signal transmitter while the recovery algorithm is the decoder to supply the approximation of the original signal. Obviously, the measurement equation shown in Equation (2) can be regarded as a channel which transfers the signals so that we can get information about the signals (recovery of signals) from measurements.
In [20], the amount of information that can be extracted from the CS measurements is investigated, and determined by the capacity of the measurement channel. Having upper bounded the information contained in the measurements, the authors investigate the minimum information needed to reconstruct the signal with distortion where    X D is the distortion level, and SNR stands for signal to noise ratio [20].
Consider a k-sparse signal where the spikes have uniform amplitude, the lower bound on number of measurements [20] is:

Fano's inequality
Suppose that we know a random variable Y and we wish to guess the value of a correlated random variable X , for any estimator X such that   X Y X forms a Markov chain. Fano's inequality [25] relates the probability of error in guessing the random variable X (  [25], the probability of error in reconstructing the support S  correlates with the conditional entropy between S  and   , YA , and further the sampling rate mn determined by the measurements matrix A . In [6](Ch3), lower bounds on the fundamental sampling rate distortion-function are derived. These bounds consist of necessary conditions which apply generally to any possible recovery algorithm.

CS-based SAR
SAR can obtain a two-dimensional image of the observed scene. The conventional SAR imaging algorithm based on Matched Filter theory limits the resolution by the transmitted signal bandwidth and the antenna length [10]. When applying CS to SAR imaging, we can utilize radar measurements significantly fewer than what are required of traditional radar imaging algorithms to obtain an accurate representation for further processing. CS can eliminate the need for the Matched Filter in the radar receiver, and reduce the required receiver A/D conversion bandwidth so that CS-based SAR systems can operate at information rates that are potentially lower than Nyquist rates, which are often extremely high for fine-resolution systems [7]. Suppose a SAR system transmits a linear frequency modulated (LFM) signals as where n is the low-time, For a sparse scene, we assume that  is k sparse when only k ( kN ) of its coefficients is nonzero or greater than zero. According to CS theory, it is possible to recover the sparse signal with only a small number of samples of measured signal S .

Experiments
In the experiment, the CS based SAR imaging system was simulated. The original signal comes from sparse binary sources [26] where the spikes have uniform amplitude 1, and then multiplied by a measurement matrix which is constructed by choosing M rows (M varies from 20 to 600 by a step of 20) from a 1000 by 1000 convolutional matrix, whose baseband is 1-D chirp signal, with bandwidth 50MHz, pulse duration s  2 , sampling rate 1.2 times of Nyquist rate. The measurements are corrupted with additive Gaussian noise. To reconstruct the spike signal, a common used algorithm called LASSO [27] was used. As the original signal is complex-valued, the real and the imaginary part of the reconstituted signal and recovered support (sparsity pattern) are displayed in Figure 1(d),1(e),1(g) and 1(h), respectively. Figure 1 shows the experiment results with a simulated CS-SAR system. For this, a total of 200 measurements of the 1-dimensional signal (a vector of length 1000) were used to reconstruct the original signal with 30 spikes. With the signal-noise ratio (SNR) set to 10 dB, the support of the spike signal was recovered with a distortion [6] (error probability) of 0.06. Figure Figure 2, as m is increasing, there is a transition from higher to lower distortion errors.
Overlaid on the intensity plots 2(a) 2(b) 2(c) are curve representing what would be evaluated for the lower bounds (for numbers of measurements) on the basis of Equations (6), (9), and (13a), respectively. As expected, the necessary measurement represented by the curves on the plot is less than that needed in practice. There are two reasons for this: 1) the lower bounds retrieved from RIP or information theory is a general result applicable to a few special types of measurement matrices instead of only the one used in the experiment, 2) the channel capacity and rate distortion function (tools used to get information theoretic bounds) are the limits that may be realized under ideal environments.

Conclusion and discussion
We have considered the problem of approximate recovery of sparse signals from noisy random linear measurements, which are easily obtained by multiplying the signal with a convolutional matrix used in a SAR imaging system. Numbers of measurements required for sparsity pattern recovery were reviewed from the perspectives of CS theorems and information theory. The experiment on a simulated SAR system suggests that CS theory (random projections on a convolution matrix) is extensible to SAR imaging. Preliminary results reveal the commonality and discrepancy between theoretic and empirical results about the required sampling rates in CS-based SAR imaging system. Further research is needed to test the applicability of CS sampling rates on SAR imaging in an operational setting where continuous signal amplitudes, complex SNR and other parameters, a priori unknown sparsity patterns, and other complications are likely involved.  Figure 2. Errors in sparsity support detection for n= 1000 and varying values of k, m: (a) red curve representing the errors calculated by Equation (6) with C = 0.5468, (b) green curve by Equation (9) with SNR = 10 dB, and (c) blue curve by Equation (13a) with distortion <= 20% and SNR = 10 dB (the gray scale indicates 0 (black) through100 (white) percent error probability).