Bounding network spectra for network design

The identification of the limiting factors in the dynamical behavior of complex systems is an important interdisciplinary problem which often can be traced to the spectral properties of an underlying network. By deriving a general relation between the eigenvalues of weighted and unweighted networks, here I show that for a wide class of networks the dynamical behavior is tightly bounded by few network parameters. This result provides rigorous conditions for the design of networks with predefined dynamical properties and for the structural control of physical processes in complex systems. The results are illustrated using synchronization phenomena as a model process.


Introduction
Complex dynamical systems are high dimensional in nature. The determination of simple general principles governing the behavior of such systems is an outstanding problem which has attracted a great deal of attention in connection with recent network and graph-theoretical constructs [1,2]. Here I focus on synchronization, which is the process that has attracted most attention, and use this process to study the interplay between network structure and dynamics. Synchronization is a widespread phenomenon in distributed systems, with examples ranging from neuronal to technological networks [3]. Previous studies have shown that network synchronization is strongly influenced by the randomness [4,5], degree (connectivity) distribution [6], correlations [7,8], and distributions of directions and weights [9,10] in the underlying network of couplings. But what is the ultimate origin of these dependences?
In this paper, I show that these and other important effects in the dynamics of complex networks are ultimately controlled by a small number of network parameters. For concreteness, I focus on complete synchronization of identical dynamical units [11], which has served as a prime paradigm for the study of collective dynamics in complex networks. In this case, the synchronizability of the network is determined by the largest and smallest nonzero eigenvalues of the coupling (Laplacian) matrix. My principal result is that, for a wide class of complex networks, these eigenvalues are tightly bounded by simple functions of the weights and degrees in the network. The quantities involved in the bounds are either known by construction or can be calculated in at most O(kN) operations for networks with N nodes and 2kN links, whereas the numerical calculation of the eigenvalues of large networks would be prohibitively costly since it requires in general O(N 3 ) operations even for the special case of undirected networks. These bounds are in many aspects different from those known in the literature of graph spectral theory [12] and are suitable to relate the physically observable structures in the network of couplings to the dynamics of the entire system.
The eigenvalue bounds are then applied to design complex networks that display predetermined dynamical properties and, conversely, to determine how given structural properties influence the network dynamics. This is achieved by exploring the fact that the quantities used to express the bounds have direct physical interpretation. This leads to conditions for the enhancement and suppression of synchronization in terms of physical parameters of the network. The main results also apply to a class of weighted and directed networks and are thus important to assess the effect of nonuniform connection weights in the synchronization of real-world networks [13]. The proposed method for network design is based on a relationship between the eigenvalues of a substrate network that incorporates the structural constraints imposed to the system and those of weighted versions of the same network. This method is thus complementary to other recently proposed approaches for identifying [14,15,16] or constructing [17,18,19] networks with desired dynamical properties.
The paper is organized as follows. In Sec. 2, I define the class of networks to be considered and announce the main result on the eigenvalue bounds, which is proved in the appendix. In Sec. 3, I discuss an eigenvalue approach to the study of network synchronization. The problem of network design and the impact of the network structure on dynamics is considered in Secs. 4 and 5, respectively. Concluding remarks are incorporated in the last section.

Eigenvalue Bounds
The dynamical problems considered in this paper are related to the extreme eigenvalues of the Laplacian matrix. This section concerns the bounds of these eigenvalues.

Class of Networks
Most previous studies related to network spectra and dynamics have focused on unweighted networks of symmetrically coupled nodes. In order to account for some important recent models of weighted and directed networks, here I consider a more general class of networks. The networks are defined by adjacency matrices A satisfying the condition that is a symmetric matrix, where k i ≥ 1 is the degree of node i, factor S i = j A ij > 0 is the total strength of the input connections at node i, and N is the number of nodes in the network. According to this condition, the in-and out-degrees are equal at each node of the network, although the strengths of in-and out-connections are not necessarily the same. MatrixÂ = (Â ij ) is possibly weighted:Â ij > 0 if there is a connection between nodes i and j = i andÂ ij = 0 otherwise, where jÂ ij = k i because of the normalization factor S i /k i . The class of networks defined by Eq. (1) includes as particular cases all undirected networks (both unweighted and weighted) and all directed networks derived from undirected networks by a node-dependent rescaling of the input strengths. The dominant directions of the couplings are determined by S i /k i and the weights by both S i /k i andÂ, where S i /k i defines the mean andÂ the relative strength of the individual input connections at node i. The usual unweighted undirected networks correspond to the case whereÂ is binary and S i = k i for all the nodes. The study of this class of networks is motivated by both physical and mathematical considerations. From the mathematical viewpoint, I show in the appendix that the conditions imposed to matrix A guarantee that the corresponding coupling matrices are diagonalizable and have real spectra. Physically, this coupling scheme is general enough to reproduce the weight distribution of numerous realistic networks [13] and to show how the combination of topology, weights, and directions affect the dynamics. Indeed, the weighted and directed networks comprised by the adjacency matrix A in (1) include important models previously considered in the literature, such as the models where S i = 1 ∀i, used to study coupled maps [20,21] and to address the effects of asymmetry and saturation of connection strengths [9,10]. It also includes the models introduced in Refs. [7,22,23], where the connection strengths depend on the degrees of the neighboring nodes, and other models reviewed in Ref. [24]. In what follows, I consider the general class of networks defined by Eq. (1) with the additional assumption that each network has a single connected component.

Coupling Matrices
The coupling matrix relevant to this study is the Laplacian matrix G = (G ij ), where The Laplacian matrix can be written as is the matrix of degrees, and L = (δ ij k i −Â ij ). As shown in the appendix, matrices G andĜ are diagonalizable and all the eigenvalues of G andĜ are real. For connected networks where all the input strengths S i are positive, as assumed here, the eigenvalues of matrices G,Ĝ, and S can be ordered as respectively. The strict inequalities λ 2 > 0 and µ 2 > 0 follow from Eq. (A.7), which expresses λ 2 (and also µ 2 if one takes S i = 1 ∀i) as a sum of nonnegative terms with at least one of them being nonzero when the network is connected. The identities λ 1 = µ 1 = 0 are a simple consequence of the zero row sum property of matrices G and G.

Extreme Eigenvalues: Bounds for Arbitrary Network Structure
I now turn to the analysis of the eigenvalues of the Laplacian matrix G. I use k min , k max and k to denote the minimum, maximum and mean degree in the network. The minimum and maximum input strengths are denoted by S min = min i {S i } and S max = max i {S i }, respectively, while k min * is used to denote the minimum degree among the nodes with input strength S min . I first state the following general theorem.
Theorem: The largest and smallest nonzero eigenvalues of matrices G,Ĝ, and S are related as where h = ( i k i /S i )/ ( i k i )( i k i /S 2 i ), g = (1 − β) −1 , and β = (k min * /S min )/( i k i /S i ), for any network with adjacency matrix satisfying (1). This theorem is important because it relates the desired and usually unknown eigenvalues of Laplacian matrix G with the input strengths and the often approximately known eigenvalues of the normalized Laplacian matrixĜ. In general, one has µ 2 ≤ N/(N − 1) ≤ µ N ≤ 2, which follows as a simple generalization of the results in Ref. [25] to the weighted and directed networks defined by Eq. (1). Physically, the eigenvalues µ 2 and µ N are related to relaxation rates [10], while ν 1 and ν N are just the input strengths S min and S max , respectively. A special case of the theorem was announced in Ref. [23]. The theorem is proved in the appendix. In the remaining part of the paper I explore applications of the theorem.

Synchronization Problem
In this section, networks of identical oscillatory systems are used to discuss how the coupling cost and stability of synchronous states are expressed in terms of the eigenvalues considered in the previous section.

Oscillator Network
Consider a network of N diffusively coupled dynamical units [11] modeled bẏ where the first term on the r.h.s. describes the dynamics of each unit, while the second and accounts for the couplings between different units: H(x j ) is the signal function that describes the influence of unit j on the units coupled to j and σ ≥ 0 is the overall coupling strength. The adjacency matrix A = (A ij ) satisfies (1) and is related to the Laplacian matrix G = (G ij ) through Eq. (2).
Completely synchronized states {x i (t) = s(t), ∀i |ṡ = F(s)} are always solutions of system (8). Since the Laplacian matrix is diagonalizable, the stability of these synchronous states can be studied using the standard master stability framework [11] (see also [14,17]). This reduces the variational equations of system (8)

Stability and Coupling Cost
In a broad class of oscillatory dynamical systems, function Λ is negative in a single interval (α 1 , α 2 ) [5,10,11]. The synchronous state is then stable for some σ if the eigenvalues of the Laplacian matrix G satisfy the condition [5] The r.h.s. of this inequality depends only on the dynamics while the l.h.s. depends only on the structure of the network, as indicated in the brackets. The smaller the ratio of eigenvalues R the larger the number of dynamical states for which condition (9) is satisfied. Moreover, when this condition is satisfied and α 2 /α 1 is finite, the smaller the ratio R the larger the relative interval of the coupling parameter σ for which the corresponding synchronous state is stable. When condition (9) is satisfied, the eigenvalues λ 2 and λ N are related to the synchronization thresholds as where σ min and σ max are the minimum and maximum coupling strengths for stable synchronization, respectively. These relations will be explored in the design of networks with predefined thresholds in Sec. 4. This characterization is not complete without taking into account the cost involved in the coupling. The coupling cost required for stable synchronization was defined in Refs. [9,10] as the sum of the coupling strengths at the lower synchronization threshold, This cost function can be expressed in terms of eigenvalues of the Laplacian matrix [17], and depends separately on the dynamics (α 1 ) and structure ( j λ j /λ 2 ) of the network. This can be used to derive an upper bound for C expressed in terms of the ratio R: Therefore, ratio R is a measure of the synchronizability and cost of the network, with the interpretation that the network is more synchronizable and the cost is more tightly upper bounded when R is smaller. The synchronization problem is then reduced to the study of eigenvalues of the Laplacian matrix G.

Design of Networks with Predefined Synchronization Thresholds
In this section, I show how the theorem of Sec. 2 can be used to design large networks with predetermined eigenvalues λ 2 and λ N . In the synchronization problem of Sec. 3, this corresponds to the design of networks with predetermined lower (σ min = α 1 /λ 2 ) and upper (σ max = α 2 /λ N ) synchronization thresholds.

Network Design
Given an arbitrary substrate network of N nodes and known eigenvalues µ N and µ 2 , the bounds in Eqs. (6) and (7) can be used to generate networks of eigenvalues for a given substrate network with normalized eigenvalues µ N and µ 2 . In (a), for given S max , the blue area (bottom) represents the uncertainty in λ N due to the factor µ N and the triangular line (top) accounts for any inaccuracy in the determination of µ N . In (b), for given S min , the black, blue, gray and black areas (top to bottom) represent the uncertainty in λ 2 due to the factors g, µ 2 , h, and any inaccuracy in the determination of µ 2 , respectively. Here, g + and h − are used to indicate the maximum and minimum of g and h, respectively, in the given interval of S min . |µ N − 1| and |g − µ 2 h|, respectively. Here, λ N and λ 2 denote the desired values and λ * N and λ * 2 denote the resulting eigenvalues, which have some uncertainty. This procedure is illustrated in Fig. 1 and can be used to systematically design robust networks with tunable extreme eigenvalues.
The rationale here is that the substrate network is chosen to incorporate topological constraints relevant to the problem, such as the nonexistence of links between certain nodes or a limit in the number of links, and that the extreme eigenvalues of the normalized LaplacianĜ of this network are calculated beforehand. Then, by adjusting the minimum and maximum input strengths S min and S max , one can define new networks with the same topology but with the desired extreme eigenvalues for the Laplacian matrix G.
More specifically, if µ N is known, one can adjust the largest input strength using Eq. (6) to obtain a new network with λ * N in the interval [λ N , λ N × µ N ] by setting S max = S * max ≡ λ N [see Fig. 1(a)]. Likewise, if µ 2 is known, one can use Eq. (7) to adjust the minimum input strength and generate a new network with λ * 2 within a desirable interval [λ 2 ×µ 2 h, λ 2 ×g] by taking S min = S * min ≡ λ 2 [see Fig. 1(b)]. Naturally, the usefulness of this construction will depend on how close to 1 are the eigenvalue µ N and µ 2 , and how close to 1 are kept h and g as the weights are changed. The former condition can be justified for most networks in the usual ensembles of densely connected random networks and also in ensembles of sparse networks with large mean degree k [25]. Note that this approach can be effective even when µ N and µ 2 are only approximately known, as represented by the upper and lower black diagonal lines in Fig. 1(a) and (b), respectively. The last observation is relevant precisely when µ N and µ 2 are estimated from an ensemble distribution or through any other probabilistic procedure. Importantly, because λ 2 is mainly controlled by S min and λ N by S max , both eigenvalues can be adjusted simultaneously. In the synchronization problem, this can be used to define networks with predetermined synchronizability R and predetermined upper bound for the coupling cost C. Moreover, this construction is not unique, that is, there are multiple choices of the substrate network and of the assignment of weights that will lead to the same pair of predefined eigenvalues λ 2 and λ N . This freedom can be explored to increase robustness against structural perturbations and to control the uncertainty by keeping h large and g small.

Numerical Example
Consider unweighted Erdős-Rényi networks, generated by adding with probability p a link between each pair of N given nodes [26]. As shown in the histograms of Fig. 2, the eigenvalues µ 2 and µ N are narrowly distributed close to 1 even for relatively small and sparse networks. Such networks can thus be used as substrate networks to generate, with good accuracy, new networks of predefined eigenvalues λ 2 and λ N by reassigning the input strengths S min and S max , respectively.
While a single realization of the substrate network and a deterministic assignment of input strengths S min and S max would suffice to generate the desired networks, the robustness of the proposed procedure becomes more visible if one considers various independent random constructions. For this purpose, I consider random realizations of the substrate network and assume that, for each such realization, the input strength of each node is assigned with equal probability to be either S min or S max . Figure 3 shows the numerically computed eigenvalues λ 2 and λ N , and the respective bounds, as functions of S min and S max . This figure is a scattered plot with 100 independent realizations of the substrate networks (and assignments of input strengths) for each choice of S min and S max . As shown in the figure, except for the lower bound of λ 2 , which exhibits observable dependence on the specific network realization, the distributions of the eigenvalues and bounds are narrower than the width of the lines in the figure. In addition, the numerically computed values of λ 2 and λ N are tightly bounded by the lower and upper limits in Eqs. (6)- (7). The difference between the bounds of λ N in Fig. 3(b) is thinner than the width of line. Moreover, as S min (S max ) is varied for fixed S max (S min ) in Fig. 3(a) (Fig. 3(b)) , the value of λ N (λ 2 ) remains nearly constant, as shown in the insets. Thus, by varying both S min and S min , one can design networks where both λ 2 and λ N are predetermined. Figure 4, shows the result of such a construction for the ratio of eigenvalues R. Note that if all the input strengths S i are re-scaled by a common factor α, the terms ν N , λ N , and ν N µ N in Eq. (6) as well as the terms ν 1 µ 2 h, λ 2 , and ν 1 g in Eq. (7) will change by the same factor α. Therefore, the ratio R and corresponding bounds do not change if, in our simulations, both S min and S max are re-scaled by a common factor.
Remark: If no constraints are imposed to the topology of the network other than the number N of nodes, then one could easily construct networks having exactly any given set of eigenvalues 0 = λ 1 < λ 2 ≤ . . . ≤ λ N and any given set of orthonormal eigenvectors u 1 , · · · , u N , where u T 1 = (1/ √ N, . . . , 1/ √ N ). The network satisfying this conditions is defined by the symmetric Laplacian G = UdU T , where d is the diagonal matrix of eigenvalues Note that matrix G is indeed a well-defined Laplacian satisfying the zero row sum condition.

Impact of the Network Structure on Synchronization
Equations (6) and (7) can be used to address the influence of the network structure on the dynamics. In particular, they imply that S max S min Therefore, under rather general conditions, the synchronizability of the network is strongly limited by S max /S min and µ N /µ 2 . The first ratio depends on the distribution of weights while the second also depends on the topology of the network. The bounds in Eq. (14) are valid for any network satisfying condition (1), but are tighter for classes of networks with µ N /µ 2 , g and h closer to 1. In this section I focus on large random networks, which forms one such class of networks.

Synchronizability of Random Networks
For concreteness, consider random networks for which the normalized matrixÂ is unweighted. That is, random networks which are either unweighted or whose weights are factored out completely in Eq. (1). For these networks, one can invoke the known result from graph spectral theory [25] that the expected values of the extreme eigenvalues of G approach 1 as µ N = 1 + O(1/ √ k) and µ 2 = 1 − O(1/ √ k) for large mean degree k. This behavior has been shown to remain valid for networks with quite general expected degree sequence [27] and to be consistent with numerical simulations on various models of growing and scale-free networks [9,10], even when the networks are relatively small and only approximately random insofar as k min ≫ 1. In addition, the distribution of the eigenvalues across the ensemble of random networks becomes increasingly peaked around the expected values as the size of the networks increases [27,28]. Furthermore, for most realistic networks, h is bounded away from zero and g approaches 1 for large N [it can be replaced by 1 if the conditions in remark 1 (appendix) apply]. For unweighted networks, in particular, h = (kk −1 ) −1/2 and g = N/(N − 1), wherek −1 is the average of 1/k i in the network. Therefore, for a wide class of complex networks, the eigenvalues λ N and λ 2 are mainly determined by S max , through S max ≤ λ N ≤ S max µ N , and by S min , through S min µ 2 h ≤ λ 2 ≤ S min g, respectively.
In the case of unweighted (and undirected) networks, the input strengths are determined by the degrees of the nodes and S max /S min = k max /k min . Thus, the bounds in Eq. (14) can be used to assess the effect of the degree distribution. As a specific example, consider random scale-free networks [29,30] with degree distribution P (κ) = cκ −γ for κ ≥ k min and γ > 2, where 1/c = N −1 κ=k min κ −γ ≈ k −γ+1 min /(γ −1) is a normalization factor. From the condition N ∞ kmax P (κ)dκ = 1 [31], one has k max /k min ≈ N 1/(γ−1) , which leads to for large N and k min [32]. This simple scaling for the expected value of R explains the counter-intuitive results about the suppression of synchronizability in networks with heterogeneous distribution of degrees reported in Ref. [6]. Random scale-free networks were found to become less synchronizable as the scaling exponent γ is reduced, despite the concomitant reduction of the average distance between nodes [33] that could facilitate the communication between the synchronizing units [6]. Equation (14) shows that this effect of the degree distribution is a direct consequence of the increase in the heterogeneity of the input strengths, characterized by S max /S min = k max /k min . Equation (15) predicts this effect as a function of both the scaling exponent γ and the size N of the network. In particular, this equation shows that scale-free networks become more difficult to synchronize as N increases and this is again because S max /S min ≈ N 1/(γ−1) increases. On the other hand, synchronizability increases as γ is increased and becomes independent of the system size for γ = ∞, indicating that networks with the same degree for all the nodes are the most synchronizable random unweighted networks (see also Ref. [34]).
In the more general case of weighted networks, the input strengths are not necessarily related to the degrees of the nodes. An important implication of Eq. (14) is that, given a heterogeneous distribution of input strengths S i in Eq. (1), the synchronizability of the network is to some extent independent of the way the input strengths are assigned to the nodes of the network, rendering essentially the same result whether this distribution is correlated or not with the degree distribution. In both cases, synchronizability is mainly determined by the heterogeneity of the input strengths S max /S min and the mean degree k. In particular, synchronizability tends to be enhanced (suppressed) when the mean degree k is increased (reduced) and when the ratio S max /S min is reduced (increased). This raises the interesting possibility of controlling the synchronizability of the network by adjusting these two parameters, which was partially explored in Sec. 4.

Structural Control of the Dynamics
As a specific example of control, consider a given random network with arbitrary input , where the topology of the network is kept fixed and the input strengths are redefined as with θ regarded as a tunable (control) parameter. For large k, synchronizability is now mainly determined by max i,j {S ′ i (θ)/S ′ j (θ)} = (S max /S min ) θ . Within this approximation, synchronizability is expected to reach its maximum around θ = 0, quite independently of the initial distribution of input strengths S i and the details of the degree distribution. This generalizes a result first announced in Ref. [9], namely that networks with good synchronization properties tend to be at least approximately uniform with respect to the strength of the input signal received by each node (but see remark below).
These optimal networks have interesting properties. For θ = 0, all the nodes of the network have exactly the same input strength. Thus, if nodes i and j are connected, the strength of the connection from j to i scales as 1/k i , while the strength of the connection from i to j scales as 1/k j . This indicates that, unless all the nodes have exactly the same degree, the networks that optimize synchronizability for that degree distribution are necessarily weighted and directed. Moreover, if k j > k i , the strength ∝ 1/k i of connection from node j to node i is larger than the strength ∝ 1/k j of the connection from node i to node j. Therefore, in the most synchronizable networks, the dynamical units are asymmetrically coupled and the stronger direction of the connections is from the nodes with higher degrees to the nodes with lower degrees. The asymmetry and the predominance of connections from higher to lower degree nodes is a consequence of the condition that nodes with different degrees have the same input strength, a condition that introduces correlations between the weights of individual connections and the topology of the network and that has been observed to have similar consequences in other coupling models [7,24]. These results combined with the interesting recent work of Giuraniuc et al. [35] on critical behavior suggest that, in realistic systems, the properties of individual connections are at least partially shaped by the topology of the network.
Remark: The above analysis shows that for the networks satisfying the condition in Eq. (1), R is more tightly bounded close to the optimal value R = 1 when the distribution of input strengths S i is more homogeneous. Indeed, the bounds in Eq. (14) leave little room for the improvement of synchronizability by changing the weights of individual links or the way the nodes are connected if S max /S min is not reduced. For classes of more general directed networks, however, one can have highly synchronizable networks with a heterogeneous distribution of S i . To see this, consider the set of most synchronizable networks among all possible networks, which is precisely the set of networks with R = 1 and eigenvalues λ 2 = · · · λ N ≡ λ > 0. As shown in Refs. [14,17], if the Laplacian matrix G is diagonalizable, then the networks with R = 1 are those where each node either has output connections with the same strength to all the other nodes (and at least one node does so) or has no output connections at all. From this and the zero row sum property of the Laplacian matrix, it follows that S i = j =i a j = λ − a i , where a i ≥ 0 is the strength of each output connection from node i. Accordingly, the input strength S i is upper bounded by λ, but not necessarily the same for all the nodes. In particular, since the strengths of the output connections can have any values a i ≥ 0 (as long as at least one is non-zero), in this case there is no lower limit for S i and the ratio S max /S min can be arbitrarily large despite the fact that R = 1. Therefore, even when the spectra is real, strictly directed networks can be fundamentally different from the directed networks considered here [36].

Concluding Remarks
I have presented rigorous results showing that the extreme eigenvalues of the Laplacian matrix of many complex networks are bounded by the node degrees and input strengths, where the latter can be interpreted as the weighted in-degrees in the networks. These results can be used to predict and control the coupling cost and a number of implications of the network structure on the dynamical properties of the system, such as its tendency to sustain synchronized behavior. I have shown here that these results can also be used to design networks with predefined dynamical properties.
While I have focused on complete synchronization of identical units, the leading role of S max /S min and k revealed in this work also provides insights into other forms of synchronization. In particular, it seems to help explain: the suppressive effects of heterogeneity in the synchronization of pulse-coupled [37] and non-identical oscillators [7]; the dominant effect of the mean degree in the synchronization of time-delay systems with normalized input signal [38]; and the dominant effect of the degree in the synchronization of homogeneous networks of bursting neurons [39]. The scale-free model of neuronal networks considered in Ref. [40], which was shown to generate large synchronous firing peaks, is also consistent with (an extrapolation of) the results above. Indeed, the networks in that model are scale free only with respect to the out-degree distribution and are homogeneous with respect to the in-degree distribution. Therefore, the results presented here may serve as a reference in the study of more general systems, including those with heterogeneous dynamical units [41,42,43,44,45]. In general, the impact of the network structure will change both with the specific synchronization model and with the specific question under consideration, and an important open problem is to understand how it changes.
Finally, since the Laplacian eigenvalues also govern a variety of other processes [46,47], including the relaxation time in diffusion dynamics [10], community formation [48], consensus phenomena [49], and first-passage time in random walk processes [50], the results reported here are also expected to meet other applications in the broad area of dynamics on complex networks, particularly in connection with network design in communication and transport problems.

Acknowledgments
The author thanks Dong-Hee Kim for valuable discussions and for reviewing the manuscript.

Appendix. Proof of the Theorem
In what follows I use the notation that, if X is a N × N matrix with eigenvalues α 1 ≤ α 2 ≤ · · · ≤ α N , then v X i denotes a normalized eigenvector of eigenvalue α i . The proof of the theorem is divided in 6 steps.  3) and (4), and the corresponding eigenvectors can be chosen to form orthonormal bases [51].
Step 2: The diagonalizability of matrices G andĜ, a condition invoked in the rest of this appendix, can be demonstrated as follows. Matrix Q is symmetric and hence has a set of orthonormal eigenvectors {v Q i } N i=1 . Then, from the identity G = S 1/2 D −1/2 QS −1/2 D 1/2 , and the fact that S 1/2 D −1/2 is nonsingular, it follows that {S 1/2 D −1/2 v Q i } N i=1 forms a set of N linearly independent eigenvectors of G. This implies that G is diagonalizable. From the special case S = I, it follows that the same holds true forĜ.
Step 3: The upper bound of λ N in Eq. (6) follows immediately from where . is the usual Euclidean norm.
Step 4: The lower bound of λ N in Eq. (6) is derived from Gv ≥ Gv , where m 1 and m 2 are the Lagrange Multipliers. This system of equations can be solved for m 2 = max | Sv, v H 1 | under the corresponding constraints by taking i of Eq. (A.13) multiplied by v i , √ k i /S i , and √ k i /S 2 i , respectively. The result is . (A.14) The lower bound in Eq. (7) follows from Eqs. (A.9), (A.10), and (A.14), and this concludes the proof of the theorem.