Selection pressure transforms the nature of social dilemmas in adaptive networks

Many biological systems, and especially human societies, show persistent cooperative patterns. The evolution of such patterns is often studied using evolutionary game theory (EGT) [1]–[4]. This framework models interactions between the individuals of a population in terms of simple games. Successful behavior—success being measured in terms of game payoff—will be imitated and spreads in the population. The prototypical game to represent dilemmas of cooperation is the conventional prisoner's dilemma [5]. In this game, a cooperative act goes at a cost c to the cooperator while conferring a benefit b to another individual (assuming b>c). Defectors receive the benefits without spending any costs and are therefore expected to have an evolutionary advantage over cooperators.

However, given the omnipresence of prosocial behavior, certain conditions exist under which cooperation becomes viable [4], [6]–[12]. It has, for instance, been recognized that the topology of the network along which individuals interact and reproduce/imitate affects drastically the evolutionary chances of cooperators [13]–[42] (see Szabó and Fáth for a review [43]). The individuals also shape and reshape their social environment themselves and are, at least partially, responsible for the specific features that characterize their social network [44]–[47]. The process of network reshaping is often coupled with the strategy dynamics: the behavior of an individual influences his social position and vice versa [48]. Networks exhibiting such feedback loops provide sophisticated examples of adaptive networks [49]. Several authors have studied the functioning of such networks in the context of cooperation [50]–[65] (see [66] for a recent review). In general, cooperation has been shown to emerge more easily when interactions that benefit both partners last longer than interactions where one partner is exploited by the other [55, 56, 60].

In structured populations, the outcome of the evolutionary process depends also on the update rule, which dictates how strategies evolve from one generation to the next [67]. The selection pressure enters this update rule, and in EGT this is no exception, also being called intensity of selection [68]. A maximal selection strength implies that individuals only imitate those with a higher game payoff. Reducing the selection strength increases the amount of noise in the imitation process, until, ultimately, evolution becomes a purely random process (neutral selection or random drift). The impact of the selection pressure on the evolutionary dynamics of a finite, well-mixed population is well understood by now [68]–[71]. Additional effects may, however, arise in adaptive networks, where the social environment of an individual changes according to his behavior. Such a coevolutionary interplay between strategy evolution and graph evolution may give rise to correlations between the strategies of connected individuals. The strategy of an individual will therefore influence not only the payoff he acquires, but also the probability that he will be considered as a potential role model by someone else. When some strategies are more popular than others, the rate of strategy update may effectively change, depending on the intensity of selection.

We study this issue in the framework of active linking (AL) dynamics [55, 56, 62, 65, 72], a model that allows us to study cooperation in adaptive networks analytically in certain limits. Section 2 introduces both our model and the framework in which we will make our analysis. In section 3, we analyze the gradient of selection as a function of the intensity of selection, assuming that the network topology evolves much faster than the individual behavior. Section 4 presents the conclusions.

Consider a finite population of N individuals interacting in symmetric, one-shot, two-player games of cooperation defined by the payoff matrix

$$\begin{equation} M = \bordermatrix{ & C & D \cr C & \!\! R & S\!\! \cr D & \!\! T & P\!\! \cr}. \end{equation}\tag{1}$$

We distinguish two possible game strategies: cooperate unconditionally (C) and defect unconditionally (D). Payoff matrix 1 shows that individuals receive a reward R upon mutual cooperation and a punishment P upon mutual defection. When a C meets a D, the C receives the sucker's payoff S, whereas the D acquires the temptation to defect T.

Individuals do not interact with everyone in the population. Instead, a network indicates who meets whom. The structure of this network is dynamic, in the sense that edges appear and disappear over time. Simultaneously, individuals may reconsider their game play. The network dynamics proceed on a characteristic time scale τ_a, the strategy dynamics on another time scale τ_s. Below, we define each of these two dynamical processes separately.

We use the AL model, developed earlier by Pacheco et al [55, 56], to define the network's evolution. Each individual has a propensity α to engage in new interactions, such that new edges are formed at a rate α². The lifetime of existing edges depends on the behavior of the individuals connected by this link. Specifically, the rate at which CC-links, CD-links and DD-links disappear is given by γ_CC, γ_CD and γ_DD, respectively.

Consider a network with k Cs and N−k Ds. The number of CC-links, CD-links and DD-links in such a network can never exceed $N_{\it CC} \equiv \frac{1}{2}k (k-1)$, N_CD≡k(N−k) and $N_{\it DD} \equiv \frac{1}{2}(N-k) (N-k-1)$, respectively. Under the assumption that the individuals stick to their game behavior, for a large number of links, we can describe the time evolution of the number of links of each type using the following ordinary differential equations,

$$\begin{eqnarray} &&\begin{array}{lll} \dot{L}_{\it CC}(t) = \alpha^2 \left[N_{\it CC} - L_{\it CC}(t)\right] - \gamma_{\it CC}L_{\it CC}(t),\\[8pt] \dot{L}_{\it CD}(t) = \alpha^2 \left[N_{\it CD} - L_{\it CD}(t)\right] - \gamma_{\it CD}L_{\it CD}(t), \\[8pt] \dot{L}_{\it DD}(t) = \alpha^2 \left[N_{\it DD} - L_{\it DD}(t)\right] - \gamma_{\it DD}L_{\it DD}(t),\\ \end{array} \end{eqnarray}\tag{2}$$

where L_ij(t) denotes the actual number of links at time t between individuals adopting strategy i and individuals adopting strategy j (i,j{C,D}). In the steady state, the number of links of each type is given by

$$\begin{eqnarray} &&\begin{array}{lll} L_{\it CC}^* = \phi_{\it CC} N_{\it CC},\\[3pt] L_{\it CD}^* = \phi_{\it CD} N_{\it CD}, \\[3pt] L_{\it DD}^* = \phi_{\it DD} N_{\it DD},\\ \end{array} \end{eqnarray}\tag{3}$$

where _ij=α²(α²+γ_ij)⁻¹ denotes the fraction of active ij-links (i,j{C,D}). Note that the stationary configuration of the network depends on the actual strategy configuration of the population, illustrating the interplay between network evolution and strategy evolution.

The second dynamical process in our model, the strategy dynamics in finite populations, is defined by the pairwise-comparison rule [18, 69]. At every strategy update event, a randomly selected individual X imitates a random neighbor Y with probability

$$\begin{equation} p=[1+\mathrm{e}^{{\beta}/{N} (\Pi_X - \Pi_Y)}]^{-1}. \end{equation}\tag{4}$$

The individual Y can be regarded as the role model of X. Π_X (Π_Y) denotes the total payoff X (Y) receives after interacting once with every neighbor. The parameter β (≥0) controls the intensity of selection. When β is large, the imitation process is driven mainly by the payoff values that the individuals acquire. The game becomes progressively less important for decreasing β.

In the following, we assume that the network dynamics proceed much faster than the strategy dynamics (τ_aτ_s). In this limit, the network always reaches a stationary configuration before a strategy update event occurs. The expected payoff of Cs and Ds during strategy update events is therefore given by

$$\begin{eqnarray} &&\begin{array}{ll} \Pi_C^* = R \ \phi_{\it CC} (k -1) + S \ \phi_{\it CD}(N-k),\\ [6pt] \Pi_D^* = T \ \phi_{\it CD}k + P \ \phi_{\it DD}(N-k-1). \\ \end{array} \end{eqnarray}\tag{5}$$

These payoff values correspond to those obtained in a well-mixed population (complete network) with the same strategy configuration, but using the following rescaled payoff matrix [55, 56],

$$\begin{equation} M'=\bordermatrix{ & C & D \cr C & \!\! R' & S'\!\! \cr D & \!\! T' & P'\!\! \cr }=\bordermatrix{ & C & D \cr C & \!\! R \phi_{\it CC} & S \phi_{\it CD}\!\! \cr D & \!\! T \phi_{\it CD} & P \phi_{\it DD}\!\! \cr }. \end{equation}\tag{6}$$

The network dynamics not only affect the payoffs individuals acquire but also influence the imitation process. Some individuals will act more often as a role model than others, depending on their strategy. In section 3, we show that such differences can have a profound effect on the strategy dynamics.

If τ_aτ_s, a D will select a C as his role model with probability

$$\begin{equation} \frac{\phi_{\it CD}k}{\phi_{\it CD}k + \phi_{\it DD}(N-k-1)}. \end{equation}\tag{7}$$

The term in the numerator corresponds to the average number of CD-links of each D, whereas the term in the denominator reflects the average total number of links of each D. The overall probability that the number of Cs will increase during a strategy update event equals

$$\begin{equation} T^+_k = \frac{N-k}{N}\frac{\phi_{\it CD}k}{\phi_{\it CD}k+\phi_{\it DD}(N-k-1)}[1+ \mathrm{e}^{{\beta}/{N} (\Pi_D^* - \Pi_C^*)} ]^{-1}. \end{equation}\tag{8}$$

Similarly, the number of Cs will decrease with probability

$$\begin{equation} T^-_k = \frac{k}{N}\frac{\phi_{\it CD}(N-k)}{\phi_{\it CD}(N-k)+\phi_{\it CC}(k-1)}[1+ \mathrm{e}^{-{\beta}/{N} (\Pi_D^* - \Pi_C^*)}]^{-1}\!. \end{equation}\tag{9}$$

In well-mixed populations (_CC=_CD=_DD=1), the difference

$$\begin{equation} g(k) = T^+_k-T^-_k=\frac{k}{Z}\frac{Z-k}{Z} \ \textnormal{tanh}\!\left(\!\frac{\beta}{N} (\Pi_D^* - \Pi_C^*)\!\right) \end{equation}\tag{10}$$

(k]0, N[) [69] can be regarded as a finite population analogue of the gradient of selection associated with the replicator equation in infinite, well-mixed populations [2], which is defined as $\dot{x}=x(1-x)(\Pi_C-\Pi_D)$, where x[0, 1] stands for the fraction of cooperators. In both cases, Cs are favored over Ds when g(k)>0 ($\dot{x}>0$), whereas the opposite is true whenever g(k)<0 ($\dot{x}<0$).

It is noteworthy that we consider the evolutionary dynamics as a discrete stochastic system while assuming that the number of links is sufficiently large, so that the linking dynamics can be described by a set of ordinary differential equations (see equation (2)). Stochastic effects could be included at the level of the linking dynamics as well, for instance by adopting a discrete version of the AL model like the one proposed recently by Wu et al [65]. Nevertheless, the continuous approximation we use here does not weaken the robustness of our conclusions, as shown in the following section by means of computer simulations (see figure 2(d)).

We investigate how the gradient of selection depends on the selection pressure. To do so, one can study the shape of g(k) or, alternatively, that of the ratio h(k)=T⁺_k/T⁻_k, which is given by

$$\begin{equation} h(k) = \frac{\phi_{\it CD} (N-k) + \phi_{\it CC}(k-1)}{\phi_{\it CD}k + \phi_{\it DD}(N-k-1)} \mathrm{e}^{-{\beta}/{N}(\Pi^*_D - \Pi^*_C)}. \end{equation}\tag{11}$$

As the solutions of g(x)=0 are sometimes more conveniently obtained solving h(x)=1, we use both interchangeably. For large N, h(x) can be approximated by

$$\begin{equation} h(x) \approx \frac{(1-a) x + a}{(a-b)x + b} \mathrm{e}^{\beta (u' x + v')}, \end{equation}\tag{12}$$

where $x = \frac{k}{N}$, $a = \frac{\phi_{\it CD}}{\phi_{\it CC}}$, $b = \frac{\phi_{\it DD}}{\phi_{\it CC}}$, u' =R' −S' −T' +P' and v' =S' −P'.

Let us start by studying the two limiting cases: the weak selection limit (β→0) and the strong selection limit (β→∞).

In the limit of strong selection, the direction of the gradient of selection depends solely—apart from finite size effects—on the sign of u'x+v' (see appendix A). g(x) will therefore exhibit one of four possible shapes (see figure 1), depending on the ordering of the payoff values of the transformed game M'. When R' >T' and S' >P', g(x) is positive for all x]0, 1[. This scenario is known as C-dominance. Selection will always favor Cs over Ds, irrespective of the strategy configuration of the population. When R' <T' and S' <P', we obtain the opposite scenario: D-dominance. The gradient of selection now satisfies g(x)<0 for all x]0, 1[, implying that Ds are always favored over Cs. When R' >T' and S' <P', g(x) has a root in

$$\begin{equation} x^* = \frac{P'-S'}{R'-T'-S'+P'}. \end{equation}\tag{13}$$

Furthermore, g(x)<0 for x<x^* and g(x)>0 for x>x^*. When the initial fraction of Cs is smaller than x^*, evolution favors Cs. Otherwise, Ds will be favored. This is an example of coordination or bistability. Finally, when R' <T' and S' >P', evolution favors a mixture of Cs and Ds. The gradient g(x) has a root in x^* again, but unlike before, selection favors Cs for x<x^* and Ds for x>x^*. Note that one has to correct for self-interactions when computing the finite population analogue of the equilibrium point x^*. The resulting equilibrium k^* is given by

$$\begin{equation} \frac{k^*}{N} = x^* + \frac{1}{N}\frac{P-R}{T-S-R+P}. \end{equation}\tag{14}$$

Zoom In Zoom Out Reset image size

In the other limit, that of weak selection, we have

$$\begin{equation} \lim_{\beta \to 0} h(x) = \frac{(1-a) x + a}{(a-b)x + b}. \end{equation}\tag{15}$$

Because this function is monotonic in x, h(x)=1 can have at most one solution and consequently the gradient g(x) can have at most one root in ]0, 1[. Hence, g(x) exhibits one of the four shapes shown in figure 1. The particular type of shape that occurs depends on the parameters a and b.

In the following, we assume that links between Cs satisfy both individuals and therefore last longer than other links. Two interacting Ds, on the other hand, are both dissatisfied and prefer to break their connection quickly. Following the same reasoning, Cs would like to break CD-links quickly, whereas Ds prefer to keep such links as long as possible. Hence, on average, CD-links will have a longer time span than DD-links, but a shorter one than CC-links. Altogether, we obtain the following ordering for the fractions of active links of each type,

$$\begin{equation} \phi_{\it CC} > \phi_{\it CD} > \phi_{\it DD}, \end{equation}\tag{16}$$

which is equivalent to

$$\begin{equation} 1 > a > b. \end{equation}\tag{17}$$

This condition ensures that h(x)>1 for all x]0, 1[. Hence, Cs are dominant in the limit of weak selection, irrespective of the game being played.

Having considered the two limiting cases, we now study the gradient of selection for general values of β. In appendix B, we show that g(x) can have either zero, one or two internal equilibria, depending on the parameter settings. First we discuss examples where the specific intensity of selection leads to a scenario with at most one internal equilibrium. Next we will see that some parameter combinations lead to a scenario with two internal equilibria, which can never occur in the limits of either weak or strong selection.

We reduce the complexity of the game space by normalizing the difference between R and P to 1, taking R=2 and P=1. The contours in figure 2 depict the sign of g(k) for three different D-dominance games, using α=0.8, γ_CC=0.4, γ_CD=0.5 and γ_DD=0.6. In the weak selection limit, these birth/death rates of links lead to dominance of Cs, irrespective of the original payoff matrix. The strong selection limit leads to different scenarios, depending on the original game. For instance, S=0.5 and T=2.1 results in bistability (see figure 2(a), S=0.9 and T=2.5 in coexistence of Cs and Ds (see figure 2(b)), and S=0.5 and T=2.5 in D-dominance (see figure 2(c)). Hence, there exist critical values of β at which the number of internal equilibria changes. We can compute these critical values using the derivative of g(x) in 0 and in 1 (see also figure 1). These derivatives are given by

$$\begin{equation} \dot{g}(0) = \frac{\phi_{\it CD}}{\phi_{\it DD}} [1 + \mathrm{e}^{\beta (P' - S')} ]^{-1} - [1 + \mathrm{e}^{-\beta (P' - S')}]^{-1}, \end{equation}\tag{18}$$

$$\begin{equation} \dot{g}(1) = \frac{\phi_{\it CD}}{\phi_{\it CC}} [1 + \mathrm{e}^{\beta (R' - T')} ]^{-1} - [1 + \mathrm{e}^{-\beta (R' - T')}]^{-1}. \end{equation}\tag{19}$$

Assuming at most one internal equilibrium, we obtain C-dominance (D-dominance) when both $\dot{g}(0)$ and $\dot{g}(1)$ are larger (smaller) than zero. Bistability occurs when $\dot{g}(0) < 0$ and $\dot{g}(1) > 0$, and coexistence when $\dot{g}(0) > 0$ and $\dot{g}(1) < 0$. From equation (18), it follows that $\dot{g}(0) > 0$ if and only if

$$\begin{equation} \ln\!\left(\frac{\phi_{\it CD}}{\phi_{\it DD}}\right) > \beta \ (P' - S' )\!, \end{equation}\tag{20}$$

and that $\dot{g}(1) > 0$ if and only if

$$\begin{equation} \ln\!\left(\frac{\phi_{\it CD}}{\phi_{\it CC}}\right) > \beta \ (R' - T'). \end{equation}\tag{21}$$

This means that $\dot{g}(0)$ will be positive for β smaller than

$$\begin{equation} \beta_0 \equiv \ln\!\left(\frac{\phi_{\it CD}}{\phi_{\it CD}}\right) \ (P' - S')^{-1}\!, \end{equation}\tag{22}$$

and negative otherwise. Similarly, $\dot{g}(1)$ is positive for β smaller than

$$\begin{equation} \beta_1 \equiv \ln\!\left(\frac{\phi_{\it CD}}{\phi_{\it CC}}\right) \ (R' - T')^{-1}\!, \end{equation}\tag{23}$$

and negative otherwise. Both β₀ and β₁ are indicated by vertical dotted lines in figure 2. They correspond clearly to the values of β at which each of the game transitions takes place.

Zoom In Zoom Out Reset image size

We further confirm the validity of the analytical results by means of computer simulations. Figure 2(d) shows the probability, as a function of β, that a population with initially 50% cooperators evolves to full cooperation. Circles, representing simulation results, fit nicely with the analytical predictions, which are indicated by a solid line. Each simulation starts with a complete network of size N=100, 50% of them being Cs, and runs until the population fixates into either full cooperation or full defection. Strategy and network structure evolve simultaneously under asynchronous updating. Strategy update events take place with probability (1+τ)⁻¹, where $\tau=\frac{\tau_a}{\tau_s}= 10^{-3}$; network update events occur otherwise. We run 10⁴ simulations for each value of β and plot the fraction of runs that end in full cooperation. The analytical predictions are calculated using the following formula [69],

$$\begin{equation} \rho_i = \frac{1+\sum_{k=1}^{i-1}\prod_{j=1}^{k} \frac{T_j^-}{T_j^+} }{1+\sum_{k=1}^{N-1}\prod_{j=1}^{k} \frac{T_j^-}{T_j^+} }, \end{equation}\tag{24}$$

with i being equal to 50. The expressions for T_j⁺ and T_j⁻ are given by equations (8) and (9).

Earlier in this section, we showed that there can be at most one internal equilibrium for the limiting cases β→0 and β→∞. At intermediate intensities of selection, two internal equilibria can, however, occur simultaneously (see also appendix B). Figure 3 shows two examples where this is the case. The upper two panels correspond to a scenario where Cs are expected to disappear from the population in case they are rare. If a sufficiently large fraction of Cs is present, however, a mixture of Cs and Ds is favored. The lower two panels illustrate an example of the opposite scenario, in which Ds are expected to go extinct when they are rare.

Zoom In Zoom Out Reset image size

In this paper, we have explored, both analytically and numerically, the evolution of cooperation in dynamical networks that evolve side by side with individuals' behavior, the evolution of the network structure being affected by the dynamics of the individuals and vice versa. We indicate that the adaptive nature of the network of contacts affects both the average payoff associated with each game strategy and the likelihood for each of the different strategies to serve as a role model. The intensity of selection, which controls the contribution of game payoff to fitness, regulates the importance of each of these two effects in the final outcome of strategy evolution. When the intensity of selection is strong, the payoff transformation resulting from the network dynamics provides the dominating contribution to evolution. Weakening the intensity of selection enhances the effect of the adaptive imitation network. By doing so, one is able to effectively transform a D-dominance dilemma into any of the conventional 2×2 symmetric games. We derive analytical conditions, valid under the assumption that there is only one internal equilibrium, that predict which range of intensities of selection leads to which game scenario. We show also that certain parameter combinations may lead to the occurrence of two internal equilibria at intermediate intensities of selection. Our results clearly demonstrate the relevance of the intensity of selection when evolution proceeds on an adaptive structured population. Furthermore, we indicate the conditions under which this coupled dynamics work as an efficient mechanism for the promotion of cooperative behavior.

SVS and TL acknowledge financial support from F.R.S.-FNRS (Belgium). JMP and FCS acknowledge financial support from FCT (Portugal).

The following inequality holds,

$$\begin{equation} \lim_{\beta \to \infty} h(x) = \lim_{\beta \to \infty} \frac{(1-a)x+a}{(a-b)x+b} \mathrm{e}^{\beta(u'x+v')} \end{equation}\tag{A.1}$$

$$\begin{equation} \hspace{4.1pc}\leqslant K \lim_{\beta \to \infty} \mathrm{e}^{\beta(u'x+v')}, \end{equation}\tag{A.2}$$

where K>0 is a constant, whose specific value depends on the parameters a and b. Therefore,

$$\begin{eqnarray} \lim_{\beta \to \infty} h(x) = \left\{\begin{array}{@{}cc@{}} \infty & (u'x+v'>0), \\ [6pt] 0 & (u'x+v'<0). \end{array}\right. \end{eqnarray}\tag{A.3}$$

Hence, the direction of the gradient of selection depends solely on the sign of u'x+v'. Cs are favored over Ds if and only if u'x+v' >0.

Let us designate the rational factor in equation (12) by h₁(x) and the exponential factor by h₂(x). Both h₁(x) and h₂(x) are monotonic in x:

• The derivative $\frac{{\rm d}}{{\rm d}x} h_1(x)$is given by $\frac{b-a^2}{((a-b)x + b)^2}$. Hence, h₁(x) increases monotonically if b>a² and decreases monotonically if b<a².
• The derivative $\frac{{\rm d}}{{\rm d}x} h_2(x)$is given by u'  e^u'x+v'. Hence, h₂(x) increases monotonically if u' >0 and decreases monotonically if u' <0.

h(x) increases (decreases) monotonically in case both h₁(x) and h₂(x) increase (decrease) monotonically. Hence, if a²<b and u' >0 (or a²>b and u' <0), then g(x) can have at most one root in ]0, 1[. If h₁(x) increases (decreases) while h₂(x) decreases (increases), then g(x) can have at most two roots in ]0, 1[.