Cognitive strategies take advantage of the cooperative potential of heterogeneous networks

Figure 1 shows the results for both unconditional and reactive strategies when the community structure is described by BA scale-free networks. Unconditional strategies fare well as long as the penalty for being cheated is below ~0.3 (b ~ 1.3 in figure 1), in which case cooperation dominates. As noted previously, the key for this success of cooperation resides in the hubs [40, 54]. If a hub is occupied by a defector, then initially she will be successful because of the high number of her neighbours among which, due to the random initial conditions, there will be some cooperators to be exploited. However, because of her success, the defector hub will influence her neighbours, who will start to imitate her strategy, thereby turning into defectors and, as such, contributing to decrease the fitness of the hub, rendering her an easy prey to be invaded by a nearby cooperator. In short, defectors in central positions become victims of their own success.

The situation of a cooperator in a hub is strikingly different: when she is imitated by one of her neighbours, this increases the hub's fitness, giving rise to a positive feedback mechanism which reinforces the position of the hub-cooperator, eventually leading to a cooperative community.

Clearly, for stricter dilemma situations this mechanism is not sufficient to ensure cooperator dominance [51] (see black squares in figure 1). Indeed, for high values of the sucker's payoff S (=1 − b), the advantage of highly connected cooperators will be undermined by the high cost paid when being exploited. As a result, the central positions will become prone to invasion by defectors and the population may finish in an overall defector state.

The situation changes considerably in the case of reactive strategies: players can react differently to different partners, allowing them to escape the trap discussed above in connection with unconditional strategies. Indeed, cooperative hubs can benefit from mutual cooperation while punishing or retaliating against defecting players. In fact, among reactive strategies, pure cooperators and pure defectors are rather rare, prompting a change in terminology: players with a high p value become the 'new' cooperators as they mostly cooperate after a cooperative act. Their q value, in turn, gives information about how generous they are, as it defines how easily they forgive a defective act. Individuals with low p and q values become the 'new' defectors, as they mostly defect independently of the partner's past decision.

It is easy to see that the hubs maintain a leading, central role in what concerns the evolutionary dynamics. However, and unlike the unconditional strategy case, now both cooperators and defectors, once occupying a hub, are able to protect themselves from invasion by other strategies. Indeed, the non-zero p and q values significantly increase their resilience to invasion compared to the p = q = 0 scenario characteristic of unconditional defectors. Due to the stochastic nature of reactive strategies and a hub's large number of partners, a mostly defecting player can still accumulate a considerable fitness even when most of her partners imitate her strategy. Overall, the message is that, in heterogeneous environments, the emergence of cooperation can be more troublesome. On the other hand, once established, cooperation is extremely robust and takes over the whole population (see below).

The accuracy with which individuals copy the behaviour of their neighbours plays a very interesting role in the process of strategy update in a hub. This uncertainty in the perception is modelled by a Gaussian centred on the real (p, q) parameters of the imitated neighbour and so the accuracy is embedded in the Gaussian width parameter σ (for the definition, see section 2). The higher the value of σ, the more inaccurate is the capacity of individuals to imitate those that perform better. For small values of σ, and despite the fact that most of the times evolution leads to cooperative scenarios, the influence of the initial strategy of those individuals occupying the hubs is so dramatic that it may dictate, to a large extent, the outcome of the evolutionary dynamics in a single evolutionary run. Hence, fully cooperating as well as fully defecting populations may emerge from random initial conditions associated with faithful copying of better strategies. However, as soon as σ increases above a critical value (which depends on the degree of heterogeneity of the network, on the population size and on the average connectivity), the window of opportunity for defectors no longer exists and cooperators dominate unconditionally, as shown in figure 1. The reason lies again in the particular behaviour of highly connected nodes. Due to their initial success, hub-players with low p and q will influence their peers to adopt a similar behaviour. However, in such an environment the fitness of players with different degrees will become comparable. Increasing the inaccuracy of copying (σ) results in a wider exploration of the space of p and q values, and so higher values can also emerge. Whenever a highly connected node adopts a higher p, due to her high number of connections she will be able not only to maintain such a strategy for a long period, but also to reward a higher number of other cooperative actions. Neighbours copying this new behaviour will increase their own and the hub-player's fitness, too. In other words, the positive feedback mechanism acts here towards increasing the cooperative behaviour nature of individuals. As a result, such behaviour will spread in the population, as hubs are frequently imitated. This said, the increase of the blur in perception has an important side effect: cooperation cannot reach 100% in this population as the strategy parameters are spread around the average value. This is the reason why the measure of cooperative acts does not reach 100% for the cognitive strategies in figure 1.

Another interesting stochastic parameter in the strategy adoption process is the parameter K, which describes the errors in decision making; that is, how frequently less successful players are imitated. Alternatively, it is often associated with the so-called intensity of selection. For low K values, selection is strong and the more successful strategies are almost always imitated. With increasing K, the importance of fitness decreases, while for K → ∞ the evolution becomes neutral and fitness plays no role in the imitation process. Consequently, for finite K the strategy of individuals with identical or lesser fitness may be adopted with a finite probability. As a result, the main role of K is to catalyse cooperation by tipping out the hub-players from absolute defecting behaviour. For positive K, in a cluster of (0,0) strategists, players will try to adopt each other's strategies despite their identical fitness, and due to σ their p and q values will begin to move away from the defective corner of the strategy phase space. As figure 1 shows, cooperation emerges for any temptation value. Surprisingly, the measure of cooperation is higher for larger b (and larger c).

Up to now we have limited our study to the prisoner's dilemma formulated in terms of a single parameter b. However, one may enhance the strength of the dilemma by varying T and S independently, as each of these parameters defines a different social tension [50, 51]. While S can be associated with the fear of being cheated (as it represents the disadvantage of a cooperator when facing a defector), T may stand as a measure of greed (as it represents the temptation to defect).

In figure 2 we disentangle these social tensions, showing that S provides the main influencing factor in the evolution of p and q. Increasing the fear of being exploited increases both the strength of retaliation (lower average q) and positive responses to cooperative acts (higher average p), driving the whole population into mutual cooperation. This surprising conclusion differs from that obtained with unconditional strategies [23, 51], highlighting the importance of simple cognitive features under severe cooperation dilemmas. Fear stands as the most significant pressure in the evolution of individuals' reactions, whereas greed (T) does not significantly influence the emergence of cooperation as long as fear is strong enough, as shown in figure 2.

An important aspect to be investigated is how robust cooperation is against invasion by defectors. To this end we follow the procedure adopted in [17], adapted to the heterogeneous nature of scale-free graphs. Due to the different neighbourhood structure of the nodes in a heterogeneous network, robustness is not straightforward to define. Replacing a cooperative player (with high p and low q values) in a hub with an unconditional defector will have an impact that is very different from doing the same on an edge node with few neighbours (leaf). Two mechanisms contribute to this effect: firstly, the hub-player interacts with more players than a leaf player, and so she can directly influence a larger fraction of the community; secondly, due to the high number of neighbours, her accumulated payoff can become potentially higher (as defector) compared to that of her neighbours, so she can spread her strategy successfully until, possibly via retaliation, her payoff eventually drops.

Given these scenarios, we tested the robustness of cooperation in two different situations. In the first one, we replaced a given fraction (μ) of the population with (0,0) strategists in every simulation step independently of the degree of the node, while in the second we only allowed the injection of defectors on nodes with a degree smaller than 1/5 of that of the biggest hub in the network. That is, in the second case we artificially protected the influential players in the network. For smaller percentages of injection of pure defectors, cooperation was robust for any temptation value in both of the above cases. However, for more frequent invasions cooperation survives only in the case where the biggest hubs are protected from direct occupation (second scenario).

So far, cognitive strategies have been able to promote the emergence of cooperation on heterogeneous networks which, once installed, proves robust against invasion. Prompted by such success, we now study the impact of participation costs in the situations studied.

Keeping track of a large number of contacts requires considerably more energy/resources than remembering the actions of just a few acquaintances [53]. Alternatively, participation costs may be considered as the price of having a larger memory capacity, being as such proportional to the number of neighbours. Such costs can be introduced into the payoff matrix by subtracting the value h (h > 0) from every payoff entry.

From a purely mathematical perspective, the prisoner's dilemma payoff ranking stays unaffected when the interaction cost h is applied to every player independently of her decision. It is also worth noting that the introduction of h does not change the results in a homogeneous environment where every individual has the same number of neighbours, as in this case it decreases the payoff of all players by the same value, so strategy update processes based on payoff differences lead to the same evolutionary dynamics. This said, however, it is also important to note that in our model players are expected to play with each and every neighbour they have. This may be reasonable whenever interacting has no cost. With participation costs, however, it is only reasonable to expect that individuals should be able to opt not to engage in all their possible interactions. It was shown that the introduction of voluntary participation can significantly change the outcome of the evolution in structured populations [58–60]. The issue becomes particularly relevant whenever the cost of each interaction is higher than the payoff resulting from both mutual defection and mutual cooperation. In this case, neither opportunistic nor altruistic behaviour pay, compared to interaction avoidance, and the strategy space of our model becomes incomplete. Hence, for the PD under study here where 1 = R > P = 0, h should satisfy h ≤ R.

The effects of participation costs are shown in figure 4 for both unconditional and reactive strategies as a function of the temptation (b) and participation cost (h). Here we adopted the payoff matrix characterized by T = b and S = 1 − b, in order to reduce the number of free parameters to b and h, while still being able to study harsh dilemma situations. As the figure shows, as long as h < R (more precisely, when 0 ≤ h ≤ 0.9) cooperation thrives, and defection will appear only residually due to the stochastic nature of the strategies. The key for this result is that, in this regime, the reward for mutual cooperation (1 − h) is positive; that is, if cooperative hub-players can turn most of their 'followers' into cooperators, then they can have the same stable, leading role as when without the participation cost, due to their superior payoff.

Zoom In Zoom Out Reset image size

Consequently, as figure 4(a) shows, the disadvantage—for those with higher number of nodes—stemming from introducing participation costs is not enough to suppress the success of reactive strategies. In fact, the basic mechanisms described above remain valid, leading to a high prevalence of cooperative actions for the entire b range. This contrasts with the scenario of unconditional cooperators and defectors, shown in figure 4(b), where cooperation is rapidly reduced by the existence of participation costs.

The dominance of cooperation shows a weak dependence on b: with increasing temptation values (and penalties) it becomes harder for cooperative individuals to take over the hubs as an occasional, successful defection against them lowers their income by a greater amount, with an associated impact which is higher and, hence, harder to counteract. Consequently, for higher b values, cooperation can only win if the participation costs are slightly lower.

In the transitional region, that is, whenever 0.9 ≤ h ≤ 1.0, the shift of the leading role from the hubs to the smallest degree players happens gradually with the increase of h. At around h = 0.9 and smaller b values, the difference in strategic success and the burden of participation costs from different numbers of interactions makes the payoff for players with different degrees comparable. Such a homogeneous and leaderless state results in a slight drop in overall cooperation. Further increase in h turns the behaviour of low degree nodes more influent, although the payoffs remain comparable. Players in cooperative clusters, which are formed due to the randomness of the initial conditions and consist of low degree individuals, can pass their strategy to other low degree individuals through higher degree nodes. These intermediate nodes are taxed by the higher participation cost, but with support from the cooperative cluster they can still influence lower degree players in a defecting environment. Thus information can flow in the population and cooperation can spread and reach its full potential. For higher b values, the probability of the cooperative clusters' formation at the start of the evolution decreases as a successful defection against a cooperative player sets back the individual much more in this case.

As expected, whenever the participation costs satisfy h > R, the evolutionary outcome resembles that of the costly participation region in [53], since highly connected individuals are so overburdened that they cannot influence the rest of the population. Such a scenario leads to a fragmented community of isolated, small degree individuals.

In summary, we can point out that reactive strategies—in contrast to unconditional strategies—grant cooperation a leading role in the evolutionarily interesting range (when actual strategy evolution happens) of participation costs and for all temptation values.