Enabling Cooperation: Chapters 1-5
The book establishes a very simple base case, that it then modifies to form each of the five cases presented in chapters 1-5. Actually just four cases, as we will see.
Base case:
You are one entity in a large population of P (say P = 10000) ‘players’. The only property of a player is its ‘strategy’, either always ‘c’ (cooperate) or always ‘d’ (defect). Repetitively, two players from the population are chosen at random to ‘interact’, i.e. play the game (defined below) according to their strategies. The game rewards each of them with some tokens.
Birth-Death: Every M interactions (say M=P), a token is chosen at random from those awarded so far. The player who had that token ‘reproduces’, i.e. another entity is created with the same strategy. Clearly a player with lots of tokens has a better chance of reproducing than a player with few tokens. Finally, a player is chosen at random (all having equal probability) to die, to be eliminated, so the population remains constant.
About parameters: There are many parameters to these models, here indicated by capital letters (like M). The goal of these researchers is to exhaustively search the entire parameter space, so that they scope out every possible case. For instance, M (the number of interactions per reproduction) is a parameter, and so they need to run simulations over all different choices of M (!) unless they can prove, mathematically, that further search won’t find any significant changes in the results. There will be dozens of parameters, and if every combination of parameter values needs to be tried (by running many simulation trials with those parameter settings), the explosion in the number of combinations of parameter values will quickly make the researchers’ goal impossible. Therefore there’s a huge premium put on designing the models to be mathematically tractable! Much of the power of these researcher’s approach lies in this interplay of simulations guided by math, and math informed from simulations.
Above I described the Birth-Death process. Suppose one first choses a player to die, and then choses someone to reproduce from among the remainder? Call this parameter B: with possible values “Birth-Death”, or “Death-Birth”. Gotta be tested, unless you can prove it’s insignificant.
Strong versus Weak Selection: Strong selection is when your fitness—the probability that you are chosen to reproduce—depends wholly on your success in the game, i.e. the number of tokens you’ve won. Weak selection is when everybody has a pretty uniform chance of being chosen, perhaps the number of tokens you’ve won only impacts 10 percent of your fitness. Maybe you mostly subsistence farm, and only occasionally barter (play the game), so your gaming success only affects a small part of your diet. Voila, another parameter, selection strength X, a real value varying between 0.0 (weakest of the weak) and 1.0 (strongest of the strong), reflecting what fraction of your fitness is determined by how many tokens you’ve won. Now a real value has an infinite number of possibilities, and indeed they do find results like “if X is greater than 0.7071 then the results are such and such, else so and so…”, so how can they test all possible values? Answer: only if the model is mathematically tractable enough that they can prove that between two values that they have tested there won’t be any significant change in the results.
The Game: One round of the game is defined by a table showing the allocation of tokens to the players depending on their choice of c or d, according to their respective strategies. In the table, if you play c, and he plays d, then you’d get S tokens and he’d get T tokens, i.e. for the example parameter settings shown in the second table, you’d get zip and he’d get 5. If you both played c, you’d both get 3.
c d for example c d i.e R = 3, S = 0, T = 5, P = 1.
c R S c 3 0
d T P d 5 1
Note that for the example values I’ve given, T > R > P > S, which makes this an example of the “prisoners’ dilemma” narrative. There are many other parameter choices that need investigating, for example if T > R > S > P, then we have a game that has many several narratives that illustrate the logic (called “snowdrift game”, “hawk/dove game”, and “chicken game”, if you want to google them). The narratives are not parameters, they’re superfluous, they don’t change the results.
In the base case each time a pair of players is chosen to interact, they only play one round, and then another pair is chosen. So your strategy can only be either: always to c (cooperate), or always to d (defect), since you are given no information (memory) from the past (previous interactions) upon which to make any smarter strategies.
Do note that for prisoners’ dilemma (T > R > P > S), for any individual transaction you are better off defecting, regardless of whether your opponent cooperates or defects. So if you start off with a large population containing a random mix of all strategies (c or d), then the d’s will slowly increase in number, and the c’s decrease in number until they all die out.
Chapters 1-5 are devoted to changing the base case so that selection favors cooperation. Chapters 1 and two discuss what happens if you are given some tidbits of information about the past, and can use a strategy that takes advantage of that info.
Chapter 1. Direct Reciprocity
The base case is changed in that, whenever you select a pair of players to play the game, instead of playing the game just one round, they play N rounds. Two bits of data are remembered from the past while playing this series: each player remembers what they did (Y = c or d) in the previous round, and what the other guy did (Z = c or d) too.
They investigate all possible strategies based on Y and Z—all possible rules of deciding whether to cooperate or defect based on these two bits of information. These rules are
YZ cc cd dc dd 1 c c c c < allc 2 c c c d 3 c c d c 4 c c d d 5 c d c c < win stay / lose shift (for T>R>P>S) 6 c d c d < tit for tat (do what he did last time) 7 c d d c 8 c d d d 9 d c c c 10 d c c d 11 d c d c 12 d c d d 13 d d c c 14 d d c d 15 d d d c 16 d d d d < alld
(There’s actually another strategy parameter, V = c or d, which tells you whether to cooperate or defect on the first round of the series, when you have no info about the past. For tit-for-tat, V = c.)
When two tit-for-tat playing players meet, they have a very productive session, since they both collect 3 points per round (using the example values of T R P and S given above). When alld meets tit-for-tat, alld wins the first round getting 5 points and tit-for-tat 0, but thereafter they both always defect so neither earns much.
Under these conditions if you start with a large population containing a random mix of all 16 strategies, slowly tit-for-tat will grow in numbers. Once most of the population is tit-for-tat, allc may ‘random walk’ to large numbers, since against tit-for-tat it does equally well so there’s little selection pressure against it. But if allc happens to random-walk its way to a large segment of the population (as given enough time it eventually will), suddenly alld will take over, since it does exceedingly well against allc, gaining 5 points per round.
Suppose further that there’s some noise in Z, i.e. occasionally you don’t remember correctly whether the other guy cooperated or defected last time. This presents a problem for tit-for-tat, because if there’s just one misperception, two tit-for-tat players will get into a less productive pattern, one defecting while the other cooperates, switching roles every game, so that they only average 2.5 per session.
Under noise conditions, if you start out with a large population containing a random mix of all of all the strategies, slowly tit-for-tat starts predominating. But once most of the population is tit-for-tat, then a blended strategy called “generous tit-for-tat” which is most often cdcd but occasionally is cccc (i.e. it can occasionally ‘forgive’ a defection and cooperate anyhow) will grow to dominate the population. (Yes, more parameters: how much to blend, and which strategies get blended.) Once cooperative strategies predominate, allc has less of a chance to random-walk itself into predominance, because win-stay/lose-shift is also sharing the space and adds a tiny pressure on allc under noise conditions (whenever cdcc misperceives cccc, cdcc wins big), and thus diminishes the chance that (increases the amount of time before) allc random-walks itself into enough of the population that alld comes crashing back.
Chapter 2. Indirect Reciprocity
Start with the base case (so unlike chapter 1, every time a pair of players is selected to play the game they play just one round) BUT each player will be given some information on the past play of the other guy against other players. In other words, each player has a ‘reputation’, a few bits (one or two or maybe, golly, three bits) of info about how this guy has conducted himself against other players. If you have one bit (lets call it G for gossip, and G = g or b for ‘good guy’ or ‘bad guy’) then there can be four strategies: always c, c iff G=g, c iff G=d, and always d. G can be calculated in many imaginable ways, such as G = g iff he’s cooperated more often than he’s defected. All to be investigated. And G could have two bits, perhaps reflecting whether he cooperated more with good guys than with bad guys. More and more things to be investigated, and of course with two bits you’d have twice as many strategies to investigate.
The net result is that given just about any shred of information from the past, cooperative strategies emerge to predominate in the population. By cooperative strategies I mean a set of strategies which, when they play against each other, nearly always cooperate. They may well be very uncooperative with other members of the population. So far, I have not seen the authors’ models get to a level of complexity where there can be two sets of cooperative strategies that are mutually uncooperative.
Chapter 3. Spatial games
Assume that geometry plays a role. For example, assume that players exist in the squares of a chessboard, so that they only interact with their immediate eight neighbors (as in a chess kings’ move). Start with the base case, but to chose a pair of players to play a round (just one), chose one player at random and an opponent at random from that player’s neighborhood. To chose who reproduces and who dies, pick a token at random and reproduce the owner of that token, and then randomly pick someone from their neighborhood to die and plant the new-born into that position. So space, geometry, plays a role in determining both who plays with whom, and who dies so that somebody new can take their place.
There are lots of different ways of defining neighborhoods (4-neighbors, hexagonal tessellations, 3D, random connections, etc), so choice of spatial structure represents a large space of parameters to investigate.
Note that in Chapter 3 there are no strategies other than always c or always d, because no information is given about the past. What’s fascinating is that even without such information, the very fact of having neighborhoods is enough that cooperation can be selected for. Let’s take the 8-neighborhood grid, and suppose that the current population is
x x x x x x x x x x
x x x x x x x x a x
x x x x x x x x x x
x x x x x x b x x x
x x x x x p x x x x
x x x x x x c x x x
x x x x x o x x x x
x x m x x x d x x x
x x x x x n x x x x
x x x x x x x x x x
where blacks are cooperators and reds are defectors. I’ve labeled a few of the positions so that I can talk about them. Player a is doing very poorly, since he’s a defector surrounded by defectors, nobody ever earns much. Player d is doing very well indeed, since he’s a defector who gets 5 points whenever he interacts with one of his 5 cooperating neighbors. Player c has three cooperators to sponge off of, and b only one. Player m is doing very well, since he’s a cooperator surrounded by cooperators and gets 3 every time he plays. Player p is doing very poorly, since he’s a cooperator who’s being robbed blind (he gets 0, they get 5) whenever he plays with any of his 5 defector neighbors. Player o is being robbed by 3 defectors, and cooperates fruitfully with 5 cooperators. Player n has 7 times as many fruitful interactions as robberies.
The point is twofold: players in the centers of clumps of cooperators do very well, and reproduce frequently and may well expand the clump, but the borders are very volatile since defectors fair very well if they are surrounded by a lot of cooperators. The result is changing kaleidoscope patterns where (depending on parameter settings, in particular the weights of T R S and P) cooperation may well grow to a substantial portion of the population, or die out, or cyclic patterns of more and less cooperators may ensue.
The result is that if the geometry permits clumps of cooperators to keep each other warm, lots of cooperation may be selected for, even with no knowledge whatsoever of the past.
Chapter 4. Group Selection
Start with the base case, only abolish death, so that the population grows. Starting with a random number of cooperators and defectors, the percentage of cooperators starts falling even though nobody dies.
Do this with many populations in parallel. Call each population a ‘group’. Note that groups that happen to start with more cooperators will grow faster than groups with a lesser number of cooperators.
If a groups’ population grows past a given threshold J, then split the group randomly into two groups—flip a coin for every player to assign them to one group or the other. Note that the two resulting groups might be different sizes, and might have slightly different proportions of c and d players.
Now reintroduce death, but at the group level. Randomly, every once in a long while, to keep the total population (summing all groups) roughly constant, eliminate an entire group, chosen wholly at random.
Who’d have thought, but this is enough to induce cooperation. Even though in every group, cooperation is declining! In splitting, the random allocation of players nearly always results in one spawned group having a greater percentage of cooperators than the parent group, while the other spawned group gets less, and then of course the group more heavily laced with cooperators grows faster.
Chapter 5. Kin Selection
This chapter is devoted to a big math argument that has exploded between the authors and the large cadre of scientists who’ve based their careers on kin selection. The authors and their defenders claim that everything that can be derived from kin selection can be derived from group selection, with much more mathematical tractability permitting a more effective search of the parameter space. Their detractors say that everything that can be derived from group selection can be better derived from kin selection. For us it doesn’t matter—we won’t judge this case, nor will the authors nor their peers. The case will be decided by the next two generations of grad students, who will sort out which formulation is most useful, and in what applications.