Cosma Shalizi directs me to:
[…]
Basically, consider the strategy space of one-period look-back mixed strategies. And take the strategy S = { p(C|CC)=1/2-ε, p(C|CD)=1/4+ε', p(C|DC)=0, p(D|DD)=0 }.
If the opponent cooperates all the time, their average score is a little better than the DD payoff 1--giving them an incentive to choose a strategy that cooperates some time. If they cooperate all the time, then your average score is a little worse than 4--pretty good. And if they do anything other than cooperate all the time, their score falls and is worse than if they cooperate all the time. Thus in this strategy space { P(C )=1 } is a dominant strategy for the opponent if you choose your strategy first.
Of course, this is not a Nash equilibrium: if the opponent is playing C you should play C as well. D and really clean up.
And two opponents each thinking that they move first and committing to S do rather badly: they end up in (D,D) for all time.
Cosma also directs me to:
KARL SIGMUND & MARTIN NOWAK: A comment on Press-Dyson (PNAS)
Being close means not being there. We had known about strategies that allow to nail down the opponent's payoff to an arbitrary level [1,2], but not about the vast and fascinating realm of zero determinant (ZD) strategies that enforce a linear relation between the payoffs for the two players. This opens a new facet in the study of trigger strategies and folk theorems for iterated games, and offers a highly stimulating approach for moral philosophers enquiring about 'egoistic' and 'tuistic' viewpoints.
Our only quibble with the Press-Dyson paper is semantic. The title speaks of 'evolutionary opponents', which suggests evolutionary game theory. But biological or cultural evolution is not a phenomenon on the level of the individual. It requires a population. The 'evolutionary' players of Press and Dyson don't evolve but adapt. With their splendidly 'mischievous' extortionate strategies, Press and Dyson contribute to classical game theory, by considering two players who grapple with each other in a kind of mental jiu-jitsu. The leverage afforded by zero-determinant strategies offers a splendid new arsenal of throws, locks, and holds.
Which of these strategies can flourish in an evolutionary setting is less clear. Being successful, in this context, feeds back at the population level. It means that more and more players will act like you, be they your offspring or your epigones. Thus you are increasingly likely to encounter your own kind. If your 'extortionate' strategy guarantees that you do twice as well as your opponent, and your opponents' strategy guarantees that she does twice as well as you, this only means that both get nothing. The only norm which is not self-defeating through population dynamics requires players to guarantee each other as much as themselves. We are then back to Tit For Tat. Press and Dyson are perfectly aware of this, of course. In a nutshell, they have uncovered a vast set of strategies linking the scores of two players deterministically (as TFT does), but asymmetrically (unlike TFT). This enriches the canvas of individual interactions, but not necessarily the range of outcomes open to evolving populations.
[1] M.A. Nowak, M.C. Boerlijst, K.Sigmund, Equal pay for all prisoners, AMS Monthly 104 (1997) 303-307.
[2] K. Sigmund, The Calculus of Selfishness, Princeton UP, Princeton, New Jersey (2010).
