Ken Binmore writes:
Robert Axelrod - The Complexity of Cooperation: On examination, it turns out that TIT-FOR-TAT was not so very successful in Axelrod's simulation. Nor is the limited success it does enjoy robust when the initial population of entries is varied. The unforgiving GRIM does extremely well when the initial population of entries consists of all 26 finite automata with at most two states.... Why not follow Linster (1990, 1992) in beginning with all finite automata with at most two states? The system then converges from a wide variety of initial conditions to a mixture in which the strategy GRIM is played with probability greater than 1/2. But GRIM is not forgiving. On the contrary, it gets its name from its relentless punishment of any deviation for all eternity...
GRIM can only be beaten only by programs that (a) defect first, and (b) thereafter follow TIT-FOR-TAT by never offering cooperation this turn in response to defection last term. GRIM can only be tied by another GRIM, by TIT-FOR-TAT, by JESUSCHRIST, or by other programs that never defect in response to cooperation.
It is important to be clear about why GRIM "succeeds": it "succeeds" not because it does well but because it ensures that everybody else does extremely badly. As soon as GRIM learns that it is not playing against another instantiation of GRIM, it then does everything it can to make the other's score as low as possible. In evolutionary game-theory set-ups, GRIM will tend to increase its share within a community, but communities that have a lot of GRIMs in them will have low average scores (and hence, presumably, be unlikely to expand much).
I would argue that Binmore is working with the wrong definition of "successful": solitudinem faciunt et "successful" appellant. "Successful" means that one does well, not that one turns one's surrounding environment into an instantiation of Road Warrior.
We're in a situation of reflective equilibrium with game theory here: results that are rigorous but are counterintuitive suggest a modeling error; results that are intuitive but unrigorous suggest a wishful-thinking error.
