Critical to fully understanding the evolution of behavioural traits is understanding some basic game theory, such as the iterated prisoner's dilemma.
The basic prisoner's dilemma is as follows: two criminals are taken to the police station and interviewed separately. They went on a crime spree together and there is enough evidence to convict them of some of the minor offences. Imagine you are one of the prisoners and you are given a choice, which you assume your partner in crime has also been given: confess to everything and thereby grass up your partner for the major charges [defect]; or keep your mouth shut [cooperate]. If you talk and your partner doesn't, then you get a reduced sentence of 2 years in prison because of helping to convict your partner; and he gets 10 years. If he talks and you don't then obviously the reverse happens. If you both give a full confession then you both get 8 years, and if neither of you talk you both get 4 years. What do you do?
Now imagine that you and your partner can both live forever, and any time either of you are not in prison you go on another crime spree together and each time you get caught and face the same decision. You know your partner's previous responses, so this will obviously affect whether you cooperate or defect in the latest interview with the police. This situation is the iterated prisoner's dilemma, and occurs (in other forms!) every day in societal life (the social contract).
The various consequences to the options can be formalised into a pay-off matrix, and different weightings given to the possible outcomes. So long as all the options are sensible, for example the benefit to you for defecting when your partner cooperates should always be more than the benefit to you for cooperating when your partner cooperates, then it is possible to make some firm conclusions about the worth of different strategies.
Axelrod created a tournament in which different programs played against each other (200 iterations for one game) and then each program's total score was added up. Some of these programs had very sophisticated algorithms to predict whether the opponent would cooperate or defect, however the overall winner was a program called tit-for-tat which simply started out cooperating and then did whatever the opponent had done the previous move. In the second tournament the winner was called tit-for-2-tats, I don't think I need to explain that one's tactics!
It is important to note that in a population it isn't as simple as there being an optimum strategy that everyone should abide by. One of the founders of modern game theory, John Nash (of “A Beautiful Mind”), showed that strategies of cooperation and defection settle to a balance (the Nash equilibrium) – so long as the frequency of defection is low, then on average the cooperators gain by continuing to cooperate and the small number of defectors can take advantage.