Two ways to revise a strategy in undercutting games

An undercutting game is a symmetric two-player game with ordered strategies 1, …, n + 1; in SBRUG, n is set by horizon. Lower strategies are called earlier, and higher strategies later. One way to picture it: two players share an activity that can last n periods; strategy i ≤ n stops it at period i, and strategy n + 1 never stops first. Continuing together pays, but each player can gain by stopping first.

SBRUG compares two ways of learning in two such games.

The traveler's dilemma

Two travelers submit claims. If they agree, both receive that amount. If their claims differ, both receive the lower claim, plus a reward for the lower claimant and an equal penalty for the higher claimant.

With the default claims of 2–100 and reward of 2, two claims of 100 earn each traveler 100. Yet claiming 99 against 100 earns 101. Against 99, the best choice is 98. The same reasoning leads all the way down to 2.

The repeated prisoner's dilemma

Two players repeatedly choose whether to cooperate or defect. Mutual cooperation benefits both, but each can gain by defecting while the other cooperates.

A strategy specifies when to start defecting. For example, strategy 10 cooperates for nine rounds, then defects. If the opponent defects earlier, the player responds by defecting from the next round. Strategy 1 defects immediately; the last strategy never defects first.

In the traveler's dilemma, higher strategies mean higher claims; in the prisoner's dilemma, they mean cooperating longer. In both games, against a known choice above the minimum, the best reply is one step below it; against the minimum, it is the minimum. The equilibrium is for both players to choose the minimum, even though both would earn more by continuing together.

Learning from others

The simulation runs two populations, each with 1000 agents by default. At each revision opportunity, one agent is chosen at random. Unless it experiments, it chooses the strategy that would earn most against the opponents it considers:

  • Sampling best response (Sampling BR). The agent observes a random sample of opponents and evaluates every candidate strategy against that same sample.
  • Best response to exact information (Exact BR). The agent knows exactly how common each strategy is and evaluates every candidate against the whole population.

Occasionally, an agent experiments instead, choosing a strategy at random. The two populations start identically and share the same revisers and experiments. Only the information used to choose best replies differs.

Time is measured in ticks: one tick contains as many revision opportunities as there are agents, so each agent revises once per tick on average.

Following an experiment

Start with 1000 travelers all claiming 2. Claims can range from 2 to 100, and the reward for undercutting is 2. Experiments occur in about one out of every 1000 revisions. Can these occasional random choices lift claims away from the minimum?

In the upper panel (Sampling BR), each traveler observes two travelers drawn at random before choosing a best reply. In the lower panel (Exact BR), each knows how common every claim is. Both populations receive the same experiments.

Read from left to right as time passes. Higher positions mean higher claims. Darker blue means more travelers using that claim; orange marks experiments choosing claims above 2.

The curve in the figure summarizes cooperation, labeled C/n in the simulator. It averages the lower claim over random pairs: 0 means the minimum of 2, and 1 means the maximum of 100.

How a high claim can spread

Suppose one traveler claims 100 while the other 999 claim 2. Exact BR still chooses 2. But a Sampling BR agent who observes claims 2 and 100 chooses 99. The high claimant counts as half the sample, even though it is only one traveler in a thousand.

Others can then observe that new claim of 99 and choose high claims too. Each reply can influence further replies. Undercutting pulls claims downward, while new experiments can lift them again.

This is one example run. Repeat the simulation to see which patterns recur.

Same experiments, different information1000 agents
Two populations of travelers start at the minimum claim of 2. High claims spread in the upper panel, which uses Sampling BR, while the lower panel, using Exact BR, remains near 2. Orange marks show shared experiments.
Cooperation over time In this run, cooperation rises and remains high under Sampling BR, while it stays near zero under Exact BR. The scale runs from 0 for minimum claims to 1 for maximum claims.
Sampling BRExact BR
Claims 2–100 · reward 2 · sample size 2 · about one experiment per 1000 revisions. Each panel shows 300 ticks.

Run the model

Suggested experiments

Try the illustrated setting yourself: load the model, press setup, then go. Click go again to pause, or use go once for a single tick. See below for the controls and things to try.

Runs in your browser; no NetLogo installation needed.Open separately

The two populations will appear here.

Start with the traveler's dilemma and watch for high claims spreading through sampled replies.

Use Original size (100%) for larger controls or Expand model for full screen. On a small screen, you can also open the model separately. Download the desktop version to run it in NetLogo.

Using the controls

Choose your settings and press setup to start a new run. Use go to run or pause, and go once to advance one tick. The last column below tells you whether a change needs a new setup.

Parameters in the simulator
ControlDefaultWhat it changesTakes effect
gameTraveler's dilemmaWhich game the two populations play.Next setup
horizon98Number of rounds in the prisoner's dilemma. In the traveler's dilemma, the maximum claim is horizon + 2, so 98 gives claims from 2 to 100.Next setup
reward2The bonus paid to the lower claimant, taken from the higher claimant.Next setup
payoff-T, payoff-R, payoff-P, payoff-S6, 5, 1, 0Prisoner's-dilemma payoffs for defecting alone, both cooperating, both defecting, and cooperating alone, respectively. Keep them in strictly descending order.Next setup
n-of-agents1000Number of agents in each population.Next setup
initial-stateEveryone at strategy 1Start everyone at the earliest strategy, everyone at the latest, or spread them randomly across strategies. Both populations start the same way.Next setup
final-time300How many ticks the run lasts, up to 5000.Next setup
sample-size2How many observations a Sampling BR agent uses to choose a reply.During a run
log10-prob-exp−3How often agents experiment. Higher values mean more experiments; −3 means about one in every 1000 revisions.During a run
mean-from100Where to start averaging cooperation. Increase it to leave out the beginning of a run.Immediately

Moving log10-prob-exp from −3 to −2 makes experiments ten times more frequent. The prob-exp monitor shows the resulting probability.

You can change sample size and experimentation during a run to see how the population responds. Use a fresh setup when comparing runs at fixed settings.

Reading the results

Strategy histories

The upper panel shows Sampling BR; the lower shows Exact BR. Time moves to the right, and higher strategies are higher up. Look for blue bands spreading, drifting downward, or being lifted by new experiments.

In the traveler's dilemma, the simulator numbers strategies from 1: strategy 1 is a claim of 2, strategy 2 is a claim of 3, and so on.

Darker blue means more agents; white means none. Orange marks an experiment with a strategy above 1. It records the experimental choice, even if that strategy disappears soon afterwards. The same orange marks appear in both populations.

In long runs, each column covers several ticks but shows only the last of them, including its orange marks. The cooperation curve and mean monitors still use every tick.

Cooperation, C/n

The curve averages over random pairs drawn from each population:

  • Traveler's dilemma: it measures how high the lower of the two claims is, on a scale from the minimum claim to the maximum.
  • Repeated prisoner's dilemma: it measures the fraction of rounds in which both players cooperate.

A value of 0 means minimum claims or immediate defection; 1 means maximum claims or cooperation throughout. High values require both members of a typical pair to use late strategies.

Average cooperation

The mean monitors average cooperation from mean-from to the present. For example, at time 300 with mean-from 100, they summarize ticks 100–300. Change the starting point to focus on the later part of a run. A dash means there are no results in the chosen window yet.

Things to try

Start with the traveler's dilemma and the settings used in the illustration. Press setup and go. Each run is different, so repeat settings to see which patterns recur.

1. Follow a rare experiment

Can an experiment leave a trace after the original experimenter has changed strategy? Find an orange mark at a high claim in the upper panel, then look to its right for claims slightly below it. Someone who observes the experiment may choose a high claim too, and that reply can influence others in turn. Look for bands that spread, drift down through undercutting, or are lifted by new experiments. Compare the lower panel.

2. Compare one observation with two

Make fresh runs with sample-size 1 and then 2. Does cooperation persist in the same way? With one observation, the reply is one strategy below the observed one, stopping at 1. With two, a rare high claim has two chances to enter the sample and can keep the reply high. Compare the bands and mean cooperation.

3. Change the frequency of experimentation

Set final-time to 1000 and mean-from to 500. Compare log10-prob-exp −4, −3 and −2: one experiment per 10,000, 1000 and 100 revisions on average. Are experiments frequent enough to replenish high claims before undercutting wears them away? How does Exact BR respond?

Finally, try log10-prob-exp 0: everyone experiments, so the two populations evolve identically. Press setup before each new run.

Downloads and citation

SBRUG is free software. You can run, inspect and modify the model under the GNU General Public License, version 3 or later.

Model citation

Luis R. Izquierdo and Segismundo S. Izquierdo (2026). SBRUG: Sampling Best Response in Undercutting Games. NetLogo model. https://doi.org/10.5281/zenodo.23734407

Related paper

Luis R. Izquierdo and Segismundo S. Izquierdo. How Sampling and Experimentation Sustain Cooperation in Undercutting Games.

Reproduction details for the illustration · Full license