How the best color-gamers got there

The posts on this blog explore the digital life of the Color Game, a gaming app launched by our lab. Its goal: inventing a universal language without words, and recording its birth in data. To find out more, visit colorgame.net.

The Color Game’s players have been getting better at playing the game. How did they do it? Practice seems an obvious answer. But what kind of practice?

When two players play together, their level of practice takes two distinct aspects. One is the pair’s experience as a pair: the total number of trials that both players played together. This “joint experience” sums up all the trials where they played jointly, developing their own habits and conventions.

There is another kind of practice to consider, though: each of the two players has also gained experience outside the pair, playing with other people. This “general experience” is an occasion to become a better player, independently of any particular pair’s progress.

I wanted to check that the players’ progress in the Color Game is due to the creation of shared conventions—that players didn’t simply improve by playing a lot, independently of establishing conventions with other players. To make sure of this, I looked at the performance of more than 4000 pairs of players*, to see which matters more: joint experience (the pair’s level of practice as a pair) or general experience (the pair’s players experience, inside or outside the pair).

Figure 1. Player pairs’ performances as a function of their joint eperience. Each dot stands for a pair of players (n = 4074 pairs, from 1412 players). Y-axis: the pair’s chance of scoring a hit (0 to 1, chance level = 0.25). X-axis: Joint experience: the number of trials played by the pair as a pair. The colour and size of each dot represents the pair’s general experience: The number of trials played by either player, separately or jointly. Scroll over the dots to see the pair’s nationalities.

Joint experience, the pair’s level of practice as a pair, correlates with better performance. This could be due to a tendency for players to keep playing with the partners they achieved high scores with. But this seems unlikely: the pairs with very high joint experience perform at extremely high levels (80% success rates being common) which pairs with less joint experience seldom reach.

General experience does not seem to matter as much as joint experience. That, at least, is what Figure 2 suggests. That is surprising, since pairs tend to differ greatly in their level of general experience. Some pairs’ players total more than 30 000 trials of experience together, while many only have 1 000 trials or less. These huge differences appear not to have such a huge effect. This is shown in Figure 2, which plots performance against a pair of players’ general experience, gained together or separately.

Figure 2. Player pairs’ performances as a function of their past performance playing together as a pair. Each dot stands for a pair of players (n = 4074 pairs, from 1412 players). Y-axis: the pair’s chance of scoring a hit (0 to 1, chance level = 0.25). X-axis: The number of trials played by the pair’s players, separately or together. The colour and size of each dot represents the pair’s joint experience: The number of trials played by either player, separately or jointly. Scroll over the dots to see the pair’s nationalities.

(Most performance scores cluster around round values (0.1, 0.2, etc.), because many pairs only play 10 or 20 trials together. The odd shape of the data in the 10K – 20 K trials range is probably due to the fact that a few highly experienced players engaged a lot of players with very little experience.)

More sophisticated analyses would be needed to explore the effects of joint practice in greater detail. I’ll stop here, though: this blog isn’t meant to report statistical tests. We reserve in-depth testing for the hypotheses we preregistered on the Open Science Framework before we launched the game. Joint experience isn’t something we made specific predictions about, but it’s good to see that Color-Gamers progress by building shared conventions.

The message for Color Gamers: Practice is good, joint practice is better!

(Text and graphs: Olivier Morin)

* All pairs that played from the game’s pre-launch (26/04/2018) up to the 22d of June. As Color-Gamers know, beginning players gain new keyboard symbols as they become more experienced, which often makes the game easier for them. I wasn’t very interested in measuring this, so I only studied all the pairs of players where one of the players had reached the level at which all 35 symbols become available. Other pairs were not included.

Contents