Comparison of human scores and “human scores” in Atari

Below is a list of the different scores reported for humans on Atari games in the reinforcement learning community and the highest scores reported in TwinGalaxies. The conditions under which these corresponding scores are achieved are different: the RL community tests humans on an emulator using the keyboard, the game can only last up to 5 minutes and there’s no sound (ie they replicate the conditions that the algorithm experiences). It also reports the average, not the maximum values obtained. On the other hand, TwinGalaxies is a site that collects records, so it is only interested in maximum values. The differences in the scores tends to be very big, often by more than 100 times.

I think the maximum values achieved are important benchmarks reflecting the levels that can be achieved by a human brain finely trained on a task. It’s also quite apparent that, even though the reported records are maxima, the average score for the people who achieved these records would not be anywhere near as low as the averages reported for humans in the RL literature. The five minute mark will become a problem in trying to beat these top scores, but that time limit can be removed. It doesn’t tend to be a bottleneck in most cases at the moment since the agents can rarely play for five minutes.

In the following table, “DNADRL human” refers to the human scores reported in “Dueling Network Architectures for Deep Reinforcement Learning” by Ziyu Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot and Nando de Freitas https://arxiv.org/abs/1511.06581. Similar scores are seen in other papers in the literature. TwinGalaxies (any) is the highest score for that game with difficulty setting B achieved in either a console or an emulator, although they almost completely consist of console scores. Factor is just the TwinGalaxies score divided by the DNADRL score.