[Paper Review] Move Evaluation in Go Using Deep Convolutional Neural Networks

Chris J. Maddison, Aja Huang+2 more|arXiv (Cornell University)|Dec 20, 2014

Artificial Intelligence in Games17 references92 citations

TL;DR

This paper proposes a 12-layer deep convolutional neural network (CNN) trained on human professional Go games to directly predict expert moves, achieving 55% move prediction accuracy—matching a 6 dan human player. Without any search, the CNN outperforms traditional programs like GnuGo and matches state-of-the-art Monte Carlo tree search (MCTS) programs, demonstrating that deep CNNs can effectively learn complex Go knowledge and serve as strong evaluation functions for gameplay.

ABSTRACT

The game of Go is more challenging than other board games, due to the difficulty of constructing a position or move evaluation function. In this paper we investigate whether deep convolutional networks can be used to directly represent and learn this knowledge. We train a large 12-layer convolutional neural network by supervised learning from a database of human professional games. The network correctly predicts the expert move in 55% of positions, equalling the accuracy of a 6 dan human player. When the trained convolutional network was used directly to play games of Go, without any search, it beat the traditional search program GnuGo in 97% of games, and matched the performance of a state-of-the-art Monte-Carlo tree search that simulates a million positions per move.

Motivation & Objective

• To investigate whether deep convolutional neural networks can learn and represent effective move evaluation functions for the game of Go.
• To determine if supervised learning from human professional games can yield a policy network that matches or exceeds human-level move prediction accuracy.
• To evaluate whether a trained CNN can directly play Go at a strong level without relying on search algorithms.
• To explore the integration of deep neural networks with Monte Carlo tree search (MCTS) for enhanced gameplay strength.

Proposed method

• A 12-layer deep convolutional neural network is trained using supervised learning on a large database of professional 19×19 Go games to predict the next move.
• The network uses a fully convolutional architecture with multiple residual blocks and ReLU activations, enabling end-to-end learning of spatial patterns in Go positions.
• Move prediction is performed by selecting the highest-probability output from the network’s final layer, representing the predicted move.
• For search integration, asynchronous node evaluation is employed: new MCTS nodes are batched and evaluated on a GPU, with results fed back into the search tree with a ~0.15s delay.
• The MCTS engine uses standard techniques such as UCT exploration, RAVE, and simple 3×3 pattern-based rollouts to guide search.
• Performance is evaluated by playing games between the CNN (with and without search) and baseline programs like GnuGo and Fuego.

Experimental results

Research questions

• RQ1 Can a deep convolutional neural network trained on human professional games achieve move prediction accuracy comparable to expert human players?
• RQ2 Can a deep CNN directly play Go at a strong level without using any search algorithm?
• RQ3 Can a deep CNN be effectively combined with Monte Carlo tree search to improve overall gameplay strength?
• RQ4 Does the knowledge learned by the CNN implicitly capture key Go concepts such as shape, joseki, tesuji, and influence?

Key findings

• The 12-layer CNN achieves a move prediction accuracy of 55% on a held-out test set, matching the performance of a 6 dan human player on the same data.
• When used without any search, the CNN wins 97% of games against GnuGo, a traditional search-based program.
• The CNN matches the performance of state-of-the-art MCTS programs like Pachi (10,000 rollouts per move) and Fuego (100,000 rollouts per move).
• When combined with MCTS using asynchronous GPU evaluation, the system wins 87% of games against the same CNN without search, using 100,000 rollouts per move.
• The network implicitly learns complex Go concepts such as fuseki, joseki, tesuji, ko fights, and influence, despite no explicit modeling of these elements.
• The network exhibits weaknesses in assessing global life-and-death status of large groups, suggesting complementarity with MCTS, which excels at global lookahead.