[Paper Review] The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition

Diego Pérez-Liébana, Katja Hofmann+6 more|arXiv (Cornell University)|Jan 23, 2019

Reinforcement Learning in RoboticsComputer Science11 references54 citations

TL;DR

The MARLÖ competition proposes a multi-agent RL benchmark in multiple Minecraft-based 3D games to promote agents that generalize across games, tasks, and opponent types, evaluated through a play-off tournament.

ABSTRACT

Learning in multi-agent scenarios is a fruitful research direction, but current approaches still show scalability problems in multiple games with general reward settings and different opponent types. The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) competition is a new challenge that proposes research in this domain using multiple 3D games. The goal of this contest is to foster research in general agents that can learn across different games and opponent types, proposing a challenge as a milestone in the direction of Artificial General Intelligence.

Motivation & Objective

• Foster research in general, multi-game multi-agent reinforcement learning.
• Develop agents capable of learning across multiple 3D games and varied opponent types.
• Provide a configurable task space with multiple instances to avoid overfitting to a single task.

Proposed method

• Define three Minecraft-based games (Mob Chase, Build Battle, Treasure Hunt) with collaborative and competitive elements.
• Provide a starter kit and test tasks to accelerate development and iteration.
• Use a round-robin play-off tournament to evaluate agents across games and tasks.
• Utilize highly parameterizable task configurations to create diverse variants of each game.
• Require agents to perform well across multiple games and against multiple other agents, discouraging overfitting.

Experimental results

Research questions

• RQ1 How well can agents generalize across different games within MARLÖ?
• RQ2 To what extent do agents generalize across different task variants within each game?
• RQ3 How robust are agents to different opponent types in multi-agent settings?
• RQ4 Can agents composed for generality outperform game-specific agents in the MARLÖ benchmark?

Key findings

• MARLÖ aims to advance generalization in multi-agent RL by testing across multiple games, tasks, and opponents.
• The competition provides a public benchmark, starter kit, and baselines to ease entry and evaluation.
• Final rankings are determined by a play-off tournament over all three games and multiple tasks, encouraging cross-game proficiency.