Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worlds (NeurIPS Spotlight Paper)

5 min read

last updated 15 Jun 2026

What Is Avalon?

Avalon is a benchmark for generalization in RL

Agents in Avalon must accomplish a wide range of tasks, all with the same sparse reward structure. Different tasks correspond to randomly generated worlds that require that skill.

Twenty tasks test a variety of general skills

See the paper and presentation (below) for more details about the benchmark.

Avalon is a fast, easy-to-use 3D simulator for constructing RL environments

The Avalon benchmark is built on top of a simulator that we created specifically to suit the needs of RL researchers. Features:

• Fast: can simulate up to 10,000 steps per second on a single GPU.
• Fully open source: built on top of the free, open source Godot game engine.
• Easy to use: includes a fully featured editor for creating and debugging new environments and game logic.
• Vibrant community: Godot has thousands of online tutorials, great docs, and a large base of existing users.
• Simple: the entire game engine is roughly 30MB including rendering, physics, etc, and Avalon is only a few thousand lines of additional code.
• Baselines included: Avalon includes clean implementations of PPO, Impala, Dreamer v2, and BYOL-Explorer, most of which have been verified to replicate the original paper results.
• Accessible: state-of-the-art agents can be trained with a single GPU.

Who Should Use Avalon?

If you want a more challenging RL benchmark

Avalon can be used as a largely drop-in replacement for Atari or other standard RL benchmarks, simply see this tutorial to get started.

If you want to do research on generalization in RL

Avalon is a great place to start, as it includes highly tuned, easy-to-understand implementations of a variety of popular and high-performance RL algorithms. See this tutorial to replicate our training.

If you want to develop your own RL tasks

Avalon is an extremely fast, extremely easy-to-use platform for research, built on top of the fully open source Godot game engine. See this tutorial to create a new environment from scratch.

Getting Started

Play in the environment

Avalon is incredibly easy to try—just download 30MB for your platform, unzip, and run.

Run experiments

To get started with Avalon, simply run the below in your own notebook!

from avalon.agent.godot.godot_gym import AvalonEnv
from avalon.agent.godot.godot_gym import GodotEnvironmentParams
from avalon.agent.godot.godot_gym import TrainingProtocolChoice
from avalon.datagen.env_helper import display_video

env_params = GodotEnvironmentParams(
    resolution=256,
    training_protocol=TrainingProtocolChoice.SINGLE_TASK_MOVE,
    initial_difficulty=0,
)
env = AvalonEnv(env_params)
env.reset();

def random_env_step():
    action = env.action_space.sample()
    obs, reward, done, info = env.step(action)
    if done:
        env.reset()
    return obs

observations = [random_env_step() for _ in range(50)]
display_video(observations, fps=10)

Tutorials

See the full documentation and source code on our Github repository, or try one of these tutorials:

• Running Avalon as a Gym environment
• Training a PPO agent to play Avalon using RLLib
• Training our PPO and Dreamer RL baseline agents
• Creating a new procedurally generated Avalon benchmark task
• Creating an RL environment from scratch using Godot

Learn More

Presentation

Our paper on Avalon was published at NeurIPS 2022. We will be presenting it in person.

Paper

Check out our paper for all the details on the environment and tasks, along with human and RL baselines (PPO, IMPALA, and Dreamer v2).

To cite Avalon, use the following:

@inproceedings{Avalon,
    title="{Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worlds}",
    author={Albrecht, Josh and Fetterman, Abraham J. and Fogelman, Bryden and Kitanidis, Ellie and Wróblewski, Bartosz and Seo, Nicole and Rosenthal, Michael and Knutins, Maksis and Polizzi, Zachary and Simon, James B. and Qiu, Kanjun},
    booktitle={NeurIPS Datasets and Benchmarks Track},
    year={2022},
}