# Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worlds (NeurIPS Spotlight Paper)

5 min read

_last updated 15 Jun 2026_

## What Is Avalon?

### Avalon is a benchmark for generalization in RL

Agents in Avalon must accomplish a wide range of tasks, all with the same sparse reward structure. Different tasks correspond to randomly generated worlds that require that skill.

### Twenty tasks test a variety of general skills

See the paper and presentation (below) for more details about the benchmark.

### Avalon is a fast, easy-to-use 3D simulator for constructing RL environments

The Avalon benchmark is built on top of a simulator that we created specifically to suit the needs of RL researchers. Features:

- **Fast**: can simulate _up to 10,000 steps per second_ on a single GPU.
- **Fully open source**: built on top of the _free, open source Godot game engine_.
- **Easy to use**: includes a _fully featured editor_ for creating and debugging new environments and game logic.
- **Vibrant community**: Godot has _thousands of online tutorials_, great docs, and a large base of existing users.
- **Simple**: the entire game engine is roughly _30MB including rendering, physics, etc_, and Avalon is only a few thousand lines of additional code.
- **Baselines** included: Avalon includes _clean implementations of PPO, Impala, Dreamer v2, and BYOL-Explorer_, most of which have been verified to replicate the original paper results.
- **Accessible**: state-of-the-art agents can be _trained with a single GPU_.

## Who Should Use Avalon?

### If you want a more challenging RL benchmark

Avalon can be used as a largely drop-in replacement for Atari or other standard RL benchmarks, simply see [this tutorial](https://github.com/Avalon-Benchmark/avalon/blob/main/notebooks/gym_interface_example.sync.ipynb) to get started.

### If you want to do research on generalization in RL

Avalon is a great place to start, as it includes highly tuned, easy-to-understand implementations of a variety of popular and high-performance RL algorithms. See [this tutorial](https://github.com/Avalon-Benchmark/avalon/blob/main/docs/avalon_baselines.md) to replicate our training.

### If you want to develop your own RL tasks

Avalon is an extremely fast, extremely easy-to-use platform for research, built on top of the fully open source Godot game engine. See [this tutorial](https://github.com/Avalon-Benchmark/avalon/tree/main/docs/tutorial_custom_env) to create a new environment from scratch.

## Getting Started

### Play in the environment

Avalon is incredibly easy to try—just [download 30MB for your platform](https://github.com/Avalon-Benchmark/avalon/releases/tag/demo), unzip, and run.

### Run experiments

To get started with Avalon, simply run the below in your own notebook!

```
from avalon.agent.godot.godot_gym import AvalonEnv
from avalon.agent.godot.godot_gym import GodotEnvironmentParams
from avalon.agent.godot.godot_gym import TrainingProtocolChoice
from avalon.datagen.env_helper import display_video
```

```
env_params = GodotEnvironmentParams(
    resolution=256,
    training_protocol=TrainingProtocolChoice.SINGLE_TASK_MOVE,
    initial_difficulty=0,
)
env = AvalonEnv(env_params)
env.reset();
```

```
def random_env_step():
    action = env.action_space.sample()
    obs, reward, done, info = env.step(action)
    if done:
        env.reset()
    return obs

observations = [random_env_step() for _ in range(50)]
display_video(observations, fps=10)
```

### Tutorials

See the full documentation and source code on our [Github repository](https://github.com/Avalon-Benchmark/avalon), or try one of these tutorials:

- [Running Avalon as a Gym environment](https://github.com/Avalon-Benchmark/avalon/blob/main/notebooks/gym_interface_example.sync.ipynb)
- [Training a PPO agent to play Avalon using RLLib](https://github.com/Avalon-Benchmark/avalon/blob/main/notebooks/rllib_tutorial.sync.ipynb)
- [Training our PPO and Dreamer RL baseline agents](https://github.com/Avalon-Benchmark/avalon/blob/main/docs/avalon_baselines.md)
- [Creating a new procedurally generated Avalon benchmark task](https://github.com/Avalon-Benchmark/avalon/tree/main/docs/tutorial_add_task)
- [Creating an RL environment from scratch using Godot](https://github.com/Avalon-Benchmark/avalon/tree/main/docs/tutorial_custom_env)

## Learn More

### Presentation

Our paper on Avalon was published at NeurIPS 2022. We will be presenting it in person.

### Paper

Check out [our paper](https://openreview.net/pdf?id=TzNuIdrHoU) for all the details on the environment and tasks, along with human and RL baselines (PPO, IMPALA, and Dreamer v2).

To cite Avalon, use the following:

```
@inproceedings{Avalon,
    title="{Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worlds}",
    author={Albrecht, Josh and Fetterman, Abraham J. and Fogelman, Bryden and Kitanidis, Ellie and Wróblewski, Bartosz and Seo, Nicole and Rosenthal, Michael and Knutins, Maksis and Polizzi, Zachary and Simon, James B. and Qiu, Kanjun},
    booktitle={NeurIPS Datasets and Benchmarks Track},
    year={2022},
}
```