Papers

Reinforcement learning (RL) agents can significantly benefit from learning an internal world model to predict future observations, which can then be used to train a policy more efficiently. We introduce World GPT, an auto-regressive world model that combines a semantic prior with a quantized latent space to capture complex environments more accurately and efficiently. In contrast to prior approaches, World GPT does not require any re-configuration of the model to generate multiple future frames. Instead, it can fully benefit from the latent space of a pre-trained VQ-GAN model, which can be trained independently of the RL task. Our experiments in the Atari 100K benchmark show that World GPT outperforms prior model-based approaches in terms of data efficiency and planning abilities in complex environments while reducing computational costs. Finally, we demonstrate that World GPT’s generation capabilities open up exciting new possibilities for exploration and real-world applications such as training free-form interactive agents.