World model: a playable dream of a room
A small room has boxes, a flashlight and fireflies. We record it, then train a model to dream it. The model gets the last frames and your key press, and it draws the next frame. In the demo below, every frame is generated by the model.
Model: VQ-VAE + 9.8M-parameter spatio-temporal transformer · Runs in your browser
Play inside the dream
The model downloads once (45 MB). It needs a browser with WebGPU (Chrome, Edge, Safari 26) for a smooth frame rate. Without WebGPU, it runs on the CPU at about one frame per second.
Keys: click the picture, then use the arrow keys. ↑ moves forward, ↓ moves back, ← and → turn the flashlight.
There is no game code here. The walls, the shadows and the fireflies come from the network only. Errors can add up over time: the room can slowly change. Click New room to start again from real frames.
What the project does
A world model predicts what happens next in an environment. Here the environment is FireflyRoom, a small 2D game we wrote:
- 2 or 3 boxes that do not move.
- A flashlight that you drive like a tank (forward, back, turn left, turn right). The light makes shadows behind the boxes.
- 1 or 2 fireflies that fly around and bounce off the walls. They move without your input.
The model must learn three things from recordings only: the room stays the same, the light follows your keys, and the fireflies move on their own.
Architecture
The model has two parts. Part 1, the tokenizer, turns each image into a small grid of integers. Part 2, the dynamics model, predicts the next grid of integers.
Part 1: the tokenizer (VQ-VAE)
A VQ-VAE is an autoencoder with a codebook. The encoder (a CNN) turns a 64 × 64 frame into a 16 × 16 grid of vectors. Each vector is replaced by the nearest of 512 codebook vectors. So each frame becomes 256 integers from 0 to 511. The decoder (a CNN) turns the 256 integers back into an image.
Why do this? Predicting 256 integers is much easier than predicting 12,288 pixel values. Each token also carries meaning, for example "box edge" or "dark floor".
Part 2: the dynamics model
The dynamics model is a transformer in the style of Genie. It reads 7 past token grids and the actions between them. Then it predicts the 256 tokens of the next grid.
- Spatio-temporal blocks. Full attention over all 7 × 256 tokens is expensive. Each block does two cheaper steps. Spatial attention: tokens in one frame look at each other. Temporal attention: each grid cell looks at the same cell in past frames. This attention is causal: a frame cannot look at the future.
- 3D RoPE. Rotary position embeddings tell the model where each token is: row, column and time. In spatial attention, half of each head rotates by the row and the other half by the column. In temporal attention, the head rotates by time. This was the largest improvement in our experiments.
- Context frames. The model sees the frames at t−1, t−2, t−3, t−4, t−8, t−16 and t−32. The recent frames show motion. The old frames help the model remember the room.
- Anchor frame. The first real frame stays in the context for the full rollout, with its true distance in time. It is a fixed memory of how the room looks.
- Action tokens. Each frame gets an embedding of the key that was pressed before it. There are 5 actions plus one "no action" token.
- MaskGIT-style output. The model predicts all 256 tokens of the next frame in parallel, not one at a time. MaskGIT can refine the frame in several passes, but one pass gave the best rollouts here.
Training
Data
A scripted player drives the flashlight in random rooms. We record 4,000 episodes of 256 steps: about one million frames with the key for each step.
python record.py --episodes 4000 --steps 256 --out data_long python pack.py --data data_long --out cache_long
Step 1: train the tokenizer
- Loss: L1 on pixels, plus L1 on image gradients (edges). Edges keep the boxes, shadows and the light beam sharp.
- Fireflies count 20 times more. A firefly is only a few pixels. Without the extra weight, the tokenizer removes it.
- Codebook: 512 codes of size 64, updated with an exponential moving average.
- No GAN loss. We tried an adversarial loss. Training became unstable, so we removed it.
Then we encode every frame one time. The dynamics model trains on integers only.
python vqvae.py --out runs/vqvae_ft --gan_start 999999 --fly_w 20 python tokenize_frames.py --cache cache_long --vqvae runs/vqvae_ft/vqvae.pt
Step 2: train the dynamics model
- Take a window of real token grids and actions from an episode.
- Hide some tokens of the target frame with a [MASK] token. The amount follows the MaskGIT cosine schedule. In half of the samples, all target tokens are hidden, the same as at play time.
- Replace up to 10% of the context tokens with random tokens.
- Predict the hidden tokens. The loss is cross-entropy.
- Weight the loss: tokens that change from the last frame count 3 times more. Tokens near the flashlight and fireflies count 9 times more. Most tokens (floor, walls) never change and are easy.
| Setting | Value |
|---|---|
| Size | width 256, 6 blocks, 8 heads, 9.8M parameters |
| Optimizer | AdamW (β = 0.9, 0.95), learning rate 2e-4, one-cycle schedule, gradient clip 1.0 |
| Steps / batch | 10,000 steps, batch 64 |
| Start weights | A first-version model (learned positions, 4 frames of context) |
python train_rope.py --cache cache_long --d 256 --layers 6 --batch 64 --steps 10000 --out runs/rope1
Why the dream does not melt
With plain teacher forcing, the model only sees perfect real frames during training. At play time, it sees its own imperfect frames. Small errors then grow, and after a few frames the room melts. Two changes fixed this:
- Train on its own predictions. The first-version model fed back its own predicted frames during training. The final model starts from those weights.
- Token noise. Random tokens in the context (step 3 above) teach the model to repair small errors.
Running it in the browser
- We export two ONNX files: one prediction step of the dynamics model (39 MB) and the VQ-VAE decoder (6 MB).
- The ONNX rollout gives the same images as PyTorch: the mean pixel difference over 40 frames is 0.000.
- Each frame is one dynamics step and one decode in onnxruntime-web (WebGPU, or WebAssembly on the CPU).
- The 8 start rooms are real frames that we encoded before. After the start, the model makes all the frames.
Papers and links
- Genie: Generative Interactive Environments (Bruce et al., 2024). The tokenizer + spatio-temporal transformer design.
- Neural Discrete Representation Learning (VQ-VAE) (van den Oord et al., 2017). The tokenizer.
- MaskGIT: Masked Generative Image Transformer (Chang et al., 2022). Parallel token prediction with masks.
- RoFormer: Enhanced Transformer with Rotary Position Embedding (Su et al., 2021). RoPE.
- World Models (Ha and Schmidhuber, 2018). The original "learn to dream an environment" idea.
- Diffusion for World Modeling: Visual Details Matter in Atari (DIAMOND) (Alonso et al., 2024). A different method (diffusion) for the same task.