Independent reproduction and improvement of offline-only Minecraft diamond collection with a world model
Difficulty 45: hundreds of TPUs in the original, no official code, and a 0.7% success rate that needs thousands of hour-long episodes to measure; the science (whether imagination training from offline data scales) is open. Application value 35: offline-to-policy training matters for robotics. Leverage 30: it tests one research programme.
The problem
Check, and then improve, the claim that an AI learned to mine Minecraft diamonds from videos alone.
- 1Watch 2,500 hours of recordings
- 2Learn a world model
- 3Practise inside the model
- 4Test 3,000 times in Minecraft
- 5Compare with the 0.7% claim
Why it matters. Robots and other real systems cannot afford millions of trial-and-error attempts, but large logs of recorded behaviour often exist. If an agent can learn a world model, a learned simulator, from recordings and then practise inside it, learning needs no risky interaction at all. Dreamer 4 is the most striking test of this idea so far.
Path to understanding
foundations → problemThe research entries this problem rests on, in the order to read them. Each links to its explainer.
- Start with
- Step 2
- Policy gradient methods and PPOAlgorithm
- Step 3
- Step 4
- Model-based RL and world models (MuZero, Dreamer)Algorithm · host
- Then
- Independent reproduction and improvement of offline-only Minecraft diamond collection with a world modelThe problem
State of the art
The preprint reports diamonds in 0.7% of 1,000 evaluation episodes and an iron pickaxe in 29%, and says Dreamer 4 substantially outperforms the offline VPT agent while using 100x less data than VPT. Training used 256 to 1,024 TPU-v5p devices. No official code or weights have been released (the project page links only the paper); unofficial reimplementations exist (for example nicklashansen/dreamer4 and lucidrains/dreamer4) but none reports the diamond result. A 0.7% rate has wide uncertainty at small episode counts.
Realistic applications
Training agents in a learned simulator from logged data is the template for robotics and other domains where online interaction is unsafe or slow.
What counts as solved
A released implementation whose offline-only agent, trained only on the VPT contractor data, obtains diamonds in at least 0.7% of at least 3,000 evaluation episodes (confidence interval overlapping the reported figure), and a follow-up result whose interval lies entirely above it.
How it gets verified
ReproductionA training run, experiment or computation rerun by someone else.
Environment: Minecraft with the VPT contractor dataset (about 2,541 hours) as the only training data, no environment interaction during training. Metric: fraction of 60-minute evaluation episodes in which the agent obtains a diamond, with at least 3,000 episodes per model (about 0.3 percentage points half-width at a 0.7% rate), 95% Wilson confidence interval, and at least 3 independent training seeds. Compute: report accelerator type and hours; the original used 256 to 1,024 TPU-v5p devices, so a scaled-down variant must say which components were reduced.
Claims under verification
+ add a claimWhere the work is
Needs training of a large world model and a long-horizon RL stage; engineering and compute dominate.
Bottlenecks
ranked by votes- #1ComputeAccelerator scale
The original used hundreds of TPU-v5p chips; a smaller reproduction must show which components survive scaling down.
Community grades
Sign in to add your own grade.
Connected problems
No connections yet.
Progress
- Sep 29, 2025Dreamer 4 preprintsource
Reports diamonds from offline data alone, 0.7% of 60-minute episodes.