Reinforcement Learning

Agents that learn from reward, from game-playing systems to the fine-tuning of language models.

Open problems, most leverage first

All open problems →

#VotesProblemWork mixDVLStatus

010Does RLVR create new reasoning ability or only sharpen the base model? A pass@k protocol that separates the twoReproductionRL with verifiable rewards and GRPO for reasoning models1 link3 bottlenecks
506052OpenD 50V 60L 52▲ 0
020Selecting the best offline-RL policy on D4RL with no environment interactionReproductionOffline reinforcement learning2 bottlenecks
504836OpenD 50V 48L 36▲ 0
030Do RL compute-performance curves for LLMs extrapolate across models, domains and recipes?ReproductionRL with verifiable rewards and GRPO for reasoning models1 link2 bottlenecks
325035OpenD 32V 50L 35▲ 0
040Do reward-model overoptimization laws hold for modern RLHF and DPO-style training?ReproductionRLHF and preference optimization (DPO)1 link2 bottlenecks
304030OpenD 30V 40L 30▲ 0
050Independent reproduction and improvement of offline-only Minecraft diamond collection with a world modelReproductionModel-based RL and world models (MuZero, Dreamer)3 bottlenecks
453530OpenD 45V 35L 30▲ 0
060A controlled DPO versus PPO comparison that attributes the gap to online sampling or to the reward modelReproductionRLHF and preference optimization (DPO)1 link2 bottlenecks
254228OpenD 25V 42L 28▲ 0
070Close the ProcGen generalisation gap: agents trained on 200 levels matching agents trained on unlimited levelsReproductionRL benchmarks and evaluation: Atari 100k, ProcGen, rliable1 link2 bottlenecks
483028OpenD 48V 30L 28▲ 0
080Tight iteration complexity of exact softmax policy gradient in tabular discounted MDPsBound / recordPolicy gradient methods and PPO1 bottleneck
45818OpenD 45V 8L 18▲ 0

Research

Filter in Research →

Reinforcement Learning</> 3 reposLibraryActive

RL benchmarks and evaluation: Atari 100k, ProcGen, rliable

The shared environments and statistics used to compare RL agents: the Arcade Learning Environment and its 100k-step variant for sample efficiency, ProcGen for generalisation, rliable for intervals.

2 problems0 dependents0 maintainers

Reinforcement Learning</> 2 reposAlgorithmActive

Offline reinforcement learning

Learning a policy from a fixed dataset of past interactions with no further data collection. The central difficulty is distribution shift: the learned policy queries actions the data never covered.

Open problems, most leverage first

Research

RL benchmarks and evaluation: Atari 100k, ProcGen, rliable

Offline reinforcement learning

Open problems, most leverage first

Research

RL benchmarks and evaluation: Atari 100k, ProcGen, rliable

Offline reinforcement learning

Model-based RL and world models (MuZero, Dreamer)

Monte Carlo tree search and AlphaZero

RL with verifiable rewards and GRPO for reasoning models

RLHF and preference optimization (DPO)

Policy gradient methods and PPO