■ Block 01 / research
Research
Foundations other work depends on. Each entry explains what it is, why it is load-bearing, its open problems, and a reviewed timeline of progress.
Foundations other work depends on. Each entry explains what it is, why it is load-bearing, its open problems, and a reviewed timeline of progress.
Computing exact gradients of any program in roughly the cost of running it once.
A rule for scaling initialization and per-layer learning rates with width so that the best learning rate found on a small model is also near-best on a much larger one, avoiding tuning at full scale.
Adam-style adaptive optimizers rescale each parameter's step by running gradient statistics and train nearly every large model; matrix-aware successors (Shampoo, Muon) and schedule-free variants are the contenders.