■ Block 01 / research

Research

Foundations other work depends on. Each entry explains what it is, why it is load-bearing, its open problems, and a reviewed timeline of progress.

Propose an entry

Low-Precision Numerics</> 3 reposTheoryEmerging

Microscaling (MX) and block-scaled 4-bit formats

Data formats that share one scale across a small block of narrow elements (MXFP8, MXFP6, MXFP4, MXINT8, and NVIDIA's NVFP4), implemented in Blackwell tensor cores.

2 problems0 dependents0 maintainers

Low-Precision Numerics</> 2 reposTheoryActive

FP8 formats and scaling recipes for training and inference

The two 8-bit floating-point encodings (E4M3 and E5M2), why tensors need scale factors, and the recipes (per-tensor, delayed, fine-grained blockwise) that make FP8 GEMMs usable for LLMs.

1 problem1 dependent0 maintainers

Research

Microscaling (MX) and block-scaled 4-bit formats

FP8 formats and scaling recipes for training and inference

Microscaling (MX) and block-scaled 4-bit formats

FP8 formats and scaling recipes for training and inference

Kernel autotuning and automated kernel generation

Triton: a tile-level language and compiler for GPU kernels

Paged attention and KV-cache kernels

FlashAttention: IO-aware exact attention

Tiled GEMM on tensor cores (CUTLASS and CuTe)

The GPU memory hierarchy and the roofline model

Tensor, data and pipeline parallelism with compute-communication overlap

NCCL and collective communication: ring and tree all-reduce