An open FP8 GEMM kernel matching cuBLAS across LLM shapes on H100
Difficulty 24: the techniques (TMA, WGMMA, warp specialisation, Stream-K) are public in CUTLASS and DeepGEMM, and the gap is confined to small-M and odd shapes, so it sits with the implementable max-flow problem (30) as engineering rather than research. Application value 50: GEMM carries most LLM FLOPs, but parity replaces an existing vendor library rather than enabling something new, so it ranks below decode bandwidth (62). Leverage 29: fused epilogues, grouped MoE GEMMs and FP8 attention reuse the work; nothing theoretical hinges on it.
The problem
Can an open FP8 matrix-multiply program run as fast as NVIDIA's closed library on every size that matters?
- 1Closed library fast, open kernels partial
- 2Fix a sweep of real LLM shapes
- 3Warp-specialised TMA and WGMMA design
- 4Tune thin and odd shapes
- 5Reproduce at 97% of cuBLAS
What is asked. Matrix multiplication () is where a language model spends most of its arithmetic. NVIDIA's H100 can do it in FP8, 8-bit floating-point numbers, at about twice the rate of 16-bit formats (see the entries on tiled GEMM and FP8). The fastest FP8 code is in cuBLAS, NVIDIA's closed library. The problem asks for an open kernel, whose source anyone can read and change, that reaches at least 97% of cuBLAS speed on matrix shape in a fixed list taken from real models, with matching accuracy and the same scaling scheme (one scale per tensor, or one per 1×128 or 128×128 block).
Path to understanding
foundations → problemThe research entries this problem rests on, in the order to read them. Each links to its explainer.
- Start with
- Step 2
- Tiled GEMM on tensor cores (CUTLASS and CuTe)Algorithm · host
- Then
- An open FP8 GEMM kernel matching cuBLAS across LLM shapes on H100The problem
State of the art
cuBLAS 12.9 added 1x128 and 128x128 block-scaled FP8 GEMMs on Hopper, so per-tensor and block-scaled kernels now both have a vendor baseline. DeepGEMM's original benchmark (H800, February 2025) compared against DeepSeek's internal CUTLASS-based kernels, not cuBLAS, and reported its largest gains at small M (1.4–2.7x at M = 64) and 1.0–1.2x at M = 4096; its current README claims it matches or exceeds expert-tuned libraries without a per-shape cuBLAS table. Small-M (decode-like) shapes and non-multiple-of-tile sizes are where open kernels most often fall behind, because wave quantisation and memory-bound behaviour dominate there. No primary source known to the curator documents a kernel meeting the 97%-on-every-shape bar.
Realistic applications
FP8 inference and training are the main consumers of GEMM FLOPs; an open, inspectable kernel lets researchers modify scaling, fuse epilogues and port to new hardware without a vendor binary.
What counts as solved
Two independent reruns of the stated harness on H100 SXM report the kernel at or above 97% of cuBLAS on every shape, with correctness checks passing, from a public repository at a pinned commit.
How it gets verified
GPU benchmarkA kernel run and timed on stated hardware against a baseline.
Hardware: one NVIDIA H100 SXM 80 GB, default clocks, power cap recorded. Dtypes: E4M3 A and B, FP32 accumulation, BF16 C; per-tensor scaling and 1x128 / 128x128 block scaling variants reported separately. Shapes: the sweep in the statement, K up to 16384. Baseline: cuBLAS (cublasLt FP8 matmul, per-tensor scaling and the VEC128 / BLK128x128 block-scale modes) from CUDA 12.9 or later, exact version recorded, default (non-fast) accumulation. Metric: median TFLOP/s over at least 100 timed iterations after warmup, CUDA events, three independent runs; also % of the datasheet dense FP8 peak. Correctness: max relative error to an FP32-dequantised reference no worse than 1.1x the cuBLAS kernel's own error on the same inputs. Harness: a public script at a pinned commit that also times cuBLAS in the same process.
Claims under verification
+ add a claimNo claimed results on the Verification board for this problem yet.
Where the work is
Mostly kernel engineering with a measurement loop on real hardware; some tuning compute and a little new scheduling insight.
Bottlenecks
ranked by votes- #1EngineeringSmall-M and odd-shape efficiency
Decode-like shapes are memory-bound and non-multiple-of-tile shapes suffer wave quantisation, so one tile configuration cannot be near-peak everywhere.
Community grades
Sign in to add your own grade.