A Triton block-scaled FP4 GEMM within 10% of CUTLASS on B200
Difficulty 31: Triton must place scale factors and drive tensor-memory MMAs without exposing them, on hardware where tile-level Python GEMMs still trail cuBLAS in BF16; harder than the H100 BF16 Triton task (27). Application value 43: open FP4 kernels in a compiler language help open 4-bit stacks, but CUTLASS and its Python DSL already cover the need. Leverage 23: limited to Blackwell low-precision kernels.
The problem
Can a Triton matrix multiply on 4-bit block-scaled numbers reach 90% of NVIDIA’s CUTLASS speed on a B200?
- 1B200 multiplies 4-bit blocks natively
- 2CUTLASS and its Python DSL are fast
- 3Triton hides layouts from the programmer
- 4Compiler must place scales correctly
- 590% of CUTLASS on every shape
What is asked. NVIDIA's Blackwell GPUs (the B200) can multiply matrices of 4-bit numbers directly. The numbers come in blocks of 16 (NVFP4) or 32 (MXFP4), each block with its own scale factor (see the microscaling entry). NVIDIA's CUTLASS library has fast kernels for this, written in C++ templates. The problem asks for the same operation written in , a Python-level tile language, using only features in a released version (such as ). It must reach at least 90% of CUTLASS speed on every matrix shape in a list taken from real models, with from 1 to 16384. NVIDIA's own CuTe Python DSL is excluded, because it exposes the same low-level controls as CUTLASS. The question is whether a compiler that hides those details can still reach vendor speed.
Built from
earlier problems → this oneThe problems this one is built from: progress or tools it needs (builds on), special cases that come first (generalizes), and weaker results on the way (implies). On the right, what builds on it.
- Start with
- A Triton block-scaled FP4 GEMM within 10% of CUTLASS on B200This problem
- Then
- An open 4-bit pretraining recipe matching BF16 loss at 1B parametersLow-Precision Numerics · D30
Path to understanding
foundations → problemThe research entries this problem rests on, in the order to read them. Each links to its explainer.
- Start with
- Step 2
- Step 3
- Step 4
- Microscaling (MX) and block-scaled 4-bit formatsTheory · host
- Then
- A Triton block-scaled FP4 GEMM within 10% of CUTLASS on B200The problem
State of the art
CUTLASS ships block-scaled kernels for Blackwell, and its CuTe Python DSL includes block-scaled (NVFP4 and MXFP4) GEMM examples; NVIDIA reports CuTe DSL dense GEMMs on par with C++. Triton has a block-scaled matmul tutorial (10-block-scaled-matmul.py, using tl.dot_scaled) that benchmarks against cuBLAS but publishes no numbers. The only independent Triton-versus-vendor GEMM figures on B200 found are for BF16, where an evaluation of CUDA Tile (arXiv:2604.23466) reports tile-level Python kernels well below cuBLAS on several LLM shapes. No source benchmarks a Triton FP4 GEMM against CUTLASS across a shape sweep.
Realistic applications
Efficient open 4-bit kernels are a prerequisite for open 4-bit training and inference stacks that do not rely on vendor libraries.
What counts as solved
A public kernel meeting the 90% bar on all shapes on B200, reproduced independently.
How it gets verified
GPU benchmarkA kernel run and timed on stated hardware against a baseline.
Hardware: one NVIDIA B200 (HBM3e) with the CUDA 12.8 or later toolkit supporting its block-scaled instructions. Dtypes: NVFP4 (E2M1 elements, 16-element blocks, E4M3 block scales, FP32 tensor scale) and MXFP4 (E2M1, 32-element blocks, E8M0 scales), BF16 output. Shapes: M in {1, 16, 128, 1024, 4096, 16384}, N and K from Llama-3 70B and DeepSeek-V3 projection sizes. Candidate: Triton kernel at a pinned upstream Triton release. Baseline: CUTLASS block-scaled GEMM (cutlass_profiler, best kernel per shape) at a pinned CUTLASS release. Metric: median TFLOP/s over at least 100 iterations after warmup; pass if at least 90% of the CUTLASS baseline on every shape. Correctness: max relative error versus FP32 GEMM of the dequantised inputs no more than 1.1x the baseline's. Harness: public script at a pinned commit.
Claims under verification
+ add a claimNo claimed results on the Verification board for this problem yet.
Where the work is
Kernel and compiler engineering on new hardware.
Bottlenecks
ranked by votes- #1EngineeringScale-factor memory layout and tensor-memory MMA support in compilers
Block-scaled instructions need specific scale layouts and tensor-memory staging that high-level languages are only starting to expose.
Community grades
Sign in to add your own grade.
Connected problems
- This problem is needed by An open 4-bit pretraining recipe matching BF16 loss at 1B parametersFast training depends on an efficient open block-scaled GEMM kernel.