Matrix-multiply and attention kernels tuned to run near the hardware's peak, measured in TFLOP/s.
Storing the transformer key-value cache in fixed-size non-contiguous blocks, with attention kernels that gather through a block table; the memory manager behind high-throughput LLM serving.
A tiled, fused attention kernel that never writes the full score matrix to GPU memory, using an online softmax; versions 1 to 3 track successive GPU generations.