Triton, CUTLASS, TVM and the languages and compilers that generate fast kernels without hand-written assembly.
Searching tile sizes, pipeline depths and schedules by measurement or learned cost models, and now asking language models to write kernels, scored against hand-written baselines.
A Python-embedded language in which a kernel is written for a block of data, with the compiler choosing memory layout, coalescing, shared-memory use and synchronisation; the default kernel backend of PyTorch's compiler.