New benchmark shows frontier LLMs still struggle with fast multi-GPU CUDA code

Together AI has released ParallelKernelBench to measure a hard edge case for coding models: writing fast multi-GPU CUDA kernels. The benchmark spans 87 real workloads and evaluates whether generated code is not just correct, but performant.

The early result is mixed. Frontier models can sometimes produce kernels that outperform public implementations, but the best performer still solves under a third of the benchmark.

That gap matters for AI infrastructure because kernel quality directly affects training and inference efficiency. It also shows why code-generation gains do not automatically translate into expert-level systems programming.