AI agents now optimize GPU kernels better with a simple prompt than with expert guidance
Make this forward pass as fast as possible
ML compilers are being rethought as AI agents take over kernel optimization. TVM argues a minimal, stable IR foundation still matters for verification and shared abstractions, while Together's TKCC finds that unrestricted source code access unlocks higher performance ceilings than traditional IRs. OpenAI's Codex is stripping heuristics from Triton to make the search landscape smoother. Strikingly, Together found that seeding agents with expert strategies lost to simply prompting 'make this forward pass as fast as possible.'
Seeding with expert-written strategies lost to literally prompting 'make this forward pass as fast as possible,' which is a great ablation.
- 0-_-0
If AI models were trained for optimising kernels, they could easily replace ML compilers. With no branches a numerical check gives you the correctness guarantees you need. However we already see anthropic deliberately nerfing kernel optimisation capabilities of their latest models, Chinese AI companies on the other hand have the opposite incentive.
- etskinner
As someone who doesn't know much about kernels and compilation, this reads like a modern-day Retroencabulator