An AI agent vibe-coded an FPGA into running an LLM at 213 tokens per second
Vibecoding an Inference Chip

A developer handed Codex a cloud FPGA and a ternary-weight model, then mostly cheered it on for several days. The agent wrote roughly 3 million lines of C++ and Verilog, ran hundreds of experiments, and pushed inference from a bit-exact baseline to 213 tokens per second — a 33× speedup — for about $10,000 in Claude and OpenAI tokens. A detailed prompt of hardware lessons from another team barely moved the needle.
It was one of the most mind blowing autoresearch loops I've run, ultimately writing roughly 3M lines across the experiment history and parsing 30M lines of output files, burning only $10,000 in Claude and OpenAI tokens to run hundreds of experiments.
- itissid
Someone mentioned that FPGAs though amazing to create bespoke chips that are fast, run into a fab capacity issue. Someone explained briefly to me that even if you could make an fpga that could do inference better than current GPUs it was economically possible to make them at scale. I can't recall whether that was for technology related issues or just because fab capacity is completely filled up.
- reuben364
Over the week or so I have vibe coded a Lean-like language with
- quantitative types
- a Rust-ish DSL that lowers into the base language
- a C, LLVM, WebAssembly backend and a Verilog backend for a subset of the language
- A RISC-V implementation written in the Verilog subset that runs Doom on my Arty FPGA.
- A category theory and real number library
- A graph storage engine with queries written using profunctor optics
- A datalog with an egglog in progress
- A Joern-like code analysis engine for C and C# using tree-sitter to parse
- A React style framework
- Editor plugins for my language
All at varying stages of completeness.
What you can do now is wild. My fear is a sort of Gell-Mann Amnesia effect where my lack of expertise in the area is causing me to miss major problems with it. But at least some part of it has to be real. I plan to announce it in a more official capacity once I've allayed those concerns (https://wiz.badlydrawnwizards.com for a sneak peak, but sorely need to update the website)
My bottleneck currently is CI capacity. I've turned every spare computer in my household into a Slurm node so I can run more testing locally to avoid wasting precious CI capacity.
- seanjhkwon
It shows the importance of automated indicators when giving AI huge tasks. Maybe more important than the intelligence of the model.
- jluxenberg
A fun and interesting project, but to spend $10k+ in tokens on a curiosity seems excessive! Outside of my budget for sure.
- cpldcpu
I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c
It seems implementing transformers is the new "hello world" of AI assisted coding.