MicroGPT in Pure C Hits 10M Tokens/sec on Apple M5
Microgpt in pure C hits 10M tps on Apple m5
A new GitHub project, microgpt-c, implements a character-level GPT entirely in dependency-free C, with training and inference in a single file. It achieves over 10 million tokens per second on an Apple M5 Pro using NEON optimizations, and 6.9M on an AMD Ryzen 5 5600H with AVX2. The model, with only 4,192 parameters, generalizes well, beating an interpolated trigram on unseen data.
c fp32+NEON 10168430 tok/sec
- pkilgore
[delayed]
- ilaksh
This is not an LLM obviously
, it's just for generating random names. But interesting to think of the possibilities of truly tiny language models if there were connected together.
- throwa356262
And the 5 years old AMD Ryzen 5 5600H is doing 7M?
Am I reading this right? Then I need to try this on Strix Halo