A 2013 iPad Air runs a 16B model at 4.5 tokens per second

I'm making LLMs work on small and old devices

A developer pushes large language models onto hardware nobody expects to run them: a 1.9 GB Xiaomi TV box and a jailbroken 2013 iPad Air with 1 GB of RAM. Through custom kernels, 1-bit lookup tables, expert paging from flash, and weight streaming, the TV box hits 7.2 tok/s on an 8B mixture-of-experts model and 5.1 tok/s on a 16B one, while the iPad runs a 1.7B model at 24–26 tok/s. Every claim is measured on the device itself.