DeepX's new chip runs 100-billion-parameter AI models on 5 watts
DeepX claims inference up to 100B parameters at less than 5 W
South Korean startup DeepX has taped out the DX-M2, a second-generation edge AI accelerator built with Samsung's LPDDR5x processor-in-memory technology. It targets 20–100 billion parameter models at 10–30 tokens per second within a 5 W budget, with memory bandwidth tripled to 150 GB/s and 80 TOPS on a 2-nm node. DeepX claims offloading GEMV operations into memory yields 6.7× the tokens per second on Llama3.1-8B, though at 2.7× the power. Samples are due March 2027.
If we deploy models up to 100 billion parameters in devices, we can reduce AI data center traffic by more than 80%. That means we could have five times larger infrastructure without adding any AI servers.
- augment_me
Quirks from the article:
1) The performance numbers are simulated
2) The details say that the performance numbers require 2.7x the overall power, so around 13.5W. Power needed for all the memory is not explicit in the numbers.
3) The compiler is also simulated and it's a prototype, so it's not a drop-in replacement for anything like Nvidia ISAAC
4) 5W seems to refer to sub-4B models, so not the 100B param ones.
- LaurensBER
I'm getting questions weekly about running AI in-house now, in Europe trust in both the US and China seems to have hit an all-time low and AI is already scary for a lot of people.
I have no doubt that local first is the future that we're heading towards. We're seeing "algorithmic progress" towards more efficient models and _some_ hardware being available (NPUs, this chip, etc).
At the moment it doesn't seem like an appealing offer since it's just a little bit too constrained and since internet is universally available but if we can run a DeepSeek 4.1 Flash class model cheap and local, it becomes a no brainer.
- PaulKeeble
80 TOP/s and 150GB/s memory gives us the fundamentals of where this sits performance wise although its not said if the TOP/s is for INT4 or FP16 or what which changes the prefill performance considerably.
A consumer graphics card (say a 9070 XT) will be something more like 750 TOP/s on INT4 (20TOP/s on FP16) and 650 GB/s. So its probably about a tenth the performance for prefill but maybe its only a third if its FP16 and can't handle more than 15GB per token due to its memory bandwidth (at Q4 that would be ~7B active parameters) for decode to achieve 10 token/s and to get its 30 token/s it would be max 2.5B active parameters at Q4. A full FP16 model on this seems very unlikely unless its a lot bigger than we think it is.
This I think is just about enough for a decent home AI, its a little slow and its only going to be of use for MOE (so no qwen 27B) but depending on price I can see it being of interest to a lot of people as some flash models aiming for 100B and 2-7B active parameters could be really quite a lot of home capability in a relatively low power consumption unit.