Redis creator launches ds4 to run frontier LLMs locally
From the creator of Redis; run LLM locally with ds4

Salvatore Sanfilippo, creator of Redis, has released DwarfStar 4 (ds4), a C inference engine for high-memory Mac, CUDA, and ROCm machines. It runs DeepSeek V4/V4.1, GLM 5.x, and Qwen3.8 Flash Next with asymmetric 2-bit quantization, KV cache disk streaming, and a CLI, HTTP server, and native agent. On an M5 Max with 128 GB, it achieves up to 790 tokens/s prefill and 39 tokens/s generation.
Asymmetric quantization targets the routed experts while preserving critical paths. The model becomes practical on high-memory machines.
- neomantra
I maintain a fork of ds4 as shared libraries and thus can be used with other languages via FFI, along with public builds/binaries [1]. I made ds4go [2] against ds4 using techniques inspired by yzma.
In addition to the library bindings, we have a small library of tools (workspace for view/edit, scratchpad for persistence) and making your own is registering a Go function. And in recent weeks, I added the Vision and Qwen support, as ds4 added them.
Even if you don't use the Go library, the ds4go binary makes it really easy to download the libraries off of HuggingFace with a TUI available vie Homebrew.
Here's some TUI toy screenshots, sorry I still haven't released that code; it's of different quality than the others. [3]
EDIT: add ds4go TUI screenshot gist [4]
[1] https://github.com/NimbleMarkets/ds4/releases/tag/v0.8.20260...
[2] https://github.com/nimblemarkets/ds4go#install
[3] https://gist.github.com/neomantra/ae47422c8daf7a458212c93992...
[4] https://gist.github.com/neomantra/40180ade13df93290250ce8c6d...
- twoodfin
https://github.com/antirez/ds4
The project GitHub page is a much better introduction for the hn crowd.
- ttoinou
Ive been using this since it was initially released with deepseek v4 flash, and it is absolutely the best launcher ever on my m5 max 128gb
Now Ive been running qwen 3.8 flash next for more than a week and it’s doing great, really fast and super long context windows. Sometimes the model is behaving stupidly by not remembering something I said earlier but it could be also a problem from the agentic AI harness. Im using oh my pi but Im wondering what people are using ds4 with here ?
- simoiacos
Nothing comparable but inspired from DwarfStar I wrote a little inference engine for Intel Xe-LP (no XMX) 32GB laptops. The only model supported right now is a quantized Gemma-4, but I don't exclude in the future to support other MoE of similar size. Too bad we have no Qwen 3.8 35B-A3B yet.
I'm also looking into expanding the protocol and the engine to support various steering techniques.
- gchamonlive
small native inference engine optimized first for DeepSeek V4 Flash (including the experimental vision model), DeepSeek V4.1 Flash (Metal, and text inference on CUDA), and additionally GLM 5.2 and 5.3, GLM 5.3 Flash and DeepSeek V4 PRO, and Qwen3.8 Flash Next (Metal and CUDA)
This is local targeting high end consumer hardware like DGX Spark or AMD Ryzen AI Halo.
For our mere mortals that were kids not long ago and can't really believe we've got our hands on a x090 series targeting Qwen3.8 27b, https://github.com/noonghunna/club-3090 is the way to go.
I'm maintaining a web frontend for this, trying to at least. You can follow it here: https://github.com/gchamon/club-3090-server