CaSA - Ternary LLM inference inside COTS DRAM via charge-sharing
Show HN: Running PrismML's Bonsai inside DRAM by breaking DDR4 timing rules
CaSA is a groundbreaking architecture that enables ternary LLM inference directly within standard COTS DRAM, bypassing the memory bus entirely. While software quantization like PrismML's Bonsai reduces model size, CaSA solves the critical hardware bottleneck of memory bandwidth and thermal throttling on edge devices. By executing AI natively inside memory through charge-sharing, it eliminates the energy drain of transferring gigabytes of data across the SoC bus. This innovation paves the way for running massive 27B+ parameter models on smartphones with true on-device privacy and efficiency.
Why are we still transferring data to the compute? Why not execute AI inference natively within the memory?
- SwellJoe
Jebus, that is some sloppy prose. Can people not even be bothered to write the summary themselves, anymore? AI doesn't want anything, so they can never have a point of view, so their prose rambles incoherently across all the various prompts they've seen in a project. This project sounds like the ramblings of a crazy person. Even though the fact that DRAM can do any computation is interesting, nobody should have to read this mess.
"why are we still transferring data to the compute? Why not execute AI inference natively within the memory?"
You already answered that question: 47.5 seconds per token from a tiny 2B 1-bit model model.
- butvacuum
Very interesting. I don't see it mentioned so I'll ask:
Would being able to alter voltage levels on the fly (eg, cells x y and z get 1.25 while abc get 1.2) expand the ability here?
- Retr0id
Very interesting. Do you have any descriptions or writeups written by a human?