AMD acquires Taalas to etch AI models into silicon, boosting inference 48x
AMD acquires Taalas to boost inference performance by etching models in silicon
AMD has acquired AI chip startup Taalas, which bakes model weights directly into silicon to dramatically speed up inference. Taalas's first chip, the HC1, served Meta's Llama 3.1 8B at 16,960 tokens per second—48x faster than Nvidia's GPUs. The technology, called model-specific integrated circuits (MSICs), requires re-spinning chips for new models, but only two metal layers need changing. AMD plans to pair Taalas chips with its Instinct-based Helios racks, potentially enabling disaggregated architectures where GPUs handle prompt processing and Taalas accelerators handle token generation.
Once the chips are deployed you’re stuck with that model. Any change bigger than something like a LoRA adapter is going to require a re-spin of the chips, which is not only expensive but time-consuming.
- LarsDu88
I'm surprised neither OpenAI nor Anthropic made this move first. The Chinese open weight models are pulling ahead and commoditizing their value proposition.
Baking models onto silicon would've been the next logical move to get a moat.
Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.
- linzhangrun
Thinking that five or six years from now, Fable-level intelligence could be provided at 100x the current speed... makes me feel lost. I cannot imagine what the future will look like.
- whythismatters
The demo: https://chatjimmy.ai/
- mNovak
What I like about this, is that it significantly increases the probability of a sci-fi scenario where you're picking up a hot chip on the black market; rumor has it, Mythos 9 weights baked in...
- yassa9
Can anyone imagine if a video generation model with the speed of ASICs baked into silicon ? real Sci-fi
- msteffen
This is neat but IMO a little crazy.
Something I personally haven’t seen much of, in all the discussions of model benchmarks and AI breakthroughs, is a distinction between “peak performance” and “reliable performance”. The “peak performance” of frontier models is very high: they’re solving open math problems, analyzing large codebases, etc. But my subjective impression is that “reliable performance” is mid at best: out of 100 random questions I might think to ask, it’s likely to say something wrong or stupid a handful of times at least.
I think there’s inherent tension between the two: the more a model reaches or outright hallucinates, the more likely it is to come up with tricky, subtle solutions to problems (I think people are somewhat like this too: Terry Tao’s brother is nonverbal, Jim Watson’s son has severe schizophrenia, etc). But then the less likely it is to generate a sensible email reply.
I use models all the time for coding, but I would not let one take over my daily correspondence. If the idea here is to run frontier models at high speed in data centers, that could be useful (the speed would be cool), but I’d be surprised if the cost of that hardware churn is worth it to frontier labs. But if the idea is to turn this into a chip that goes in your phone as some kind of routine, low-power inference thing…taking something too kooky to be relied on and baking it into your phone’s hardware like that doesn’t make sense to me.
- yumraj
Given the fast churn of the models, how does it work out?
Won’t the silicon etched model already be 1 or more versions behind by the time the silicon comes out.
Though if it’s cheap enough, there certainly can be a market for cheaper model inferences.
- A_D_E_P_T
This is probably a win-win. The team gets paid, and we get greater assurance that their best ideas and architectures -- which are truly impressive -- are going to see the light of day in actual products.
- kristianp
I've been eagerly awaiting their 2nd gen HC2, which uses multiple chips to host a "mid sized reasoning" [1] model. Its due in summer according to the article, I wonder if it will ever be released in that form now.
[1] https://www.forbes.com/sites/karlfreund/2026/02/19/taalas-la...
- ratsbane
Smart move by AMD. Chatjimmy is very fast and not very good, but I think it might become very fast AND very good.