HybridAgent reasons in latent space to cut tokens by up to 55% and run 2.6× faster
Hybrid Reasoning Agents: 35–55% fewer tokens and up to 2.6× faster inference
HybridAgent lets LLM agents switch between latent reasoning and text-based tool use within a single generation, using a lightweight HybridLink and continuous working memory to keep hybrid context intact across turns. Post-training with HybridSFT and HybridRL teaches the model when to think silently and when to act through text. Across 11 benchmarks at three model scales, it improves performance by 10.4% while cutting tokens 35.5–54.7%, speeding inference 1.8–2.6×, and reducing cost 2.7–3.7×.
How can we build an agent that thinks beyond human language, yet still acts through it in the external environment?
- sinuhe69
[delayed]
- bhattisatish
The code is available at: https://github.com/togethercomputer/HybridAgent
LLM Models and data: https://huggingface.co/collections/togethercomputer/hybridag...
Evaluations: https://arxiv.org/html/2610.18861v1#S5
- amluto
I wonder how this compares to going all-in: widen the input embedding to fit the output (post-“Hybrid Link”) latent vector and a token embedding and pass both through even when in text mode. After all, you’re paying essentially the full cost of running the latent-mode model while generating text, so why not let it latently reason at the same time?
(Or just add the vectors instead of using wider vectors - everyone seems to have good results from munging vectors together with plain addition.)
- notnullorvoid
Excited to see this applied to Qwen 3.8 27B. It's reasoning output tends to include lots of restatement, which I guess would be more likely to be kept in latent space with this hybrid reasoning.
- SwellJoe
I'm running the 14B in llama.cpp (a version from yesterday), the 8-bit quantization from bartowski, and it looped on variations of the same paragraph for five minutes before running out its turn budget, in response to a pretty simple chat request that other similar-sized models (e.g. Gemma 4 12B) have no trouble with, a web search and data summarization task, sort of the sweet spot for models this size. It's possible llama.cpp needs some update to work with this model, but it's based on Qwen 3, so I would expect it to be pretty well-supported. The chat template is the same one upstream ships, which may or may not be correct. I'm currently downloading the 16-bit version, to see if it's quantization pathology.
I'm also not confident it knows what a CSV is, either, as it put the data into a Markdown table during its reasoning phase as an example of how to present the data (my prompt asks for data in a CSV). My experience with most Qwen fine-tunes is they get dumber, but their benchmark results suggest otherwise, so I have to assume it's a llama.cpp incompatibility.
Edit: Same looping in BF16. So, probably have to wait on a llama.cpp update.
- incognito124
How does this differ from Astra's looped transformer? That architecture also reasons in the latent space
- iforgotmypasswo
This is so far beyond me, so I’m going to ask and hope someone knowledgeable is here:
Could this idea of reasoning through fundamentally different (it sounds like?) models be used to link even more distant models or systems?
If you can make a small leap, then you can construct bigger leaps through a chain. What else can latent space link to?
Instead of tool use through text, tool use through analog signals?
Instead of language prompts, analog sensors?
- ajey_ds
I always thought, sometimes language would be limiting factor for llms, vector embedding based internal reasoning would be key to faster better models.