NVIDIA's Nemotron 3.5 Lightning and NeMo Switchyard Cut AI Agent Costs by Two-Thirds
Nvidia Nemotron 3.5 Lightning and NeMo Switchyard
NVIDIA unveiled Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model designed for high-volume agentic AI tasks, delivering up to 4x faster output and 30% faster task completion. Alongside it, NeMo Switchyard, an open-source model routing library, intelligently directs each request to the most suitable model, cutting task completion costs to nearly one-third of using a single frontier model. Both are available for customization and deployment across PCs, workstations, data centers, and the cloud.
In a system of models, enterprises can create powerful AI agents with improved tokenomics.
- kentonv
Coincidentally I've been playing with small (~30B) self-hostable models for coding tasks today -- specifically plugging them into Cloudflare OS (which I work on) and asking each to build a collaborative whiteboard.
I'm finding that the Mixture-of-Experts (MoE) models (Qwen 3.6-35B, and Nemotron 3.5 Lightning) are, well, terrible at this. They just couldn't get the job done at all. Went way off the rails. They are really fast though!
Whereas ~30B dense models (not MoE) are pretty decent. I tried Muse Glimmer, Gemma 4-31B, Qwen 3.6-27B, and Laguna XS[0]. They were all able to build a working collaborative whiteboard app, without any guidance (other than feeding back error logs to the model). I also asked each to then draw a monkey by calling the API of the whiteboard it has just built. Laguna drew random scribbles but the rest all managed to produce something monkey-like.
(Frontier models in comparison will write the app in one shot with no errors at all.)
Note that both Qwen 3.6 and Gemma 4 each have both MoE and dense variants. I find this very confusing, because e.g. ollama's model index typically only distinguishes variants by their size, but MoE vs. dense makes a huge difference in how they actually perform. IMO they should use a suffix, like Qwen 3.6-moe vs. Qwen 3.6-dense, or maybe Qwen 3.6-fast vs. Qwen 3.6-smart...
[0] EDIT: Turns out Laguna XS is MoE, I misunderstood. It performed similarly to the dense models. But maybe this explains why it couldn't write code and th […]
- jmward01
One major consequence of the ramapocalypse, I think, is an even higher focus on small efficient models. I personally believe that the multi-trillion parameter models are fundamentally missing things and the push to smaller, more efficient will drive evolutionary structural changes that will lead to future gains
- thehamkercat
> NeMo Switchyard, an open source library for smart routing
> When deployed, NeMo Switchyard can intelligently direct each request to the most capable and suitable model for the job
How do routers like this handle prompt caching when you send the second request?
Sticky models per session? but then the second message of that session won't be sent to a suitable model, and will only be sent to the same model as previous one.
- docheinestages
They conveniently decided not to include the Qwen range of models in the Artificial Analysis graph, except the out-of-league Max variant. At least be brave and honest.
- average_bloke
I would like to propose something:
- problem: massive deluge of information because of AI
- solution: human beings should adopt a minimalist style of communicating in writing.
- e.g. this entire website page can be ten bullet points.
- WalterGR
24 comments so far about Nemotron on this earlier submission: https://news.ycombinator.com/item?id=49257947
- mark_l_watson
I love the wave of new small model releases. Pleasantly surprising that an NVIDIA model runs so well on Apple Silicon using MLX! I was using nemotron-3.5-lightning:30b-mlx with OpenCode on my old (cheap) Mac this morning and no bad experiences except for running slowly.
- jadbox
Nemotron 3.5 Lightning runs on how little GPU vram? Can q4 run on 16gb?