Nvidia's Nemotron 3.5 Lightning: 30B Model Runs on a Single GPU
Nvidia Nemotron 3.5 Lightning
Nvidia unveiled Nemotron 3.5 Lightning, a 30B-parameter MoE hybrid model with 3B active parameters, optimized for single-GPU deployment on DGX Spark or H100. It supports up to 1M token context and features speculative decoding via DSpark, DFlash, or MTP. The model excels in long-running agent tasks, with strong SWE-bench Verified and GPQA Diamond scores, and is available under the OpenMDW-1.1 license.
The model has 3B active parameters and 30B parameters in total.
- NitpickLawyer
This links to the nvfp4 version of the model, so they only compare it to the bf16 in this model card. If you're looking for other similar model comparisons they are in the model card of the bf16 version here [1].
While it looks "behind" the qwen equivalent model on most benchmarks, a few personal notes:
- nemotron models feel to me a bit less benchmaxxed / "stubborn". That means that they generalise a bit better, or can be tasked to solve similar but not quite identical task types to the training data (something that's hard to do w/ qwen/ds models)
- nemotron series are also open training (w/ open training recipes and some training data public)
- nvda will have an incentive to continue this kind of releases, even if other parties slowly abandon the open release of models. Whatever other incentives 3rd party labs have (i.e. meta, goog w/ gemma, the chinese labs that IPOd, etc) nvda will always want to sell hardware so their incentive to keep pushing open models is evident and will likely continue "forever".
[1] - https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
- simonw
Runs fast on my Mac - ~100 tokens/second - but it's a bit of an over-thinker. It sketched out four different SVGs in the reasoning trace before returning a (bad) pelican riding a bicycle: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
I used LM Studio and this model file: https://lmstudio.ai/models/nvidia/nemotron-3.5-lightning
I got a pretty solid (and reasonably fast) result for this simply coding agent activity (with simonw/llm-coding-agent):
llm code -m lmstudio/nvidia/nemotron-3.5-lightning \
'how does auth work?'
https://gist.github.com/simonw/a8741f79280cedc86bcb8d90edfb4... - took just under a minute.
- bearjaws
Crazy to see how well Qwen3.6 35b-a3b is holding up, sure it is ~20% larger but it's scores are also ~20% higher with the same number of active params (excluding the IFBench).
Hopefully Qwen follows up their 3.8 launch with a new 35b-a3b
- gaodean
Developing on the Mamba 2 architecture is a really interesting point to note. It seems to be catching up to “regular” transformer architectures.
- kamranjon
You might look at this and and be a bit disappointed by the performance against qwen and gemma models - but this is an entirely open source training pipeline, this is quite impressive and I don't think another model this performant exists with fully open source data and recipes alongside the weights.