Inkling 2 Mini runs on a laptop and beats every open-weights model under 100B

Thinking Machines released Inkling 2 Mini, a 26.4B-parameter Mixture-of-Experts model (3.4B active) that loops its middle layers three times per token, letting it reason longer without writing everything down. It tops open-weights models under 100B on DeepSWE v1.1 (48.2%) and Vals Index v2.2 (46.5%), fits in ~20GB, and ran offline on a MacBook Pro to build a playable flight simulator mid-flight.

Looping allows the model to reason longer without writing its reasoning down, which matters for monitoring: our ability to oversee and understand what the model does by reading its chain of thought.
  1. montroser

    This is bonkers good if these benchmarks hold up in real-world scenarios. To be at the level of Qwen3.8-27B with a looped transformer model is quite the flex. They just left this note on reddit:

    Hope that folks find this model more amenable to local usage - the sizing was largely informed by what folks on this subreddit ask for the most haha!

    There are two particularly interesting aspects of this model for local users:

    We do our training end-to-end in nvfp4, and although we haven't exhaustively benchmarked this, folks like Unsloth have noticed that our models seem to respond very well to being quantized, maintaining 90% top-1 even at at 1-bit quantization.

    This model is looped, which allows more depth at the same parameter count, which we particularly notice seems helpful for coding tasks. :)

    This is our first model targeted at local users, so let us know how this model is and anything you want to see out of future iterations!

    https://www.reddit.com/r/LocalLLaMA/comments/1y1diuk/thinkin...

  2. lilytweed

    Whoa, not just a model that acknowledges and embraces using a looped architecture (GPT Astra 6+ and Sol 6.1+ are only [heavily] rumored to be looped), but it's open weights, too?

    I'm willing to wager that most open models will go looped in the wake of this.

    Wonder what it means for safety and interpretability at this level of availability.

  3. hedgehog

    Seems pretty promising in a few quick coding tests. 27 tokens/s decode on Ryzen 395 and llama.cpp Vulkan. Besides the obvious unmerged PRs (32694, 32657, 32429, 32704) you'll want below tiny patch for about 30% speed boost.

    Also, if anyone from the team is reading, if you have access for researchers I'd be interested in doing some higher volume tests & harness tuning than I have hardware for personally.

    --- a/src/llama-graph.cpp

    +++ b/src/llama-graph.cpp

    @@ -2670,10 +2670,12 @@ ggml_tensor \* llm_graph_context::build_moe_ffn(

    const float beta = hparams.situ_beta;

    const float lb = hparams.situ_linear_beta;

    - ggml_tensor \* act = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, cur, 1.0f/beta)), beta);

    + // gate/up are strided views of the fused gate_up output; ggml-vulkan's SCALE

    + // needs a contiguous src, so without the cont both scales fall back to the CPU

    + ggml_tensor \* act = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, ggml_cont(ctx0, cur), 1.0f/beta)), beta);

    act = ggml_mul(ctx0, act, ggml_sigmoid(ctx0, cur));

    if (lb > 0.0f) {

    - up = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, up, 1.0f/lb)), lb);

    + up = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, ggml_cont(ctx0, up), 1.0f/lb)), lb); […]

  4. shahbazac

    I don’t understand the business model of these labs which obviously spend a huge amount of money on compute then release their models. I definitely glad they do, but how does it make business sense?

  5. NiloCK

    > a looped transformer, running the middle layers of the model three times for every token.

    Completely unqualified take here, but unconditional re-runs of a fixed set of layers doesn't strike me as meaningfully 'loopy'. The exact performance profile is available in a more trad architecture by just inserting the layers after one another?

    My presumption on loopy architectures was that they looped longer on token selections that were more difficult / higher stakes, with automatic kick-outs after some n loops to avoid machine analysis paralysis.

    ALTHOUGH I guess that the presented architecture may also be useful insofar as it can fit in less memory than the same model with a tripled set of , and possibly that training over a looping architecture converges faster / truer than over naive architectures of the same size.

  6. nsingh2

    I've been thinking about how small looped models are probably going to be the next big thing in local LLMs, and here it is. Exchanging a lower memory capacity requirement for more compute instead.

    If the GPT-6.x models are looped, no doubt memory was a big reason.

  7. monksy

    The GGUF version of this that can run on Ollama/Llama.cpp is: https://huggingface.co/unsloth/Inkling-2-Mini-GGUF

  8. dofm

    Quite an interesting model, but the second of my test prompts in "extra high" thinking in Unsloth really underscore what they say about knowledge here:

    > While looping improves the model’s reasoning, it doesn’t enable its weights to store more facts—knowledge is mostly a function of total parameter count. Inkling 2 Mini’s score on AA-Omniscience, a measure of factuality, is in line with other models of the same size [...] but knowledge-intensive benchmarks doesn’t improve with more passes...

    > Inkling 2 Mini overcomes the knowledge gap by efficiently using search and tools.

    I tested the Unsloth Q4 version.

    My prompt asked it to write a singleton class, single file plugin for WordPress that hooks user login and stores last login time as metadata. It has some code style requirements. The prompt also asked it to make no tool calls so it didn't search and only used its knowledge.

    The model burned through 19200 thinking tokens on this task (which was the hard limit I had set) and didn't finish. Its thinking suggests it was going to make at least one mistake.

    Both Qwen 3.6 35B and Gemma 4 26B can cope with this task, from memory, in far fewer tokens (even taking into account Qwen's tendency to loop in thinking).

    Rerunning the same prompt with tool use allowed, it searched with browser use, pretty effectively, and got to a solution, but it still made the mistake the thinking traces suggested it would earlier (wrong hook name). Other than that it wrote good code.

    But 20K-ish tokens i […]

More from this day

2026-11-20