Recurrent Looped Transformer gives each token 96 logical blocks of depth

The Recurrent Looped Transformer (RLT) pairs a causal encoder with a recurrent decoder that carries its final hidden state and sliding-window attention cache across every prompt and response token. This design enables latent reasoning with unbounded temporal depth, where the path through decoder blocks grows with sequence length while per-token block count stays fixed. It unifies pretraining, SFT, sampling, and RL replay under one complete-state transition, though realized reasoning gains and hardware efficiency remain to be established.

Infinite depth refers to an extensible temporal path, not infinite work within a token.
  1. vatsachak

    Paper written by AI

  2. dankai

    Is this just a theory about an architecture or are there actually some benchmarks/results to substantiate it?

  3. jal278

    not sure w/o context why this is important -- no results/implementation & i believe there are prior combinations of transformers/RNNs. but perhaps I'm missing the relevance/insight

More from this day

2026-09-13