How do looped transformer layers actually behave during inference?

Looping an LLM's layers in the latent dimension improves reasoning, but it has been unclear how the internal dynamics differ from a standard feedforward model. This mechanistic analysis answers through the lens of stages of inference — the idea that LLM computation decomposes into distinct computational stages.

The core result is geometric. For many looped models, each layer in the cycle converges to a distinct fixed point, so the recurrent block follows a consistent cyclic trajectory in latent space. As those fixed points are reached, attention-head behavior stabilizes, producing constant behavior across recurrences. And empirically the recurrent blocks learn stages of inference that closely mirror feedforward models — repeating those stages in depth with each iteration. This appears to be emergent: it shows up even when training does not explicitly encourage it. The repeated application of a shared block necessarily implies one of two regimes — either the block's contribution vanishes asymptotically, or it traces a constant cyclic trajectory.

The implication that matters: recurrent depth is learned re-application of computation, not the discovery of genuinely new computation per loop. The loop re-runs the same inferential stages rather than adding qualitatively different ones. This is the mechanistic complement to Does looping layers beat adding depth in diffusion models?: it explains why reused computation can match or beat added depth — the network was re-enacting stages anyway, and looping makes that reuse explicit and parameter-free. Recurrent block size, input injection, and normalization govern whether these cyclic fixed points emerge and stay stable.