Can looping enable reasoning capabilities that fixed-depth transformers fundamentally cannot achieve?

This explores whether looping a transformer's computation — feeding its layers back through themselves — buys reasoning power that a fixed-depth stack can never reach, no matter how it's trained. The corpus says yes, but with an important twist: the gain is about *depth of computation*, not a different kind of thinking. The cleanest theoretical case comes from complexity theory. Fixed-depth transformers sit inside a shallow circuit class (AC0/TC0), which provably can't express certain step-by-step procedures. Recurrence breaks that ceiling: a hierarchical model that couples slow planning with fast computation across two timescales solves Sudoku and mazes — where chain-of-thought collapses entirely — with only 27M parameters Can recurrent hierarchies achieve reasoning that transformers cannot?. Looped, parameter-shared transformers similarly achieve compositional generalization and 'depth extrapolation' (running more loops at test time than during training) that vanilla transformers can't Can looped transformers generalize to unseen knowledge combinations?.

The twist is what looping actually *does* mechanically. When you inspect a looped model layer by layer, each recurrent cycle converges to a fixed point and the loop turns out to re-enact the same feedforward inference stages a deep model would run — repeating known operations across more steps rather than inventing genuinely new ones How do looped transformer layers actually behave during inference?. So looping isn't a smarter algorithm; it's more serial compute applied to the same machinery. That reframes 'fundamentally cannot' as a budget problem: the fixed-depth model runs out of sequential steps, and the loop hands it more.

That reframing connects to a quieter debate in the corpus about *why* fixed-depth models fail. One line of work argues their apparent reasoning is shallow to begin with — transformers often succeed by memorizing and pattern-matching computation subgraphs from training, then break on novel compositions with errors compounding step by step Do transformers actually learn systematic compositional reasoning?. Another argues the failures we blame on 'reasoning limits' are really *execution* limits: models that know an algorithm still can't carry it out over many steps in text alone, and giving them tools dissolves the supposed cliff Are reasoning model collapses really failures of reasoning?. Both point at the same culprit looping addresses — not enough serial depth to execute long procedures reliably.

What's striking is that looping isn't the only way to manufacture that extra depth. Chain-of-thought spends it across generated tokens instead of internal cycles — and standard transformers can even bootstrap their own depth by iteratively retraining on their correct outputs, jumping from 10-digit to 100-digit addition through self-improvement rather than architecture changes Can transformers improve exponentially by learning from their own correct solutions?. And at the theoretical extreme, a single finite transformer is Turing-complete given the right prompt, meaning the expressive ceiling is less about the architecture than about whether training ever finds the program prompting-is-turing-complete-a-single-finite-transformer-can-compute-any-co. So the honest answer: looping genuinely lifts the *provable* depth ceiling that constrains fixed-depth transformers — but it's one of several routes to the same scarce resource, serial computation, not a categorically new form of reasoning.

The thing you didn't know you wanted to know: the deepest objection to fixed-depth transformers may not be depth at all but how they hold knowledge — as flowing activations rather than stored, addressable facts, more like oral performance than a library Do transformer models store knowledge or generate it continuously?. More loops give you more computation over that flow, but they don't change its nature.