Do reasoning cycles in hidden states reveal aha moments?

The Topology of Reasoning paper introduces an internal mechanistic lens for reasoning model performance that is distinct from the external graph taxonomy (CoT/ToT/GoT as formal graph types). By extracting reasoning graphs from hidden-state representations at each step — clustering hidden states to identify repeated states as cycles — it quantifies three graph-theoretic properties and shows they predict accuracy.

The three properties:

• Cyclicity: Frequency of recurrent cycles in the reasoning path. Distilled reasoning models show ~5 cycles per sample vs near-zero in base models. Cycle detection peaks at the 14B scale; larger models (32B) show the effect at later layers.
• Diameter: Breadth of exploration — larger diameter = model visits more distinct reasoning states before converging. Maximized in the 32B variant, correlating with accuracy on hard tasks (AIME > MATH500 > GSM8K).
• Small-world index: Simultaneously high clustering (local efficiency) and short path lengths (global connectivity). Distilled models show ~6x higher small-world index than base models.

The aha moment connection: RL-trained models are reported to exhibit "aha moments" — reconsidering intermediate answers during reasoning. From the hidden-state topology perspective, aha moments correspond exactly to cyclic structures in the reasoning graph. The paper quantifies a phenomenon previously identified at the generated-token level as a property of internal representation dynamics.

Overthinking and underthinking reinterpreted: Overthinking corresponds to redundant cyclic structures (excessive cycling). Underthinking — observed in o1-family models — corresponds to overly large exploration diameter without adequate cycling back to check.

Design implication: Supervised fine-tuning on an improved dataset systematically expands reasoning graph diameters in tandem with performance gains, providing concrete guidelines for dataset construction aimed at boosting reasoning.

This adds a mechanistic dimension to Can reasoning topologies be formally classified as graph types?, which covers external topology. Together they provide a two-layer analysis: what reasoning structure looks like from outside (CoT = chain, ToT = tree, GoT = graph) and what reasoning dynamics look like from inside (cycles, diameter, small-world).