Can reasoning happen at the sentence level instead of tokens?
Does moving from token-level to sentence-level reasoning in embedding space preserve the capability for complex reasoning while enabling language-agnostic processing? This challenges assumptions about how LLMs must operate.
Current LLMs operate at the token level — every reasoning step is a next-token prediction. Meta's Large Concept Model (LCM) challenges this by operating at the sentence level, reasoning in an abstract embedding space (SONAR) where each "concept" corresponds to a sentence.
The architectural difference is fundamental. The LCM:
- Does not see tokens — it receives and produces sentence-level embeddings
- Is language-agnostic — the same reasoning process works for any language or modality because SONAR encodes meaning, not surface form
- Separates reasoning from instantiation — reasoning happens once in the abstract space; decoding to a specific language happens afterward and can target any language without re-running the reasoning
The hierarchical structure adds a planning layer. The LCM predicts a sequence of concepts auto-regressively until it produces a "break concept" — analogous to a paragraph break. At that point, a Large Planning Model (LPM) generates a plan to condition the LCM for the next sequence. This two-level architecture (sentence-level prediction + paragraph-level planning) is designed to produce more coherent long-form output than flat token-level generation.
The comparison to JEPA (LeCun, 2022) is instructive: both predict representations in embedding space rather than raw observations. But where JEPA emphasizes learning the representation space via self-supervision, LCM focuses on accurate prediction within an existing embedding space (SONAR). The embedding quality is assumed, not learned end-to-end.
This connects to the latent reasoning thread through a different mechanism. Can models reason without generating visible thinking tokens? achieves reasoning without tokens via recurrent depth in continuous space. LCM achieves it via sentence-level embeddings. Both challenge the assumption that verbalized token-by-token generation is necessary for reasoning, but from different angles: depth-recurrent models reason within a single token's representation; LCM reasons between sentence-level units.
The practical implication: if reasoning can happen at the concept level rather than the token level, then the verbalized chain-of-thought paradigm is not the only path to sophisticated reasoning. The question is whether sentence-level granularity captures enough structure for complex reasoning tasks, or whether some tasks require finer-grained (sub-sentence) reasoning steps.
Inquiring lines that read this note 56
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What causes reasoning models to fail or wander off track?- Does sentence-level granularity capture enough structure for complex reasoning tasks?
- Why does removing semantic content collapse reasoning in language models?
- How do embedding dimension limits constrain what concept models can represent?
- Why do unit-sphere spaces fail at distinguishing word order and negation?
- What is the relationship between reasoning depth and verbalization requirements?
- Does the DeepSeek R1 single token insertion represent genuine reasoning?
- Can instance-adaptive reasoning happen without sequential token dependencies?
- How do soft thought tokens differ from decoded assistant outputs?
- Can latent reasoning in continuous space scale beyond supervised reasoning tasks?
- How do hidden embeddings preserve more information than discrete tokens?
- Do latent sequence vectors outperform per-token latent iterative computation for reasoning?
- Can models hide their reasoning in continuous space rather than natural language?
- Can latent space represent reasoning dimensions that text cannot?
- How do recursive language models rethink where to store reasoning?
- Can latent reasoning achieve the same substitution without tokens?
- Can reasoning happen in latent space without chain of thought?
- Can articulating latent reasoning processes improve transfer across domains?
- Why does latent-level prediction beat token-level prediction for reasoning?
- Can latent reasoning scale test-time compute without verbal tokens?
- What affordances do normalizing flows add over opaque vector reasoning?
- Does latent manipulation outperform token-level prediction for efficiency?
- Can latent reasoning scale test-time compute without verbalized tokens or special training?
- Can latent reasoning stay readable without explicit token-by-token decoding?
- How should meaning spaces be systematically modeled across different applications?
- Are static embeddings analogous to the formal linguistic competence layer?
- How do static embeddings and contextualized representations divide semantic labor?
- Do discrete tokenized modalities preserve information better than continuous embeddings?
- Why does semantic decoupling specifically break LLM reasoning abilities?
- What internal mechanisms explain LLM reasoning and representation limits?
- How does in-context semantic reasoning differ from symbolic reasoning in concept fusion?
- Can language models perform genuine symbolic reasoning without semantic grounding?
- Does structured decomposition improve LLM reasoning in other compound tasks?
- Do reflection tokens and symbolic tokens serve different roles in reasoning?
- Why does hierarchical formal language training improve token efficiency more than natural language?
- Can capability boundary collapse be addressed by operating at representational rather than token level?
- How does tokenization change what gets counted as valuable knowledge?
- What semantic information is lost if analysis skips the token embedding layer?
- How do reasoning-invariant tokens dilute learning signals in uniform averaging?
- What evidence shows that reasoning chains encode token-level functional structure?
- Why does token ordering in LLMs create sequences rather than true temporal flow?
- How does hierarchical routing in the concept module influence token-level generation?
- Does reasoning happen in hidden space or in generated tokens?
- What semantic information is necessary to preserve for sound LLM reasoning?
- What computational structures can actually scale serial reasoning depth?
- What role does embedding space geometry play in multi-hop reasoning?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can models reason without generating visible thinking tokens?
Explores whether intermediate reasoning must be verbalized as text tokens, or if models can think in hidden continuous space. Challenges a foundational assumption about how language models scale their reasoning capabilities.
alternative latent reasoning via recurrent depth; LCM is a third approach (sentence-level, not token-level or depth-recurrent)
-
Can models reason without generating visible thinking steps?
Do machine reasoning systems actually require verbalized chains of thought, or can they solve complex problems through hidden computation? This challenges how we measure and understand reasoning.
the shared question: does reasoning require verbalization? LCM says no, operating at sentence granularity
-
Do embedding dimensions fundamentally limit retrievable document combinations?
Can single-vector embeddings represent any top-k document subset a user might need? Research using communication complexity theory suggests there are hard geometric limits independent of training data or model architecture.
LCM relies entirely on embedding quality (SONAR); the mathematical limits of embeddings constrain what LCM can represent
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Large Concept Models: Language Modeling in a Sentence Representation Space
- Training Large Language Models to Reason in a Continuous Latent Space
- Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space
- Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs
- DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning
- What Makes Effective Supervision in Latent Chain-of-Thought? An Information-Theoretic Analysis
- Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
Original note title
Large Concept Models enable sentence-level reasoning in a language-agnostic embedding space — hierarchical abstraction beyond token-level processing