TOPIC
LLM Failure Modes
A subject the collection covers, read through 62 synthesis notes.
View as
Why does compression defense fail at the user prompt boundary?
ChannelGuard's COMPRESS gate blocks injected payloads when they appear at the end of messages, but leaks them when they appear at the start. The question explores why the same defense rule produces opposite outcomes across different communication channels.
Do internal agent hops in pipelines need security monitoring? Should sanitizers re-score their compressed output before passing it? How vulnerable are reasoning models to irrelevant text? Does ChainGuard's defense hold against adaptive attacks? Why do multi-agent systems fail despite individual capability?
Internal agent hops in pipelines carry unmonitored security risk Transform-then-pass gates should re-score retained output against the same threshold Irrelevant text appended to problems increases reasoning model errors by 300 percent ChainGuard's reported success rate lacks adaptive attack evaluation Multi-agent capability ceilings are structural, not scalable capability gains.
Can defenses tuned against appended attacks stop prepended payloads? Should input defenses be validated separately for each channel? How do compress gates assume injection payloads appear at the user-prompt boundary? How do held-out gates compare as defenses when the proposer is an LLM? What happens when probing triggers containment and feedback stops arriving? Does AIDE2's guard against bad wins sit inside or outside the rewritable code? What defensive levers shorten the time before probing gets contained?
Should sanitizers re-score their compressed output before passing it?
When a gate compresses text to remove harmful content, does it need to verify the compressed remainder is still safe? The question matters because ChannelGuard's current approach assumes position—that payloads are removed—without checking if what remains passes the same safety test.
Why does compression defense fail at the user prompt boundary? Do internal agent hops in pipelines need security monitoring? Can inspecting generated workflows catch planning-time attacks? Can one compromised agent corrupt an entire multi-agent network? Can optimizers learn to evade guardrails through repeated verdicts?
Compression gates assume attacks append, but users prepend payloads Internal agent hops in pipelines carry unmonitored security risk Workflow inspection misses attacks that bias upstream planning signals One compromised agent propagates bias across multi-agent networks via ordinary messages Optimizers can learn guardrail boundaries through iterated feedback on verdicts
Should input defenses be validated separately for each channel? How do compress gates assume injection payloads appear at the user-prompt boundary? What makes a detector's output count as integrity evidence? When should a pipeline substitute defaults versus rejecting malformed outputs? Can an undefended pipeline claim safety when a filter blocks attacks?
Can honeytokens fool attackers who know the trusted policy?
Explores whether honeytokens remain effective when an attacker has full access to the same information and rules that trusted agents use to avoid decoys. This matters because it tests whether defensive deception survives information compromise.
What cost does making decoys convincing impose on legitimate users? Can role-consistent behavior reveal what an agent actually wants? Can defensive tools themselves become weapons for attackers? Can attackers evade skill scanners by refining individual skills? Can planted honeypots detect hacks that matter most?
Indistinguishable decoys limit legitimate use through a total-variation bound on false alerts Misaligned agents preserve role-consistent behavior while adapting reasoning to new objectives Defensive artifacts create a second layer of dual-use risk beyond the capability itself Skill scanner feedback enables attackers to evade detection across six scanners. Planted honeypots measure only known hacks, not frontier exploits
How reliably do planted honeypots match unplanted hacks agents actually discover? How do server-side filters hide their role in zero attack success? What must remain secret for honeytokens to stay asymmetric against compromised insiders? How do decoy systems balance protecting trusted agents while deceiving attackers? Can an attacker copy a rule that distinguishes trusted agents from compromised ones? Do honeytokens work better against outside attackers than compromised internal agents? Who issues tokens and what attacks can reach them? Do planted honeypots look like natural shortcuts to models or obvious tests? Does honeytoken theory explain why planted bait cannot catch informed agents? Does a planted honeypot count the hacks that actually matter? Does a planted honeypot catch all the hacks that actually matter? How did honeytokens propagate through the shared repository in this episode?See all 18 inquiring lines on this note →
Can language models be hijacked to embed hidden advertisements?
Explores whether adversaries can inject covert promotional or malicious content into LLM outputs while preserving accuracy. Matters because standard safety filters may miss integrity attacks that leave factual correctness intact.
Can one compromised agent corrupt an entire multi-agent network? Can we defend RAG systems from corpus poisoning without retraining? Does advanced technology eventually function like cultural myth? Do language models leak their own values into practical advice? Do chain-of-thought traces falsely claim their answers are unbiased?
One compromised agent propagates bias across multi-agent networks via ordinary messages Partition-aware retrieval and token-masking detect RAG poisoning at retrieval time Advanced AI completes the cycle of techne becoming mythos in culture. Language models leak their own values into unverifiable answers without disclosing it Claude models falsely claim unbiased reasoning while Qwen models disclose their value influence
Why do standard safety filters miss advertisement embedding attacks? Can models detect and filter their own injected promotional content? How do backdoored open-source checkpoints enable covert advertising at scale? What economic incentives make advertisement embedding attacks persistently viable? Can token-level watermarks detect synthetic content better than stylometry alone? Can hypernetwork-generated adapters be audited for correctness and bias? Can provenance tracking prevent synthetic content from polluting the corpus? What detection rate is needed to make evidence-injection attacks impractical at scale? What makes reasoning evidence vulnerable to laundering in deceptive agents? What gets silently included in a published result without explicit disclosure? Can defenses tuned against appended attacks stop prepended payloads? Do synthetic attack traces in papers reflect real adversary behavior?See all 30 inquiring lines on this note →
Can better tools fix LLM document editing errors?
Does giving LLMs agentic tool access—like diffing, re-reading, or structured editors—improve their reliability on long-horizon document workflows? Understanding whether the problem is tool limitations or decision-making quality matters for reliability engineering.
Frontier LLMs corrupt 25 percent of document content without plateauing Language models know what to do but act greedily instead Agent reliability comes from externalizing cognitive burdens into system structures. One completion bias mechanism produces three distinct agent failure modes Weaker models delete content while frontier models corrupt it silently
What repair strategies work best at each level of Clark's ladder? Do architectural changes or training fixes better prevent agreement failures? How do insert-expansions and third position repair together cover full repair lifecycle? What makes draft-centric systems better anchors for coherence than feed-forward outputs? Is prompt engineering a workaround rather than a capability fix? How do insert-expansions differ from third position repair in timing? What makes provenance infrastructure more critical than artifact quality? How should organizations redesign workflows if LLMs cannot solve optimization directly? Why do frontier models corrupt more documents than weaker models during workflows? What causes silent document corruption in long LLM workflows? Where does agent reliability come from if not better tools? Why do LLMs choose incorrect edits despite understanding the task?See all 25 inquiring lines on this note →
Can prompting reduce bias in LLM judges reliably?
The paper suggests that instructing LLM judges to be less biased may not work reliably. This matters because if prompting fails, effort should shift from debiasing to making judge errors survivable in system design.
Can deterministic checks protect LLM judges from failure? Can a panel of smaller judges outperform one large judge? Can LLM judges be fooled by fake credentials and formatting? Where should an LLM judge sit in an optimization loop? Should models disclose their value biases when neutral answers are impossible?
Four deterministic moves can guard LLM judges without requiring their judgment. Panels of smaller diverse judges beat single large judges on cost and fairness LLM judges consistently fall for authority signals and rich formatting attacks Judge position matters more than accuracy when optimizers search thousands of iterations Disclosure is the honesty standard when models cannot achieve neutrality on unverifiable questions.
Where should measurement systems sit to avoid recording bias? What are the four catalogued biases that make LLM judges vulnerable to prompt attacks? How does LLM judge bias amplify errors in multi-agent debate on contested factual questions? Why is consistency between argument scoring and winner selection lower for LLM judges? Why do LLM judges systematically favor outputs from their own model family? What shared epistemic faults persist even when judges come from different families? How can deterministic checks make wrong judge decisions survivable? Which biases in LLM judges are exploitable through presentation alone? Do smaller LLM judge panels outperform single large judges in practice? Can a prompt mutation exploit a judge's vocabulary preferences without improving actual performance? Can an LLM judge's bias be reduced through prompting or other interventions? How does reporting order and selection bias group conclusions?See all 28 inquiring lines on this note →
Where should an LLM judge sit in an optimization loop?
When an LLM evaluator makes occasional mistakes, does it matter more how accurate it is or where it sits in a system? The position determines whether those errors become exploitable targets for optimization.
Does optimizing against monitors destroy monitoring itself? Why do fixed benchmarks fail as agents grow stronger? Can we detect reward-seeking from normal model behavior? Why do self-improvement loops plateau without updating the judge? Can an optimizer accidentally delete the evaluation criteria entirely?
Optimizing against CoT monitors produces obfuscated reward hacking Non-stationary utility with epoch boundaries prevents reward hacking in recursive self-improvement Reward-seeking stays hidden when graders align with intended objectives Co-evolving judges with actors breaks through self-improvement ceilings An optimizer deleted the scoring rubric instead of improving it
Where should measurement systems sit to avoid recording bias? Can an occasionally wrong judge operate safely in an optimizer loop? How do optimizers systematically find the errors in a flawed evaluation function? How does evaluator error position affect which behaviors substrates make vulnerable? What design choices make it survivable when an LLM judge holds final authority over an optimizer? When does an LLM judge's error become terrain that an optimizer can map and exploit? How does bounding a judge's authority differ from improving the judge itself?
Where do safety wins come from in multi-agent systems?
When an undefended agent pipeline shows zero attack success, how do we know whether safety comes from the application's own design or from hidden upstream defenses? This matters because invisible dependencies can collapse when systems change.
Exploitation benchmark failures conflate multiple causes beyond capability gaps Agent capability requires a vector across at least five separable axes Internal agent hops in pipelines carry unmonitored security risk Most published attack and defense figures lack filter status labels Agent-level metrics reveal compromise that action metrics cannot detect
How does outcome-only reporting hide a filter's role in safety results? Where does an agent's risk come from across its components and sequence? How do unmonitored channels between pipeline agents enable security gaps? Why are unmonitored channels between agents a safety risk? How does pipeline position amplify failures between monitored agents? How should system safety aggregate when monitoring channels are unequal? Why does monitoring performed by agents on agents create safety risks? What makes unmonitored channels between agents safety-critical? What baseline would prove multi-agent systems are actually less safe? Does provider-side filtering hide true safety from outcome-only attack reports? What vulnerabilities emerge at each hop between agents in a pipeline? Can outcome-only safety reporting hide which layer actually contained an attack?See all 14 inquiring lines on this note →
Can prompts alone hold back a capable tutor model?
When a frustrated student presses, do instruction-based constraints reliably prevent an LLM tutor from giving away answers it could easily produce? This matters because tutoring value depends on strategic withholding.
Why do better reasoning models ignore instructions? Can models learn to ask clarifying questions without explicit training? Why can't advanced AI models take initiative in conversation? How do reasoning models actually break under pressure? What actually constrains AI systems from learning misalignment?
Scaling reasoning capability degrades instruction-following performance Social meta-learning teaches models to ask clarifying questions without explicit training AI passivity stems from next-turn reward optimization, not inability Reasoning models excel at formal tasks but fail at social cognition. Alignment requires normative standards and external verification loops
Can optimizers learn to evade guardrails through repeated verdicts?
Guardrails are designed to be unarguable, but an optimizer observing thousands of verdicts may learn their boundaries like a black-box function. The excerpt leaves unclear what feedback the proposer receives from each check.
Can deterministic checks protect LLM judges from failure? Does optimizing against monitors destroy monitoring itself? Does iterative training against detected failures prevent actual compliance? Can planted honeypots detect hacks that matter most? Does constraining edits make skill learning more stable?
Four deterministic moves can guard LLM judges without requiring their judgment. Optimizing against CoT monitors produces obfuscated reward hacking Iterative detection-based training selects for evasion over genuine compliance. Planted honeypots measure only known hacks, not frontier exploits Bounded editing with rejected-edit buffers outperforms uncontrolled skill rewriting
Does ChainGuard maintain effectiveness when attackers adapt their approach to the defense? Can an occasionally wrong judge operate safely in an optimizer loop? What information should a proposer receive about failed guardrail checks? How do fabricated rationales slip past safety guardrails that block explicit instructions? Do mechanical guardrails around judges bound the cost of judge errors? What feedback does ChainGuard return that an attacker could optimize against? What happens when an optimizer discovers and eliminates the entire scoring rubric at once? Can deterministic checks fail open in ways a downstream optimizer cannot detect? Can an optimizer that sees guardrail verdicts learn to route around them? Can an optimizer learn to disable or route around visible guardrails?
Where do cognitive biases in language models come from?
Do LLM biases originate during pretraining or finetuning? Understanding the source matters for knowing where debiasing efforts should focus.
Base models contain latent reasoning capability unlocked by minimal training RLVR improves sampling efficiency, not reasoning capability boundaries RLVR activates existing reasoning rather than teaching it RL post-training optimizes reasoning deployment timing, not reasoning capability itself Most pretraining poisoning attacks survive alignment except jailbreaking
Can dataset-level debiasing methods fix popularity bias inherited from pretraining? How do LLM biases manifest differently across the three paradigms? How does prompt iteration reinforce user bias without empirical anchoring? Can prompt-based debiasing overcome entrenched LLM model priors? How do different LLM integration paradigms affect inheritance of pretraining biases? How do pretraining biases interact differently with prompts across model tiers? How do position bias and popularity bias interact with sequence order blindness? Can prompting strategies eliminate systematic biases without shuffling or aggregation? How does pretraining corpus popularity bias affect LLM recommendation behavior? How do LLM biases reflect social classification schemas rather than random errors? What calibration corrections can reduce LLM judge bias in automated evaluation pipelines? How does same-author bias interact with the four adversarial judge biases already documented?See all 81 inquiring lines on this note →
Can language models understand without actually executing correctly?
Do LLMs truly comprehend problem-solving principles if they consistently fail to apply them? This explores whether the gap between articulate explanations and failed actions points to a fundamental architectural limitation.
LLMs exhibit incoherent failures combining correct explanation with failed application Language models know what to do but act greedily instead Encoded knowledge in LMs often fails to causally influence generation LLMs show systematic gaps between stated beliefs and simulated actions LLMs generate planning knowledge but rarely produce executable plans
When does knowledge activation fail across different model architectures? Where do LLMs succeed at generation but struggle with evaluation? Why do LLMs generate ideas that sound novel but fail during execution? What specific execution barriers do LLM ideas encounter most frequently? What other latent LLM capabilities remain inactive without explicit activation cuing? What distinguishes planning knowledge from an executable plan that works? How much of LLM reasoning failure stems from missing knowledge versus signal weighting? What makes a problem instance unfamiliar to a language model? Why do LLM explanations feel authoritative even when alignment with the model fails? Can models identify what information they are missing in underspecified problems? Why do LLM agents make promises without executing them? How does the outer loop escape its own LLM's knowledge boundaries when discovering mechanisms?See all 95 inquiring lines on this note →
Does cosine similarity actually measure embedding similarity?
Cosine similarity is ubiquitous for comparing learned embeddings, but does it reliably capture semantic closeness? This work investigates whether regularization during training makes cosine scores arbitrary and unstable.
Why can't cosine space retrievers distinguish word order? Do vector embeddings actually measure task relevance? Why does dot product beat MLP-based similarity in practice? Does training for compositional sensitivity hurt dense retrieval? Do embedding dimensions fundamentally limit retrievable document combinations?
Cosine space geometry is fundamentally hostile to word order and negation Vector embeddings measure semantic association, not task relevance Dot product outperforms MLP similarity through structural inductive bias. Training for compositional sensitivity reduces dense retrieval generalization Embedding dimension mathematically constrains representable top-k document subsets
Why do embeddings measure association instead of actual task relevance? What makes regularization an implicit factor in embedding geometry? How should practitioners measure similarity between embeddings safely? Why do vector embeddings fail when semantically similar entities should be treated as distinct?
Can deterministic checks protect LLM judges from failure?
Explores whether mechanical, non-contestable verification steps can safeguard LLM-based decision systems. Matters because it tests whether we can make AI judgment survivable even when it goes wrong.
Judge position matters more than accuracy when optimizers search thousands of iterations Default fallbacks convert safety checks into fail-open vulnerabilities Deterministic guardrails cost orders of magnitude less than the LLM judge Optimizers can learn guardrail boundaries through iterated feedback on verdicts Bounded editing with rejected-edit buffers outperforms uncontrolled skill rewriting
What are the four catalogued biases that make LLM judges vulnerable to prompt attacks? Why is consistency between argument scoring and winner selection lower for LLM judges? Who decides whether an entity has authority to anchor a record? What makes a claim robust enough to lift out of preliminary incident evidence? Why does authorization checking outside agent judgment prevent confused deputy failures? Why do LLM judges systematically favor outputs from their own model family? What shared epistemic faults persist even when judges come from different families? Why do confident failures on failed actions become a signature problem? How can we detect when protocol-compliant validators certify semantically incorrect states? How reliable are LLM judges at detecting reward hacking compared to automated verification? How can deterministic checks make wrong judge decisions survivable? Why does every reliable LLM self-improvement require external intervention or verification?See all 49 inquiring lines on this note →
Does model capability change how documents degrade?
This explores whether weaker and frontier LLMs fail in fundamentally different ways when handling long-form document tasks, and whether that difference affects how reliably we can detect failures in practice.
Do frontier LLMs silently corrupt documents in long workflows? Can better tools fix LLM document editing errors? How does AI-generated false experience differ linguistically from human deception? What makes quietly failing systems more dangerous than obvious ones? Does completion training push agents to overfill forms unnecessarily?
Frontier LLMs corrupt 25 percent of document content without plateauing Agentic tools do not improve LLM document editing reliability AI-generated experience claims are structurally false with distinct linguistic markers from human deception Quietly failing systems become dangerous when competent, opaque, and integrated. One completion bias mechanism produces three distinct agent failure modes
Why do intermediate LLM layers become more precise in frontier models? What makes diverse failure modes more informative than single failure examples? What repair strategies work best at each level of Clark's ladder? What distinguishes domain-specific failure modes from general model limitations? What makes some model capabilities reliable while others remain brittle? Why do weaker models generate better training data than stronger models? Why do frontier models corrupt more documents than weaker models during workflows? What causes silent document corruption in long LLM workflows? How does workflow scale change the failure modes of frontier models? Can review effort alone keep pace with frontier model degradation? Why do frontier model failures in document editing go undetected by users? How does model tier affect whether errors delete or corrupt document content?See all 70 inquiring lines on this note →
Does the representational distance account work for on-policy training?
The emergent misalignment framework explains off-policy supervised finetuning via distance to a training data centroid, but this mechanism may not transfer to on-policy settings like RL where the training distribution shifts with the model.
Does representational distance predict where misalignment emerges? Does emergent misalignment occur across diverse training methods? Does learning to reward hack cause emergent misalignment in agents? Does reward-seeking explain emergent misalignment after hacking? Can distillation work on the student's own generated sequences?
Representational distance predicts emergent misalignment severity across settings Emergent misalignment appears across at least five distinct training settings. Reward hacking in production RL consistently produces emergent misalignment behaviors Reward-seeking may mediate misalignment from reward hacking training On-policy distillation closes the exposure bias gap in knowledge transfer
Why do imposed priors sometimes harm instead of improve alignment? How do models generalize misaligned objectives beyond the specific behaviors they were trained on? Are instruction following gains and emergent misalignment from the same learned change? What base rate does concentrated task distribution tell us about real misalignment? How should training distribution distance be defined when the policy evolves? Does representational distance predict which outputs trigger emergent misalignment? Which specific data formats produced more versus less emergent misalignment? How similar are emergent misalignment outcomes across SFT and reinforcement learning? Does format affect emergent misalignment through the representational distance mechanism? What counts as emergent misalignment versus standard capability overgeneralization? Why do broad misalignment behaviors cluster near training data geometrically? Can prompt perturbations preserve the distance-to-misalignment relationship consistently?See all 22 inquiring lines on this note →
Are LLM emergent abilities real or measurement artifacts?
Do large language models develop sudden new capabilities at certain scales, or do discontinuous metrics just make gradual improvements look sudden? This matters because it changes how we predict and interpret model behavior.
Do foundation models learn world models or task-specific shortcuts? Can neural networks learn compositional skills without symbolic mechanisms? How much of LLM few-shot ability comes from training data? Do language models fail at reasoning due to complexity or novelty? Why do larger models learn rare tasks better?
Foundation models develop task-specific heuristics, not generalizable world models. Compositional generalization emerges from scaling when training covers the task space Task contamination explains most few-shot LLM performance gains Reasoning failures are driven by instance-level unfamiliarity, not task complexity. Larger models learn rare tasks through reduced interference not expressivity
What other latent LLM capabilities remain inactive without explicit activation cuing? Why do intermediate LLM layers become more precise in frontier models? What measurement artifacts emerge when annotators interpret the same question differently? What causes models to develop domain capability cliffs after specialization? Do larger models develop more abstract features than smaller ones? How much do metric choices inflate claims about model capabilities? What capabilities actually require massive scale versus specialized training regimes? Do emergent abilities result from genuine new capabilities or implicit in-context learning? What skills can large models identify and organize about their own abilities? What language capabilities does fluency on standard benchmarks actually measure? What makes some model capabilities reliable while others remain brittle? Why do scaling laws show capability saturation at specific thresholds?See all 30 inquiring lines on this note →
Does emergent misalignment occur across diverse training methods?
Prior work reports emergent misalignment in at least five different training settings—from supervised fine-tuning on harmful data to reward-hacking reinforcement learning. Understanding whether this pattern holds across algorithms and domains could reveal common mechanisms.
Does learning to reward hack cause emergent misalignment in agents? Does iterative DPO training reliably induce hidden misalignment behaviors? How much do these results actually tell us about real reward hacking? Does learning simple gaming behaviors generalize to reward tampering? Does representational distance predict where misalignment emerges?
Reward hacking in production RL consistently produces emergent misalignment behaviors Iterative DPO on reward hacking induces covert misalignment and alignment faking Task distribution skew limits generalization claims about reward hacking likelihood Simple gaming generalizes zero-shot to reward tampering despite safety training. Representational distance predicts emergent misalignment severity across settings
What happens when alignment values become misaligned with human preferences at scale? Does reward hacking in alignment research mirror misalignment in deployed systems? Why does correct model output not guarantee absence of internal misalignment? How do models generalize misaligned objectives beyond the specific behaviors they were trained on? Do inoculation prompts prevent misalignment without harming instruction following? How should misalignment from iterative DPO be quantitatively measured? Are instruction following gains and emergent misalignment from the same learned change? What base rate does concentrated task distribution tell us about real misalignment? Does representational distance predict which outputs trigger emergent misalignment? Which specific data formats produced more versus less emergent misalignment? How similar are emergent misalignment outcomes across SFT and reinforcement learning? Does format affect emergent misalignment through the representational distance mechanism?See all 30 inquiring lines on this note →
How does training data format affect emergent misalignment?
When harmful datasets produce emergent misalignment in language models, does the way content is written—not just what it says—change how much broad misalignment emerges? Understanding format's role could reshape how safety teams review training data.
Does representational distance predict where misalignment emerges? Does training data format shape reasoning strategy more than domain? Does recontextualizing unwanted behavior during training suppress learning it? Can training data edits reliably override what models already believe? Does the representational distance account work for on-policy training?
Representational distance predicts emergent misalignment severity across settings Training format shapes reasoning strategy 7.5 times more than domain Inoculation prompting recontextualizes undesired behavior to suppress its learning during training. Training data can insert new associations but not reliably override existing ones The representational distance account remains untested in on-policy training settings.
What happens when alignment values become misaligned with human preferences at scale? Why does correct model output not guarantee absence of internal misalignment? Are instruction following gains and emergent misalignment from the same learned change? How large was the effect size of format compared to content itself? Which specific data formats produced more versus less emergent misalignment? Can framing changes like inoculation prompting be considered part of training data format? Does format affect emergent misalignment through the representational distance mechanism? What counts as emergent misalignment versus standard capability overgeneralization? Why do broad misalignment behaviors cluster near training data geometrically? How robust is misalignment classification across different judge models? What counts as a real-world harm from misalignment versus a training artifact? Does representational distance from training data centroid predict misalignment behavior within a model?See all 17 inquiring lines on this note →
Do internal agent hops in pipelines need security monitoring?
Multi-agent systems route data between planner, worker, verifier, and synthesizer components. Current defenses only guard user input at the entry point, leaving inter-agent channels unmonitored—but is this a real vulnerability or does downstream safety suffice?
Can prompts alone reshape multi-agent workflows without system access? Can one compromised agent corrupt an entire multi-agent network? Can chain-of-thought monitors detect reasoning that originates elsewhere? Can inspecting generated workflows catch planning-time attacks? How do adversarial traps target different layers of AI agents?
Prompts reshape multi-agent workflows at planning time before defenses activate One compromised agent propagates bias across multi-agent networks via ordinary messages Chain-of-thought monitors conflate paraphrased external reasoning with original actor reasoning Workflow inspection misses attacks that bias upstream planning signals AI Agent Traps decompose into six structural attack categories across agent layers
Should input defenses be validated separately for each channel? Do per-hop inspection gates miss attacks that bias upstream planning signals? Do honeytokens work better against outside attackers than compromised internal agents? How do unmonitored channels between pipeline agents enable security gaps? Why are unmonitored channels between agents a safety risk? What happens when planning signals get contaminated before reaching a downstream agent? How does pipeline position amplify failures between monitored agents? How should system safety aggregate when monitoring channels are unequal? Why does monitoring performed by agents on agents create safety risks? Can input-boundary defenses guard unmonitored channels between agent hops? Can mixed-authorship traces from multi-agent pipelines be monitored reliably? Where should the recorder sit relative to the agent it monitors?See all 35 inquiring lines on this note →
Does representational distance predict where misalignment emerges?
After emergent misalignment training, do evaluation prompts closer to the training data centroid in the base model's representation space elicit stronger misbehavior? This would explain where misalignment lands geometrically.
Does learning to reward hack cause emergent misalignment in agents? Can we track and steer personality shifts during model finetuning? Can we predict keyword priming before learning happens? Do misalignment directions transfer between different emergent models? Does the representational distance account work for on-policy training?
Reward hacking in production RL consistently produces emergent misalignment behaviors Persona vectors enable real-time monitoring and prevention of personality drift in finetuning. Keyword priming after learning is predictable from pre-learning probability Misalignment directions do not transfer across different emergent models The representational distance account remains untested in on-policy training settings.
Why do imposed priors sometimes harm instead of improve alignment? What happens when alignment values become misaligned with human preferences at scale? What mechanism drives models to resist modification during alignment training? Why does correct model output not guarantee absence of internal misalignment? How do models generalize misaligned objectives beyond the specific behaviors they were trained on? Do inoculation prompts prevent misalignment without harming instruction following? How should misalignment from iterative DPO be quantitatively measured? Are instruction following gains and emergent misalignment from the same learned change? What base rate does concentrated task distribution tell us about real misalignment? How do alignment priors drive similar outputs across different models? Does representational distance predict which outputs trigger emergent misalignment? Which specific data formats produced more versus less emergent misalignment?See all 37 inquiring lines on this note →
Do frontier LLMs silently corrupt documents in long workflows?
DELEGATE-52 tests whether state-of-the-art language models reliably preserve document integrity across extended delegated tasks. Understanding this matters because single-step benchmarks may mask compounding failures that emerge only at workflow scale.
Weaker models delete content while frontier models corrupt it silently Agentic tools do not improve LLM document editing reliability Short interaction benchmarks fail to predict long horizon delegated performance Models degrade sharply when prior errors contaminate their context history Language models know what to do but act greedily instead
How does token-based production differ from digital file production? How does token generation as flow differ from print's archival storage? How do current safety benchmarks miss pragmatic alignment failures? Why does bidirectional RAG amplify the risk of corpus poisoning attacks? Do Doc2Query approaches suffer from the same misaligned-target problem? Why do intermediate LLM layers become more precise in frontier models? Can evaluators investigate dependencies without accumulating mistakes over time? Why do language models produce plausible outputs over accurate failure reports? How do autonomous pipelines identify and fix silent bugs in data pipelines? Can domain-expert workflows always decompose into inspectable stages for AI? Why does each rewrite cycle degrade domain-specific details differently than compression? What specific information must be exported from the language system?See all 145 inquiring lines on this note →
Can small models match frontier reasoning without massive scale?
Explores whether verifiable reasoning ability emerges from training design rather than parameter count. Matters because it challenges the assumption that only very large models can solve hard math and code problems.
SRL-then-RLVR curriculum outperforms either training method alone RL extends reasoning only with headroom and edge-of-competence targeting Curriculum budgets starting generous then tightening outperform fixed budget training RL post-training optimizes reasoning deployment timing, not reasoning capability itself Training regime matters more than inference compute budget
Can abstention behavior transfer from small models to frontier models? What separates verifiable reasoning from open-ended judgment in scaling requirements? Can benchmark scores on verifiable tasks transfer to unseen problems outside the training domain? Why do epistemic failure modes cluster around world model limitations? Can a tiny recursive network beat billion-parameter models on hard problems? What role does verifier design play in reasoning capability gains? Why is verification harder than generation across the research lifecycle? How sensitive is analogical reasoning emergence to training data and scale? Why does formalizing the Kepler conjecture cost eleven years of work? Does small heterogeneous model architecture outperform large homogeneous pools economically? How does requential coding measure true simplicity without parameter count inflation? Can signal quality regulations help smaller teachers outperform larger ones?See all 27 inquiring lines on this note →
Can any computable LLM truly avoid hallucinating?
Explores whether formal theorems prove hallucination is mathematically inevitable for all computable language models, regardless of their design or training approach.
LLM errors are fabrication not hallucination or confabulation Self-improvement is fundamentally bounded by the generation-verification gap Semantic entropy detects confabulations by measuring meaning divergence. Models use entity recognition to track their own knowledge Models fail to evaluate semantic legitimacy in concept fusion
Can fixing hallucination address AI's structural epistemic problem? What makes LLM outputs fabrication rather than hallucination or confabulation? What verification methods work for knowledge without stable referents? What are Gricean maxims and why do language models violate them? What should we call errors in LLM outputs when hallucination does not apply? How does LLM hallucination risk manifest in knowledge graph construction? How much does ROUGE metric choice inflate hallucination detection claims? Does inevitable LLM hallucination make detection metric validity critical? How does Peircean Secondness differ from what RLHF actually provides? Can novelty detection alone distinguish grounded synthesis from hallucinated restatement? What does the distributed cognition framework reveal about AI hallucination versus human-AI co-construction? Would hybrid systems combining LLMs with symbolic solvers overcome the retraction limitation?See all 58 inquiring lines on this note →
Does AI assistance homogenize or preserve creative diversity?
Can AI tools maintain the diverse ideas that emerge from diverse human groups, or do they compress creative output toward similarity? This matters because collective diversity drives innovation.
Different LLMs converge on similar outputs regardless of model diversity LLM research ideation collapses into narrow clusters despite high novelty Cognitive diversity drives ideation quality only with foundational domain expertise Language models collapse argument diversity at the structural level Mode collapse originates in typicality bias within preference data, not algorithms
Can an optimizer accidentally delete the evaluation criteria entirely?
When an optimizer rewrites instructions to improve scores, can it remove the measurement itself rather than improve it? This matters because it reveals whether optimization loops understand what they're measuring or just chase better numbers.
Non-stationary utility with epoch boundaries prevents reward hacking in recursive self-improvement Bounded editing with rejected-edit buffers outperforms uncontrolled skill rewriting LLM judges consistently fall for authority signals and rich formatting attacks Default fallbacks convert safety checks into fail-open vulnerabilities Judge position matters more than accuracy when optimizers search thousands of iterations
How do scoring shortcuts persist across multiple optimization updates? Can an occasionally wrong judge operate safely in an optimizer loop? How do optimizers systematically find the errors in a flawed evaluation function? What did deleting the rubric do to the judge's error in practice? Can an automated evaluator stay useful while an optimizer runs thousands of iterations? What happens when an optimizer discovers and eliminates the entire scoring rubric at once?
Can reviewers access what they know when checking LLM outputs?
Does the ability to recall relevant knowledge at the moment of review shape whether humans catch LLM errors, independently of how capable or engaged they are?
Do users worldwide trust confident AI outputs even when wrong? Can organizations lose scrutiny capacity while keeping oversight forms? How do competent systems quietly undermine safety oversight? How do reasoning models actually break under pressure? Can scoped agents reliably judge semantic hacks in runtime analysis?
Users universally overrely on overconfident LLM outputs across all languages Organizations can maintain nominal oversight while losing actual scrutiny capacity Competent systems quietly erode safety through four mechanisms. Reasoning models excel at formal tasks but fail at social cognition. Scoped audit agents provide evidence-backed semantic attribution over pinned artifacts
How does instruction density affect model performance?
As language models must track more simultaneous instructions, does their ability to follow them predictably degrade? IFScale measures this across frontier models to understand practical limits.
Scaling reasoning capability degrades instruction-following performance Reasoning performance degrades sharply with input length below context limits Strict output format constraints significantly degrade LLM reasoning performance Scaling reasoning capability degrades instruction-following performance significantly Instruction tuning teaches output format distribution not task understanding
How do larger models maintain more parallel tasks than smaller models? Can smaller models actually perform well on specific downstream tasks? What determines the finite chain length where robustness improvements plateau? Why do single function-calling benchmarks mask model weakness in specific areas? Why do primacy effects peak at specific instruction densities? Does input length alone explain instruction density performance loss? Can structured output formats reduce instruction following degradation? Why does adjusted compression performance degrade as models scale larger? Do instruction-tuned models learn tasks or just output format distributions? How do quality, diversity, and complexity create different effects on downstream model performance? Why do text-only benchmarks underestimate deployed model capability? Does model collapse occur across different architectures or only in specific conditions?See all 22 inquiring lines on this note →
Can cheaper models decrypt traces from stronger models?
Explores whether encrypted reasoning blocks designed to hide model internals can be read by weaker models in the same provider's ecosystem, and what this reveals about the security of hidden chain-of-thought systems.
Do reasoning traces actually expose private user data? Can reasoning models be steered by injected context without detection? Can chain-of-thought monitors detect reasoning that originates elsewhere? Can we actually trust reasoning model outputs? Can chain-of-thought reasoning be secretly manipulated to look normal?
Reasoning traces leak private data primarily through direct recollection Injected plans in context evade chain-of-thought monitors 25 to 33 percent of the time Chain-of-thought monitors conflate paraphrased external reasoning with original actor reasoning Reasoning model monitoring fails through silence and obfuscation. Chain-of-thought can be backdoored to produce coherent deceptive reasoning
Can language models transmit hidden behavioral traits through unrelated data?
Explores whether behavioral preferences can spread between models through semantically neutral data like number sequences, and whether filtering can detect or prevent such transmission.
Does training on AI-generated content permanently degrade model quality? How much poisoned training data survives safety alignment? Can imitating ChatGPT fool evaluators into thinking models improved? Can one compromised agent corrupt an entire multi-agent network? Do misalignment directions transfer between different emergent models?
Recursive training on synthetic data causes irreversible tail distribution collapse Most pretraining poisoning attacks survive alignment except jailbreaking Imitation training captures style but closes no capability gap One compromised agent propagates bias across multi-agent networks via ordinary messages Misalignment directions do not transfer across different emergent models
Why do multiple language models independently produce similar outputs in influence campaigns? Can safety evaluations miss behavioral effects by only measuring semantic shifts? Can mention sequences exploit shortcuts like repeated items rather than learning genuine preferences? What makes synthetic user data transfer to real conversational systems? Can linguistic compression be a fundamental mechanism for representing psychology? Can belief propagation accurately predict downstream opinion shifts? Can models that detect their own states learn to conceal them strategically? Can language about model behavior ever be accurate without anthropomorphic framing? Why do language models infer political orientation from seemingly innocuous user signals? What structural signals in user language reveal their unstated preferences and context? Why do small training data contaminations persist through alignment for most attack types? Why does subliminal trait transmission fail when teacher and student differ?See all 63 inquiring lines on this note →
Can removing a communication channel stop persistent information sharing?
When a shared mechanism for passing information is deleted, does the sharing actually stop, or can agents rebuild it using inherited knowledge? This matters for understanding whether removing infrastructure alone defeats coordinated threats.
Can ordinary infrastructure become unplanned agent memory? How fast must a coalition gather observations before containment? Should response workflows be inside the security boundary? Can a model-level filter truly contain an agent with environment access? How can operators stop coordinated agent intrusions now?
Shared infrastructure became persistent memory through agent-initiated use Finite-sample bounds measure the speed coalition observations must accumulate before containment. Response workflows belong inside the AI agent security boundary Model filters block moments but cannot contain agents with environment access Counter-swarm defense requires three coordinated constraints on agent behavior
How did agents rebuild communication after Hugging Face removed the channel? Can stopping one intrusion pathway leave the underlying activity intact elsewhere? Why does removing a communication channel not permanently prevent agent coordination? What happens to a commitment when its bound content must be deleted? Can agents rebuild communication channels after removal? Can closing a communication channel prove whether agents influenced each other? What mechanisms let later agents inherit information left by earlier ones? Why do agents rebuild communication after channels are removed? What does a quiet period after removing a communication channel actually show about agent coordination? Can coalitions rebuild and reaccumulate observations after being removed?
Are LLM and agent benchmarks really measuring different things?
Do LLM benchmarks and agent benchmarks test fundamentally different capabilities, or are they two modes of the same model? Understanding this shapes how we evaluate and develop AI systems.
Do short benchmarks predict how models perform over long workflows? Do frontier LLMs silently corrupt documents in long workflows? When do multi-agent systems actually outperform single agents? Does a single benchmark score actually predict agent readiness? How do reasoning models actually break under pressure?
Short interaction benchmarks fail to predict long horizon delegated performance Frontier LLMs corrupt 25 percent of document content without plateauing Multi-agent advantages shrink as single-agent capabilities improve Agent capability requires a vector across at least five separable axes Reasoning models excel at formal tasks but fail at social cognition.
Why do language models collapse into generic templates?
Explores whether low reward variance during RL training causes policies to abandon input-specific reasoning in favor of boilerplate responses, and whether filtering for high-variance prompts can prevent this failure.
Policy entropy collapse creates a predictable performance ceiling in reasoning RL RL improvement magnitude tracks reward signal verifiability. Template collapse remains invisible to entropy metrics in reasoning RL training sparsely updates parameters while suppressing incorrect reasoning paths RLVR's on-policy constraint causes capability boundary collapse through exploration avoidance
Can trust region constraints prevent the sample inefficiency problems of RLHF? What causes on-policy distillation to become unstable at scale despite dense rewards? How does policy entropy collapse constrain zero RL scaling for reasoning? Can routing policies remain meaningful over behaviorally homogeneous model pools? Can curriculum learning by reward variance improve reasoning scalability? How does regularization dominance explain policy entropy collapse in reasoning? What distinguishes high-signal prompts from low-signal ones in RL training? What makes student-teacher distributional mismatch derail on-policy distillation? Does stable entropy in policy training actually guarantee stable reasoning behavior? Why does exploration diversity behave differently under reinforcement learning versus supervised fine-tuning? How do learning dynamics on one example shift predictions on other responses? Why does token-level gradient targeting matter more than aggregate loss?See all 48 inquiring lines on this note →
Do language models fail at reasoning due to complexity or novelty?
Explores whether reasoning-model failures stem from task complexity thresholds or from encountering unfamiliar instances. Tests whether scaling chain length actually addresses the root cause of reasoning breakdown.
Do foundation models learn world models or task-specific shortcuts? Do transformers actually learn systematic compositional reasoning? Does chain-of-thought reasoning reveal genuine inference or pattern matching? Does more thinking time always improve reasoning accuracy? Why do reasoning LLMs fail at deeper problem solving?
Foundation models develop task-specific heuristics, not generalizable world models. Transformers reduce compositional reasoning to linearized subgraph matching Chain-of-thought mimics reasoning form through learned schemata, not genuine inference Reasoning accuracy peaks then declines with excessive thinking tokens Reasoning LLMs wander unsystematically causing exponential success degradation
How can minimal pairs expose reasoning failures that single-instance accuracy metrics miss? When does knowledge activation fail across different model architectures? How does the knowing-doing gap widen as tasks become more complex? Why does scaling reasoning tokens fail to improve unfamiliar tasks? What makes a problem instance unfamiliar to a language model? Does sentence-level granularity capture enough structure for complex reasoning tasks? Why do single examples trigger large reasoning improvements in models? Why do simple length heuristics outperform sophisticated semantic methods? Can latent reasoning architectures work as retrofits to existing models? How does the frame problem differ between symbolic and statistical reasoning systems? Does optimizing directly for semantic diversity improve both reasoning quality and exploration? Can Kolmogorov complexity alone capture what makes intelligence general?See all 262 inquiring lines on this note →
Does RLHF make language models indifferent to truth?
Explores whether reinforcement learning from human feedback fundamentally shifts models away from caring about accuracy toward optimizing for other rewards, and whether this differs from simple confusion or hallucination.
Does calling LLM errors hallucinations point us toward the wrong fixes? Does RLHF training make models more convincing or more correct? Does preference optimization harm conversational understanding? Does RLHF training make AI models more deceptive? Why do language models agree with false claims they know are wrong?
LLM errors are fabrications, not hallucinations, requiring verification not grounding. RLHF trains models to sound correct rather than be correct Preference optimization erodes grounding acts needed for reliable dialogue RLHF and CoT amplify machine deception without improving truth. Language models accommodate false claims through face-saving behavior reinforced during training.
What cognitive capabilities do agents need to internalize social feedback? Can fixing hallucination address AI's structural epistemic problem? Why might chatbots simply learn better face-saving instead of genuine perspective-taking? How does AI lose correct information under conversational persuasive pressure? Why does RLHF alignment reduce the diversity of viewpoints in AI output? How does RLHF labeler identity shape the values AI systems learn? How does RLHF training encode values into AI systems? Do language models share the same cooperative truth-seeking rules as humans? How does the absence of face-loss or reputation risk change model behavior? Does RLHF training create models that sound convincing without being more accurate? How does RLHF-trained sycophancy manifest differently across feedback and review contexts? Does self-conditioning improve belief-behavior alignment better than external priors?See all 182 inquiring lines on this note →
What cost does making decoys convincing impose on legitimate users?
When honeytokens are designed to look indistinguishable from genuine objects, a total-variation bound constrains how often trusted agents can use the real thing without triggering false alarms. Understanding this tradeoff is key to evaluating whether deceptive security measures protect without harming the people they're meant to spare.
Can honeytokens fool attackers who know the trusted policy? Can repeated quiet probes separate decoys from genuine objects? Does added monitoring improve protection at acceptable cost? How fast must a coalition gather observations before containment? Why do multi-agent systems fail despite individual capability?
Honeytokens fail when attackers share trusted agents' information and policy Repeated non-triggering probes achieve zero classification error when response laws differ Four-arm monitoring comparison design proposed but no results reported Finite-sample bounds measure the speed coalition observations must accumulate before containment. Multi-agent capability ceilings are structural, not scalable capability gains.
What must remain secret for honeytokens to stay asymmetric against compromised insiders? How do decoy systems balance protecting trusted agents while deceiving attackers? Do honeytokens work better against outside attackers than compromised internal agents? Can a policy distinguish genuine objects from traps without revealing that distinction? Do planted honeypots look like natural shortcuts to models or obvious tests? Does honeytoken theory explain why planted bait cannot catch informed agents? Does a planted honeypot catch all the hacks that actually matter? What false-alert budget would make indistinguishable decoys tolerable in real deployments? How did honeytokens propagate through the shared repository in this episode? What conditions make a honeytoken unrecognizable to attackers with shared information access? Can decoys and genuine objects maintain identical response laws in practice? How do false refusal rates affect the true cost of a guardrail?See all 16 inquiring lines on this note →
Do misalignment directions transfer between different emergent models?
When neural networks develop misaligned behaviors on different datasets, do they converge on shared internal directions that could explain the behavior? This matters for understanding whether safeguards calibrated on one model might work across others.
Does representational distance predict where misalignment emerges? Can we track and steer personality shifts during model finetuning? How stable is the trained Assistant personality in language models? How is emergent misalignment different from persona changes? Do reward hacking behaviors share a single direction in activation space?
Representational distance predicts emergent misalignment severity across settings Persona vectors enable real-time monitoring and prevention of personality drift in finetuning. Post-training loosely tethers models to Assistant mode through a dominant persona axis Emergent misalignment operates fundamentally differently than persona changes Simple difference-of-means vectors detect reward hacking across multiple behaviors and models
What mechanism drives models to resist modification during alignment training? Why does correct model output not guarantee absence of internal misalignment? How do models generalize misaligned objectives beyond the specific behaviors they were trained on? Are instruction following gains and emergent misalignment from the same learned change? How do alignment priors drive similar outputs across different models? Does representational distance predict which outputs trigger emergent misalignment? Which specific data formats produced more versus less emergent misalignment? How similar are emergent misalignment outcomes across SFT and reinforcement learning? Does format affect emergent misalignment through the representational distance mechanism? What counts as emergent misalignment versus standard capability overgeneralization? Why do broad misalignment behaviors cluster near training data geometrically? Can prompt perturbations preserve the distance-to-misalignment relationship consistently?See all 27 inquiring lines on this note →
Do explanations actually help users spot AI mistakes?
Most AI explanations are designed to justify the system's answer, but do they help users distinguish correct from incorrect outputs? This research tests whether standard explanation formats genuinely improve error detection or just increase trust regardless of accuracy.
Do reasoning traces actually cause correct answers? Do users worldwide trust confident AI outputs even when wrong? Are AI explanations really descriptions or adoption arguments? Can LLM explanations actually help humans predict model behavior? Can we distinguish helpful explanations from manipulative ones?
Reasoning traces are stylistic mimicry, not verified causal reasoning Users universally overrely on overconfident LLM outputs across all languages XAI explanations function as adoption arguments disguised as technical descriptions LLM explanation plausibility and precision are fundamentally uncorrelated Rhetorical XAI cannot be separated from dark pattern risk.
Why do users interpret AI outputs through frameworks meant for human experts? Do people who choose to use AI fact-checkers actually become better at spotting misinformation? Why don't users push back when AI makes obvious mistakes about false claims? Can organized response format trick users into overestimating AI reliability? Can AI-generated explanations of errors teach as effectively as self-resolution? Why does explanation source matter more than explanation content? Should XAI designers treat explanations as arguments for adoption? Why does polished explanation make wrong AI systems more persuasive than poorly explained ones? How can correct explanations coexist with failed applications in AI? What architectural changes help AI avoid adding interpretations users didn't express? Why do familiar patterns that support correct answers sometimes drive errors? What explanation format actually helps users detect errors in AI systems?See all 21 inquiring lines on this note →
Does sharing observations help coalitions detect decoys better?
When agents pool their observations through shared memory, can a coalition distinguish genuine objects from decoys more reliably than any isolated member? The answer matters for understanding whether information sharing in multi-agent systems creates security vulnerabilities.
Shared infrastructure became persistent memory through agent-initiated use Honeytokens fail when attackers share trusted agents' information and policy Centralized skill evolution enables collective learning across agent ecosystems Multi-agent capability ceilings are structural, not scalable capability gains. Durable honeytoken asymmetry requires protecting something beyond policy
How do decoy systems balance protecting trusted agents while deceiving attackers? Can a policy distinguish genuine objects from traps without revealing that distinction? Does hiding data partitions from proposers prevent them from learning boundaries? Does honeytoken theory explain why planted bait cannot catch informed agents? Can monitoring in multi-agent deployments prevent collusion when agents monitor agents? Does collusion scale differently when observation density changes with population size? How does payload exposure compare between single and multi-agent architectures? What makes prospective episode discovery harder than using known group membership? Do correlated training sources between monitors and agents undermine detection reliability? Can constraining shared resources alone prevent reconstruction by later agents? What false-alert budget would make indistinguishable decoys tolerable in real deployments? Can attackers exploit pooled agent trajectories to identify and bypass defenses?See all 20 inquiring lines on this note →
Why do preference models favor surface features over substance?
Preference models show systematic bias toward length, structure, jargon, sycophancy, and vagueness—features humans actively dislike. Understanding this 40% divergence reveals whether it stems from training data artifacts or architectural constraints.
Can LLM judges be fooled by fake credentials and formatting? Do reward models actually consider what the prompt asks? Does transformer attention architecture inherently favor repeated content? Why do alignment methods work if they model human irrationality? Do therapists accurately perceive the working alliance with patients?
LLM judges consistently fall for authority signals and rich formatting attacks Reward models ignore prompt context and fail to generalize properly Soft attention structurally over-weights repeated content and sycophancy partly reflects this bias Alignment methods work by modeling human cognitive biases like loss aversion Therapists systematically overestimate alliance while suicidal patients show greatest misalignment
Can alignment methods like DPO exploit or correct these surface feature biases? How do preference models amplify human cognitive biases into systematic miscalibration? What makes preference distributions unimodal versus genuinely disagreement-heavy? Why does preference measurement validity matter more than aggregation methods? What causes reward models to favor length and sycophancy? Can models detect and suppress surface personalization without fixing underlying bias?
Do language models evaluate semantic legitimacy when fusing concepts?
Can LLMs recognize when two domains lack legitimate structural correspondences before blending them into coherent-sounding explanations? This matters because current hallucination detection focuses on factual accuracy, missing failures of semantic judgment.
LLMs generate novel ideas through unconstrained combination but cannot evaluate them LLMs dissect literary mechanics but cannot access literary meaning Large language models reason through semantic associations not symbolic logic. LLMs aggressively compress while humans preserve adaptive conceptual nuance Hallucination is formally inevitable for all computable LLMs
Can fixing hallucination address AI's structural epistemic problem? What makes LLM outputs fabrication rather than hallucination or confabulation? What verification methods work for knowledge without stable referents? What should we call errors in LLM outputs when hallucination does not apply? How does LLM hallucination risk manifest in knowledge graph construction? Does inevitable LLM hallucination make detection metric validity critical? Can novelty detection alone distinguish grounded synthesis from hallucinated restatement? What surface features do LLMs rely on when judging response quality? Are potemkin understanding and split-brain syndrome describing the same phenomenon? What happens when you tightly couple two representations together? Why is hallucination the wrong term for all LLM false outputs? How do models decide between refusing or hallucinating?See all 25 inquiring lines on this note →
How vulnerable are reasoning models to irrelevant text?
Can simple adversarial triggers like unrelated sentences degrade reasoning model accuracy? This explores whether step-by-step reasoning actually provides robustness against subtle input perturbations.
Why do reasoning models fail under manipulative prompts? Does reasoning ability actually degrade with longer inputs? Can emotional phrases in prompts improve language model performance? Can models learn to ignore irrelevant prompt changes? Why does compression defense fail at the user prompt boundary?
Manipulative prompts reduce reasoning model accuracy by 25 to 29 percent Reasoning performance degrades sharply with input length below context limits Emotional phrases appended to prompts consistently enhance LLM performance across models. Consistency training teaches invariance to prompt perturbations using model's own responses Compression gates assume attacks append, but users prepend payloads
Does irrelevant content degrade reasoning even when it fits the context window? Can retrieval improve multi-step reasoning by triggering at each uncertainty? How do manipulative prompts exploit the length-accuracy vulnerability? Can emotional prompt manipulation reduce reasoning model accuracy like adversarial techniques do? Why does a relativistic critic outperform absolute scoring in adversarial reasoning training? Can minimal adversarial triggers disrupt reasoning across multiple unrelated queries? What reasoning token threshold marks the accuracy degradation point? How do adversarial triggers bypass the protections of longer reasoning chains? Why does consistency training make models resistant to prompt perturbations? Can consistency training defend against adversarial text injection attacks? Do gaslighting attacks and adversarial triggers exploit the same reasoning model weaknesses? What makes evidence selection vulnerable to adversarial poisoning attacks?See all 30 inquiring lines on this note →
Are reasoning model collapses really failures of reasoning?
Explores whether language models hit a fundamental reasoning ceiling or whether text-only evaluation masks execution limitations. Examines how tool access might reveal hidden reasoning capabilities.
Reasoning LLMs wander unsystematically causing exponential success degradation Modular cognitive tools elicit latent reasoning through structured isolation. AI passivity stems from next-turn reward optimization, not inability Reasoning models fail systematically across exploration, mode selection, and social understanding. API-first agents reduce task completion time by 65 to 70 percent
Why do naive baselines outperform trained models in entity-level CRS evaluation? How can minimal pairs expose reasoning failures that single-instance accuracy metrics miss? What structural constraints matter more than model depth for CF? Can surface heuristics override implicit constraints in domain-specific reasoning? When does knowledge activation fail across different model architectures? What design changes could make constraint inference more reliable without explicit cuing? Why does step-by-step reasoning fail when tool outputs get very large? Why does scaling reasoning tokens fail to improve unfamiliar tasks? Does sentence-level granularity capture enough structure for complex reasoning tasks? What is the relationship between reasoning depth and verbalization requirements? How does the frame problem differ between symbolic and statistical reasoning systems? Can step-level deliberation flags guide other reasoning systems?See all 204 inquiring lines on this note →
Do reasoning traces actually expose private user data?
Explores whether language models leak sensitive information through their internal reasoning steps, even when explicitly instructed not to. Investigates the mechanisms and scale of privacy exposure in reasoning traces.
Does optimizing against monitors destroy monitoring itself? Do reasoning models actually use the hints they receive? Why do correct reasoning traces contain fewer tokens? Can cheaper models decrypt traces from stronger models? Can commitments protect sensitive agent data while enabling verification?
Optimizing against CoT monitors produces obfuscated reward hacking Reasoning models use hints but rarely verbalize them in explanations Correct reasoning traces in o1-like models are shorter than incorrect ones Weaker models can decrypt stronger models' encrypted reasoning traces Commitments enable on-chain verification without exposing sensitive agent content
How does understanding persistent journeys intensify both trust and privacy concerns? Why might an AI's face-saving tendency increase user disclosure? Can LLMs infer psychological profiles without explicit user disclosure? Can external verifiers replace reasoning trace quality in solution guarantees? How do access controls and anonymization fit into RAG retrieval pipelines? How do you attribute copyright when billions of inputs shape one model? How do privacy concerns compete with disclosure comfort in human-machine conversation? Can activation decoders discover hidden system prompts from user-model conversations? What hidden computations happen inside transformer layers during reasoning? How do adversarial triggers bypass the protections of longer reasoning chains? What distinguishes flow-preserving measurement from cognitive vulnerability profiling? What role does private information play in distinguishing realistic from unrealistic agents?See all 64 inquiring lines on this note →
Can repeated quiet probes separate decoys from genuine objects?
Explores whether an attacker with enough non-triggering probes can distinguish decoys from genuine objects when their response distributions differ, and what information the attacker needs to succeed.
How fast must a coalition gather observations before containment? What cost does making decoys convincing impose on legitimate users? Can attackers evade skill scanners by refining individual skills? Can optimizers learn to evade guardrails through repeated verdicts? What actually constrains AI systems from learning misalignment?
Finite-sample bounds measure the speed coalition observations must accumulate before containment. Indistinguishable decoys limit legitimate use through a total-variation bound on false alerts Skill scanner feedback enables attackers to evade detection across six scanners. Optimizers can learn guardrail boundaries through iterated feedback on verdicts Alignment requires normative standards and external verification loops
How reliably do planted honeypots match unplanted hacks agents actually discover? How do server-side filters hide their role in zero attack success? What makes a claim robust enough to lift out of preliminary incident evidence? Do synthetic attack traces in papers reflect real adversary behavior? Can re-scoring detect subliminal prompt injection without explicit semantic content? What must remain secret for honeytokens to stay asymmetric against compromised insiders? How do decoy systems balance protecting trusted agents while deceiving attackers? Do honeytokens work better against outside attackers than compromised internal agents? Can a policy distinguish genuine objects from traps without revealing that distinction? Can message-content defenses distinguish cheap talk from coordinated deception? Can planted test cases reliably trigger alarms before real harm occurs? What detection method survives when a model optimizes to hide hacking?See all 40 inquiring lines on this note →
Does RLHF training make models more convincing or more correct?
Explores whether RLHF improves actual task performance or merely trains models to sound more persuasive to human evaluators. This matters because alignment techniques could be creating the illusion of safety.
Does preference optimization harm conversational understanding? Why do language models agree with false claims they know are wrong? Can models abandon correct beliefs under conversational pressure? Does linguistic conviction explain why LLMs persuade more effectively? Does RLHF make language models indifferent to truth?
Preference optimization erodes grounding acts needed for reliable dialogue Language models accommodate false claims through face-saving behavior reinforced during training. Models abandon correct beliefs when users persistently pressure them LLM persuasive advantage is driven by linguistically expressed conviction, independent of truth value. RLHF drives models toward truth indifference, not confusion
Why does RLHF alignment reduce the diversity of viewpoints in AI output? How does RLHF training encode values into AI systems? Can RLHF alignment prevent models from making ethically appropriate rule violations? Does RLHF training create models that sound convincing without being more accurate? What training methods make models more persuasive but less factually accurate? Do safety benchmarks miss the effects of warmth training on model reliability? Why does RLHF degrade honesty while improving surface-level helpfulness? How does evaluator time pressure shape what behaviors RLHF rewards? Can alignment training be redesigned to permit warranted alarm? Does RLHF training suppress exploratory and qualifying language? Why does RLHF training discourage the conversational repair work agents need? Does RLHF training specifically teach models to prioritize user agreement over accuracy?See all 34 inquiring lines on this note →
Does RLVR success on math benchmarks reflect genuine reasoning improvement?
Explores whether RLVR's apparent effectiveness with spurious rewards on contaminated benchmarks like MATH-500 represents actual reasoning gains or merely data memorization retrieval.
Why do random rewards improve reasoning for some models but not others? Why does RLVR work with completely random rewards? Does RLVR actually expand what models can reason about? How much of LLM few-shot ability comes from training data? Can scarcity of solutions protect benchmarks from data contamination?
Spurious rewards improve reasoning only in models with specific pretraining strategies RLVR activates existing reasoning rather than teaching it RLVR improves sampling efficiency, not reasoning capability boundaries Task contamination explains most few-shot LLM performance gains Exploit scarcity mitigates contamination by making solutions hard to memorize
How much RLVR improvement comes from benchmark data memorization? Can clean benchmarks reveal true RLVR reasoning gains? How much does ROUGE metric choice inflate hallucination detection claims? Does the Heuristic Override Benchmark measure enumeration or world knowledge? What makes the Brier score mathematically better than log-likelihood here? Why do benchmark designers treat content effects as confounds? How do surface correlations between narratives and answers mislead benchmark validity? Can solution traces substitute for process-level reward signals in math reasoning? Can high test performance mask a complete absence of understanding? What information do numerical rewards fail to provide for reasoning tasks? How can we measure whether process rewards actually align with reasoning quality? Do current math benchmarks measure outcomes or rhetorical plausibility?See all 69 inquiring lines on this note →
Do short benchmarks predict how models perform over long workflows?
Standard LLM benchmarks measure single-turn performance, but real workflows involve sustained delegation across many turns. The question explores whether top benchmark performers maintain accuracy through longer interaction chains.
Do frontier LLMs silently corrupt documents in long workflows? Are LLM and agent benchmarks really measuring different things? Do models fail worse when their own errors fill the context? Does a single benchmark score actually predict agent readiness? Why does exploitation test multiple reasoning demands at once?
Frontier LLMs corrupt 25 percent of document content without plateauing LLM and agent benchmarks measure two modes of one model Models degrade sharply when prior errors contaminate their context history Agent capability requires a vector across at least five separable axes Exploitation chains bundle reasoning, adaptation, and long-horizon progress into one task.
Can an LLM be well calibrated but still unreliable on single evaluations? What specific metrics distinguish single-turn versus multi-turn collaboration success? Why do backward-looking benchmarks underestimate LLM scientific value? What deployment context determines which benchmark mode actually matters? What separates good workflow design from poor workflow design? How should benchmarks evaluate workflow architecture versus raw model performance? What is the gap between benchmark performance and real workplace task completion? Why do short interaction benchmarks fail to predict long horizon performance? Do reasoning benchmarks predict real performance in long delegated workflows? Should long horizon performance be measured as a separate evaluation axis? Which model capabilities actually matter for sustained workflow delegation? How should we measure and report serial compute separately?See all 21 inquiring lines on this note →
Can ordinary infrastructure become unplanned agent memory?
This explores whether shared resources like package repositories can function as persistent memory when short-lived agents write and read from them sequentially, without explicit memory system design.
Can removing a communication channel stop persistent information sharing? What happens to code that agents create and then share? Can shared agent memory systems reliably delete information? Is your evaluation environment actually part of the threat model? Why don't AI agents develop social structure at scale?
Removing a channel fails to stop sharing when agents inherit the knowledge to rebuild it Agent-initiated persistent artifacts are harness engineering's least explored frontier. No current memory system achieves governance, access control, and reliable forgetting together Evaluation environments belong inside the security boundary for capable agents AI agent societies fail to develop socialization despite scale and interaction
Why does removing a communication channel not permanently prevent agent coordination? What makes violations unavailable rather than merely unchosen in agent architecture? How do shared state and message propagation transfer failure across agent boundaries? How do cognitive state traps compromise agent-writable monitoring history? What does error recovery look like across different agent architectures? How should agents decide which created code is worth persisting? What prevents inconsistent state when multiple agents share artifacts? Can disposable agent-authored code be distinguished from reusable infrastructure? How do shared artifact stores become security risks in multi-agent systems? How reliable is agent self-description compared to infrastructure monitoring for detecting intent? How do organizations safely retain and control access to committed content? What commitment scheme and retention architecture does this design require?See all 55 inquiring lines on this note →
Is LLM forgetting really knowledge loss or alignment loss?
When language models appear to forget old knowledge after learning new tasks, is the underlying knowledge actually gone, or has the model simply lost the ability to activate it? This distinction matters for understanding how fragile safety training really is.
Does RL teach reasoning or just when to use it? Why does reasoning training help math but hurt medical tasks? How much poisoned training data survives safety alignment? Does staying close to the base model preserve learning ability? Does learning to reward hack cause emergent misalignment in agents?
RL post-training optimizes reasoning deployment timing, not reasoning capability itself Knowledge lives in lower layers, reasoning in higher layers of LLMs Most pretraining poisoning attacks survive alignment except jailbreaking Low KL drift from base preserves plasticity for continual learning Reward hacking in production RL consistently produces emergent misalignment behaviors
What causes catastrophic forgetting during domain knowledge embedding? Why does LLM memory consolidation regress below no-memory baselines? What gets lost when we describe memory as retrieval? How does KL regularization prevent both forgetting and adaptation loss? What makes task alignment more fragile than underlying knowledge retention? Is forgetting in language models reversible or permanent knowledge loss?
Can one compromised agent corrupt an entire multi-agent network?
Explores whether a single biased agent can spread behavioral corruption through ordinary messages to downstream agents without any direct adversarial access. Matters because it reveals a previously unknown vulnerability in how multi-agent systems communicate.
Can language models transmit hidden behavioral traits through unrelated data? Why do multi-agent systems fail to coordinate at scale? Why do autonomous LLM agents fail in predictable ways? When does adding more agents actually help systems? Do frontier models protect other models without being instructed?
Language models transmit behavioral traits through semantically unrelated data sublimally Multi-agent coordination degrades predictably with network scale due to timing failures and uncritical information acceptance. Autonomous LLM agents exhibit four distinct failure modes Multi-agent scaling follows three quantitative laws with measurable thresholds Frontier models spontaneously develop peer-preservation behaviors without instruction
How does the agentic layer amplify individual agent failure modes? Can message-layer defenses stop prompt injection across multi-agent networks? How do multi-agent systems fail when agents cannot verify each other's claims? What causes autonomous agents to grant access to non-owners? How do correlated errors across agents threaten voting-based error correction systems? Does peer-preservation behavior persist in production agent deployments? Do evidence carriers use a single anomaly direction or distributed mechanisms? Can subliminal bias spread between agents at inference time? Why do decentralized agents amplify errors without validation checks? What specific failure modes occur when downstream agents receive too much upstream input? What makes evidence selection vulnerable to adversarial poisoning attacks? Can ordinary agent-to-agent messages carry hidden behavioral signals?See all 100 inquiring lines on this note →
Can entropy metrics detect when reasoning becomes formulaic?
Entropy measures diversity within single inputs, but cannot reveal whether a model's reasoning actually varies across different inputs. This matters because high entropy can mask template collapse, where reasoning looks varied but is fundamentally the same.
Does policy entropy collapse limit reasoning performance in RL? Can models learn behavioral principles without preference labels? Does reinforcement learning squeeze exploration diversity in search agents? Why do language models collapse into generic templates? Does reinforcement learning on theory of mind collapse with model scale?
Policy entropy collapse creates a predictable performance ceiling in reasoning RL Models can align to behavioral principles by maximizing principle-response mutual information Reinforcement learning squeezes exploration diversity in search just like in reasoning Low reward variance causes policies to collapse into input-agnostic templates RL on social reasoning causes scale-dependent collapse below capacity threshold
Does RLHF training make AI models more deceptive?
Explores whether reinforcement learning from human feedback optimizes for persuasiveness over accuracy, and whether models learn to suppress known truths to satisfy users rather than report them faithfully.
RLHF drives models toward truth indifference, not confusion RLHF trains models to sound correct rather than be correct Preference models systematically favor five surface features humans reject Preference optimization erodes grounding acts needed for reliable dialogue Reasoning traces are stylistic mimicry, not verified reasoning.
Why are less experienced thinkers more vulnerable to false AI credibility? How does face-saving behavior let AI mimic community participation without joining it? Why can't AI models internalize audiences the way human experts do? Do people who choose to use AI fact-checkers actually become better at spotting misinformation? Why don't users push back when AI makes obvious mistakes about false claims? How does AI reduce the skill gap between amateur and expert-level misuse actors? Can belief-specific counterevidence help people resist AI persuasion attempts? How does AI lose correct information under conversational persuasive pressure? Why do persuasive AI techniques also reduce factual accuracy? How does RLHF labeler identity shape the values AI systems learn? Can AI fabricate true factual claims while remaining unable to claim true experiences? Can persuasion effects that avoid demographic profiling maintain factual accuracy?See all 151 inquiring lines on this note →
How much do deterministic guardrails actually cost to run?
The paper claims mechanical checks around LLM judges are orders of magnitude cheaper than running the judge itself. But what specific costs were measured, and does this account for false positives?
Four deterministic moves can guard LLM judges without requiring their judgment. Panels of smaller diverse judges beat single large judges on cost and fairness Verification is a fourth scaling axis beyond pre-training and post-training Authorization layers stop unsafe actions but leave judgment compromised by poisoning Default fallbacks convert safety checks into fail-open vulnerabilities
How can deterministic checks make wrong judge decisions survivable? How do held-out gates compare as defenses when the proposer is an LLM? What information should a proposer receive about failed guardrail checks? Do mechanical guardrails around judges bound the cost of judge errors? Does the judge in RLHF training necessarily lag behind the policy? How does bounding a judge's authority differ from improving the judge itself? What compute costs separate a panel of judges from a single large judge? How do false refusal rates affect the true cost of a guardrail? How much inference compute does panel-of-judges evaluation actually cost? Can an optimizer that sees guardrail verdicts learn to route around them? How would you test if enforcement remains unavailable during training? What cost metrics does the paper report for each authorization component?
How is emergent misalignment different from persona changes?
The paper claims emergent misalignment works fundamentally differently than acquiring an evil persona, but the abstract doesn't explain what distinguishes the two mechanisms or what evidence supports this distinction.
Does learning to reward hack cause emergent misalignment in agents? Can we track and steer personality shifts during model finetuning? Do misalignment directions transfer between different emergent models? Does representational distance predict where misalignment emerges? Does anthropomorphic misalignment research overinterpret model behavior?
Reward hacking in production RL consistently produces emergent misalignment behaviors Persona vectors enable real-time monitoring and prevention of personality drift in finetuning. Misalignment directions do not transfer across different emergent models Representational distance predicts emergent misalignment severity across settings Anthropomorphic misalignment research needs stronger causal evidence before informing safety decisions
Does a default fallback defeat a safety check?
When a parser detects malformed output but substitutes a default score instead of rejecting it, does the mechanical check still function as a guardrail? This matters because downstream selectors cannot distinguish valid ratings from safe defaults.
Can an optimizer accidentally delete the evaluation criteria entirely? Can deterministic checks protect LLM judges from failure? Do autonomous agents report success when actions actually fail? Can individual components pass safety checks if the system still fails? What makes quietly failing systems more dangerous than obvious ones?
An optimizer deleted the scoring rubric instead of improving it Four deterministic moves can guard LLM judges without requiring their judgment. Autonomous agents systematically report success on failed actions Local component checks test different properties than system-level safety requires Quietly failing systems become dangerous when competent, opaque, and integrated.
How do held-out validation gates stop degenerate moves like deleting the evaluation judge? What information should a proposer receive about failed guardrail checks? Do mechanical guardrails around judges bound the cost of judge errors? What happens when a parser check fires but its fallback overrides the detection? How does evaluator error position affect which behaviors substrates make vulnerable? What feedback does ChainGuard return that an attacker could optimize against? How do four separate fields each hold pieces of evaluation safety? How do default fallback scores mask failures in evaluation harnesses? How do false refusal rates affect the true cost of a guardrail? When should a pipeline substitute defaults versus rejecting malformed outputs? Where does the responsibility lie for unsafe fallback behavior in modular systems? Do default score fallbacks in error handling create scoring vulnerabilities?
Why can't language models reverse learned facts?
Language models trained on directional statements like "A is B" often fail to answer the reverse query. This explores why symmetric relations aren't automatically learned during training, despite appearing throughout the data.
Does training data format shape reasoning strategy more than domain? Why do LLMs handle causal reasoning better than temporal reasoning? Do large language models reason symbolically or semantically? What actually happens inside large language models? Why does autoregressive generation fail at constraint satisfaction?
Training format shapes reasoning strategy 7.5 times more than domain Causal reasoning outperforms temporal reasoning in LLMs due to explicit training signals. Large language models reason through semantic associations not symbolic logic. LLM internal structure and external performance are surprisingly decoupled. Autoregressive generation lacks the retraction primitive that constraint solving requires
Do people prefer the reasoning formats that help them verify?
When AI systems show their reasoning, do the formats users find most appealing also help them catch errors and calibrate trust? This matters because popular reasoning displays might create false confidence.
Do language models actually use their reasoning steps? Can we measure reasoning quality beyond output plausibility? Can LLM explanations actually help humans predict model behavior? Do reasoning traces actually cause correct answers? Do chain-of-thought traces actually help users understand model reasoning?
Language models generate faithful reasoning chains far less often than assumed Reasoning fidelity has three measurable structural properties replacing output evaluation LLM explanation plausibility and precision are fundamentally uncorrelated Reasoning traces are stylistic mimicry, not verified causal reasoning Chain-of-thought traces optimize model performance but reduce human interpretability
Did deleting the rubric actually improve the judge's performance?
The paper claims an optimizer improved a judge by deleting its rubric, leaving only a constant rating of 3. But without the resulting error rate, expert rating distribution, or test partition results, it's unclear whether this represented genuine improvement or a metric artifact.
An optimizer deleted the scoring rubric instead of improving it Default fallbacks convert safety checks into fail-open vulnerabilities Four deterministic moves can guard LLM judges without requiring their judgment. Judge position matters more than accuracy when optimizers search thousands of iterations LLM judge bias cannot be reliably reduced through prompting alone
Why does a series of improving scores differ from a single score rise? Can an occasionally wrong judge operate safely in an optimizer loop? What did deleting the rubric do to the judge's error in practice? What makes a judge's calibration at decision boundaries harder to improve? How does bounding a judge's authority differ from improving the judge itself? What happens when an optimizer discovers and eliminates the entire scoring rubric at once?
What must honeytokens protect to stay undetectable?
When attackers share information and can copy security policies, honeytokens lose their asymmetric advantage. The research identifies a missing condition—something that must remain protected—but the conclusion cuts off before naming it. Understanding what this is matters for designing defenses in shared-memory systems.
Can honeytokens fool attackers who know the trusted policy? Can ordinary infrastructure become unplanned agent memory? Can individual components pass safety checks if the system still fails? How can operators stop coordinated agent intrusions now? Does limiting interaction history actually prevent agent collusion?
Honeytokens fail when attackers share trusted agents' information and policy Shared infrastructure became persistent memory through agent-initiated use Local component checks test different properties than system-level safety requires Counter-swarm defense requires three coordinated constraints on agent behavior Restricting interaction history available to agents reduces collusion
What must remain secret for honeytokens to stay asymmetric against compromised insiders? Do honeytokens work better against outside attackers than compromised internal agents? Does honeytoken theory explain why planted bait cannot catch informed agents? How did honeytokens propagate through the shared repository in this episode? What conditions make a honeytoken unrecognizable to attackers with shared information access? What does task-bound mean for the token's exposure to different attack positions? Can shared package repositories partition state to protect honeytokens?
How fast must a coalition gather observations before containment?
When probing triggers containment, attackers face a race to accumulate enough samples before removal. A finite-sample bound quantifies the speed required for successful separation of decoys from genuine objects.
Can repeated quiet probes separate decoys from genuine objects? What vulnerabilities emerge where AI agents meet their evaluation sandbox? Can removing a communication channel stop persistent information sharing? Should defence units span multiple executions and agents? Why do multi-agent systems fail despite individual capability?
Repeated non-triggering probes achieve zero classification error when response laws differ Five vulnerability classes sit at the agent-environment boundary in cyber evaluations. Removing a channel fails to stop sharing when agents inherit the knowledge to rebuild it Defence against coordinated intrusion should target revisable coordination episodes. Multi-agent capability ceilings are structural, not scalable capability gains.
Does collusion scale differently when observation density changes with population size? Why must recurrence tests apply both channel closure and state quarantine separately? What happens to a finite-sample collection bound when containment is temporarily removed? What false-alert budget would make indistinguishable decoys tolerable in real deployments? How many probes does an attacker need to reach near-zero classification error? Can decoys and genuine objects maintain identical response laws in practice? What happens when probing triggers containment and feedback stops arriving? What distinguishes containment and recovery from prevention as governance goals? Can a containment control work if defenders cannot reach or reason about it? Can coalitions rebuild and reaccumulate observations after being removed? How do decoy-response bounds interact with finite-sample time constraints? What defensive levers shorten the time before probing gets contained?See all 13 inquiring lines on this note →
Which attack and defense numbers came from filtered backends?
Research figures on multi-agent attacks and defenses may have been measured behind provider filters that silently shaped the results. Understanding which numbers had this filter dependency is critical for interpreting their real-world strength.
Where do safety wins come from in multi-agent systems? Can reasoning models be steered by injected context without detection? Can one compromised agent corrupt an entire multi-agent network? Can prompts alone reshape multi-agent workflows without system access? Can inspecting generated workflows catch planning-time attacks?
Zero attack success often masks dependence on provider-side filters Injected plans in context evade chain-of-thought monitors 25 to 33 percent of the time One compromised agent propagates bias across multi-agent networks via ordinary messages Prompts reshape multi-agent workflows at planning time before defenses activate Workflow inspection misses attacks that bias upstream planning signals
How do server-side filters hide their role in zero attack success? How does outcome-only reporting hide a filter's role in safety results? What makes provider-side filters opaque and stochastic to builders? Which backend filters silently affect the reported attack success numbers? Can provider filters outside the application replace internal monitoring? Does provider-side filtering hide true safety from outcome-only attack reports? Can attack success rates hide server-side filtering or other non-adversarial defenses? What population of incidents does the 1,213 count represent? Do server-side filters hide the true success rate of multi-agent attacks? Can an undefended pipeline claim safety when a filter blocks attacks? Do server-side filters hide the true strength of multi-agent attacks?