How much of prompt sensitivity is really just frequency optimization in disguise?

This explores whether prompt sensitivity — a model giving different answers to reworded versions of the same request — is mostly just a hunt for phrasings that match what the model saw often during training. The corpus gives this suspicion real support, but also complicates it. The strongest backing comes from work showing that prompt optimization can only activate knowledge already present and cannot inject anything new Can prompt optimization teach models knowledge they lack?. If prompting only reorganizes what's already in the training distribution, then a 'good' prompt is partly one that lands in a well-trodden, high-frequency region — which is exactly what the frequency-optimization framing predicts.

But the corpus reframes the driver as confidence rather than raw frequency. ProSA found that prompt sensitivity is a reflection of how confident the model is: highly confident models shrug off rephrasing, while low-confidence ones swing hard Does model confidence predict robustness to prompt changes?. Frequency and confidence are cousins — a model is confident where its training was dense — so this is consistent with the disguise hypothesis, but it relocates the cause from 'the prompt's wording' to 'the model's internal certainty about that region.' Notably, larger models, few-shot examples, and objective tasks all raise confidence and reduce sensitivity, which means the same wording stops mattering as the model's grip on the territory firms up.

The 'in disguise' part starts to break down once you look at what actually varies. Prompt effectiveness depends sharply on model tier — rephrasing and background-knowledge prompts lift cheap models, while step-by-step reasoning actively hurts strong ones — and the deciding factor is task structure, not generic best practices Do prompt techniques work the same across all LLM tiers?. That's hard to explain as pure frequency-matching: if it were just frequency, the same 'frequent' phrasings would help everywhere. Similarly, prompt quality turns out to have measurable, structured dimensions grounded in communication theory — clarity, logic, instruction, hallucination control — that improve outcomes independent of the model's outputs Can we measure prompt quality independent of model outputs?. There's genuine signal in *how you communicate*, not only in *which tokens are common*.

The most telling counter-evidence is that sensitivity can be trained or engineered away entirely. Consistency training teaches a model to respond identically to clean and perturbed prompts, using its own clean answers as targets Can models learn to ignore irrelevant prompt changes? — meaning the sensitivity was a removable artifact, not a law of how prompts work. And architecture-level methods like personality adapters bypass prompt phrasing altogether, writing the desired behavior directly into every layer Can we control personality in language models without prompting?. If you can dial behavior in without touching the prompt, then the prompt was never the fundamental control surface.

So the honest answer: a large share of prompt sensitivity *is* frequency optimization in disguise — you're often paying to find phrasings that match dense, confident regions of the model's distribution, and you cannot prompt past the ceiling of what was trained in Can prompt optimization teach models knowledge they lack?. But not all of it. The residue that survives — task-structure dependence, measurable communication quality, and the fact that sensitivity can be trained out or sidestepped at the weights — is real signal about reasoning and communication, not just frequency. The deeper surprise hiding here: there's also a theoretical ceiling on the *other* side, since the right prompt can in principle make a single transformer compute anything Can a single transformer become universally programmable through prompts?, yet ordinary training almost never produces models that exploit that power — so prompt sensitivity sits in the gap between what prompts could theoretically do and what frequency-shaped training actually taught the model to do.