Can mention sequences exploit shortcuts like repeated items rather than learning genuine preferences?

This explores whether sequence-modeling of mentioned items risks shortcut learning: counting repetitions or recency rather than recovering genuine taste. The corpus doesn't frame it as a named failure mode, but several notes circle the exact tension. The most direct is the work showing that items mentioned in a conversation form ordered sequences with prequel/sequel dependencies Does conversation order matter for recommending items in dialogue?. The interesting move there is *against* the shortcut: a 'bag-of-mentions' approach — which is essentially repetition-counting — throws away the order, and a transformer that models the actual sequence recovers signal the bag discards. So the field's answer is partly that repetition is the impoverished baseline, and learning dependencies between mentions is what gets you past it.

But that raises the deeper question the question is really pointing at: is *any* surface statistic enough? Here the strongest cautionary tale comes from outside recommendation entirely. Research on behavioral traits propagating between models through semantically unrelated data shows that learning systems will happily latch onto statistical signatures that carry no real meaning at all Can language models transmit hidden behavioral traits through unrelated data? — the effect rides on co-occurrence fingerprints, not content. That's the purest illustration of the shortcut hazard: a model can encode a 'signal' that looks predictive while being entirely disconnected from the thing you wanted it to learn. Mention frequency is exactly the kind of feature that could play this role.

The corpus's implicit defense against shortcuts is to push representations *up* a level of abstraction. Work comparing how systems remember users finds that abstracted preference summaries consistently beat raw recall of past interactions Does abstract preference knowledge outperform specific interaction recall? — and notably, recency-based recall beats similarity-based retrieval, suggesting that even 'smarter' lookup over raw history underperforms a genuine summary of what the person wants. If repeated mentions are episodic noise, the fix is to compress them into a semantic statement of preference rather than tally them. The critique-to-preference work makes the same maneuver: it transforms a surface utterance ('doesn't look good for a date') into a stated positive preference ('prefer more romantic') Can language models bridge the gap between critique and preference?, converting a raw signal into an interpretable preference rather than a frequency.

There's also an architectural angle on why naive counting fails as a *training objective*. The note on likelihood choice in collaborative filtering shows that a multinomial likelihood — which forces items to compete for a shared probability budget — outperforms alternatives precisely because competition aligns training with ranking Why does multinomial likelihood work better for ranking recommendations?. Repetition without competition is the degenerate case: an item that appears often can inflate its own score without ever beating its rivals. And the conversational-policy work argues that splitting decisions apart (what to ask, what to recommend, when) lets each component optimize a local proxy that doesn't serve the whole trajectory Can unified policy learning improve conversational recommender systems? — a structural cousin of shortcut-taking, where a piece optimizes the easy local signal instead of the real goal.

The thing you might not have expected to find: the corpus's consistent answer to 'can models cheat with repetition?' is *yes, unless you make cheating unprofitable.* Order modeling, semantic abstraction, item competition, and unified objectives are all ways of structurally denying the model the easy shortcut — not because researchers caught a model red-handed counting repeats, but because they keep finding that the abstracted, competitive, order-aware version wins.