Can cross-view learning align semantic, entity, and item representations of the same user?
This explores whether you can train a model to reconcile the several different 'pictures' of the same user — their meaning-level profile, the entities they touch, and the specific items they pick — into one aligned representation, and what the corpus says about bridging representation types in general.
This reads the question as being about representational alignment: a single user shows up in a system as a semantic profile (what they seem to like, in words), as entities (the things they're connected to), and as concrete item interactions — and 'cross-view learning' would mean training these views to agree. The honest headline is that the corpus doesn't contain a paper that does exactly this multi-view alignment under that name. But it's dense with the building blocks and the tensions you'd have to resolve to attempt it, so the more useful answer is to show you where the seams are.
The first thing the corpus pushes back on is the premise that a user even *has* one stable representation to align toward. Two notes argue that a user is better modeled as several latent personas, not a single taste vector, with attention deciding which persona is relevant to the item in front of them Can attention mechanisms reveal which user taste explains each recommendation? Can modeling multiple user personas improve recommendation accuracy?. That reframes your question: maybe the goal isn't collapsing semantic/entity/item views into one point, but learning *when* each view governs a prediction — which is closer to mixture-of-views than to forced agreement.
The corpus is most concrete on the bridge between the semantic (text/meaning) view and the item view, which is the hardest seam in your trio. VQ-Rec deliberately *breaks* the tight coupling between an item's text and its recommendation embedding by routing text through discrete codes, precisely because over-aligning to text similarity introduces bias and hurts transfer Can discretizing text embeddings improve recommendation transfer? Can discrete codes transfer better than text embeddings?. That's a warning shot for any naive alignment scheme: tighter alignment between semantic and item views can make the system *worse*, not better. The opposite bet appears in P5, which unifies every recommendation signal into one text-to-text space so a single encoder handles all task families Can one text encoder unify all recommendation tasks? — unification as composability, at a cost in efficiency. So the corpus actually hands you both poles: deliberately decouple (VQ-Rec) vs. deliberately unify (P5).
The semantic view itself turns out to be contested. PRIME finds that an *abstracted* semantic memory of a user — distilled preference summaries — beats replaying their specific past interactions Does abstract preference knowledge outperform specific interaction recall?, which suggests the 'semantic' and 'item-history' views aren't just different encodings of the same truth; they carry different and sometimes competing information. And there's a deeper caution underneath all of this: the user signals you'd align on may not be one clean thing. Annotation responses decompose into genuine preferences, non-attitudes, and constructed-on-the-spot answers, and treating them uniformly contaminates everything downstream Do all annotation responses measure the same underlying thing?. If the raw signal feeding each view is itself a mixture, forcing the views into agreement can be aligning on noise.
The thing you might not have known you wanted to know: the corpus's center of gravity is that 'align all the views of a user' is often the wrong objective. The most successful patterns here either keep views deliberately separate and learn to route between them (personas, decoupled codes) or pick one view that abstracts well (semantic memory) — and the one big unify-everything bet, P5, buys composability by paying in efficiency rather than by claiming the views were secretly the same all along.
Sources 7 notes
AMP-CF represents each user as multiple latent personas weighted dynamically by candidate item. This makes recommendations both diverse and interpretable—each suggestion traces to the specific persona preference it satisfies—without requiring post-hoc reranking.
AMP-CF separates user representation into latent personas weighted by attention to the candidate item. This candidate-conditional approach improves accuracy by adapting the user representation at prediction time and produces inherent explanations for why items were recommended.
VQ-Rec uses product quantization to map item text to discrete codes that index learned embeddings, breaking the tight coupling between text and recommendations. This decoupling prevents text-similarity bias and allows lookup tables to adapt to new domains without retraining the text encoder.
VQ-Rec demonstrates that mapping item text to discrete codes via product quantization, then to embeddings, improves cross-domain transfer compared to direct text encoding. The discrete intermediate reduces text bias and enables efficient per-domain fine-tuning.
P5 converts user-item interactions and metadata into natural language and trains a single encoder-decoder across five recommendation task families, matching task-specific models while achieving zero-shot transfer to new items and domains. Unification trades efficiency for composability.
PRIME framework shows semantic memory (preference summaries, parametric encodings) consistently beats episodic memory (retrieved past interactions) across models. Recency-based recall outperforms similarity-based retrieval, and task fine-tuning exceeds preference tuning methods.
Behavioral science reveals that annotations contain genuine preferences, non-attitudes, and constructed preferences—distinguishable by consistency across measurement conditions. Treating them uniformly contaminates reward model training and downstream alignment.