Can discrete codes replace text-only item representations in recommenders?
This explores whether mapping item descriptions into discrete codes (instead of feeding raw text or text embeddings straight into a recommender) actually buys you something — and what you lose by giving up text-only representations.
This explores whether discrete codes can stand in for text-only item representations in recommenders, and the corpus suggests the answer is a qualified yes — not because codes are richer than text, but because the *decoupling* they create solves problems that pure text causes. The clearest case is VQ-Rec, which uses product quantization to turn item text into discrete codes that then index a learned embedding table Can discretizing text embeddings improve recommendation transfer?. The insight is counterintuitive: text embeddings carry a hidden bias, because two items with similar descriptions get pushed together whether or not users actually treat them as similar. Inserting a discrete bottleneck breaks that tight coupling, so the recommender learns from behavior rather than inheriting the text encoder's notion of similarity. That same decoupling is what makes codes *transfer* across domains better than raw text embeddings — you can refit the lookup table for a new domain without retraining the encoder Can discrete codes transfer better than text embeddings?.
But 'replace' is the wrong frame, and the corpus pushes back on it from a different angle. TransRec argues that neither pure IDs nor pure text alone gets you everything: IDs give you distinctiveness, text gives you semantics, and you need structural grounding for a model to actually generate valid item references Can item identifiers balance uniqueness and semantic meaning?. Read alongside VQ-Rec, the lesson is that the winning move is usually a *hybrid* representation — discrete codes don't win by exiling text, they win by sitting between text and the recommender so each does the job it's good at.
There's also a quieter, practical reason discrete codes matter that has nothing to do with semantics. Recommenders have historically leaned on hashed ID tables, and those break in an ugly way: real catalogs follow power-law frequencies, so hash collisions pile up exactly on the high-traffic users and items you most need to get right Why do hash collisions hurt recommendation models so much?. A learned discrete codebook is a more principled version of the same idea — a compact code space — but one where the codes are derived from content rather than assigned by an arbitrary hash, which is part of why the VQ-Rec style holds up where naive hashing degrades.
It's worth seeing what the alternative camp does, because it sharpens the trade-off. P5 goes the opposite direction entirely: render every interaction as natural language and let one text-to-text model handle all recommendation tasks, trading raw efficiency for composability and zero-shot reach to new items Can one text encoder unify all recommendation tasks?. So the field is genuinely split — text-everywhere for flexibility versus discrete-codes for transfer, debiasing, and efficiency. Discrete codes can replace text-*only* representations, then, in the specific sense that they remove text from the recommender's inner loop while keeping it as the source the codes are built from. What you'd be trading away — the open-ended, generate-anything quality of a language interface — is exactly what the text-native approaches are chasing.
Sources 5 notes
VQ-Rec uses product quantization to map item text to discrete codes that index learned embeddings, breaking the tight coupling between text and recommendations. This decoupling prevents text-similarity bias and allows lookup tables to adapt to new domains without retraining the text encoder.
VQ-Rec demonstrates that mapping item text to discrete codes via product quantization, then to embeddings, improves cross-domain transfer compared to direct text encoding. The discrete intermediate reduces text bias and enables efficient per-domain fine-tuning.
TransRec shows that combining numeric IDs, titles, and attributes into structured identifiers solves three problems simultaneously: distinctiveness from IDs, semantics from text, and generation grounding from structural constraints. Neither pure IDs nor pure text alone achieves all three.
Monolith's empirical work shows that real recommendation systems have power-law distributed frequencies, causing collisions to accumulate precisely on the entities models need most accurate. Fixed-size hashed tables worsen this over time as new IDs arrive.
P5 converts user-item interactions and metadata into natural language and trains a single encoder-decoder across five recommendation task families, matching task-specific models while achieving zero-shot transfer to new items and domains. Unification trades efficiency for composability.