The earlier parts walked you through the complete industrial chain of discriminative recommendation — retrieval, ranking, re-ranking. But a real recommender system is far more than "ordering candidates well". It must also confront biased data, items and users with no history, and an ongoing paradigm shift: from "scoring every candidate" to "directly generating recommendation sequences".
This part does not add another standalone algorithm module. Instead, it pulls the camera back so you can see the three approaches industry and academia take when correcting, completing, and reshaping recommender systems.
What This Part Covers
| Section | Topic | The Big Idea |
|---|---|---|
| 5.1 | Model debiasing | Data is naturally biased and sits inside a feedback loop; correct it with IPS reweighting and PAL's position decoupling |
| 5.2 | Cold start | Use content, meta-learning, and segmentation architectures to build effective representations for items and users with no history |
| 5.3 | Evolution of the generative paradigm | From discriminative scoring to generative sequence generation — the leap from memorization · generalization to understanding · reasoning |
What You'll Be Able to Do After This Part
- 🟢 Explain where a recommender's data biases come from (selection/exposure/conformity/position) and how result biases (popularity/unfairness) get amplified through the feedback loop
- 🟢 Apply inverse propensity score (IPS) reweighting, and use PAL to decouple position effects from user preference structurally
- 🟡 Compare the different solutions for content cold start (CB2CF / MetaEmbedding) and user cold start (MeLU / POSO)
- 🟡 Describe the three evolutionary paths of generative recommendation: generative retrieval (HSTU / TIGER), generative ranking (GenRank / MTGR), and end-to-end unified generation (OneRec)
- 🔴 Argue, against Part 1's two paradigms, how the discriminative cascaded architecture can be replaced by a generative end-to-end one
Core Concepts
| Concept | Section | Relevance |
|---|---|---|
| Selection bias / exposure bias / position bias | 5.1 | The root of untrustworthy training data; requires active correction |
| Inverse Propensity Score (IPS) | 5.1 | Weighting by for an unbiased risk estimate |
| Position-bias Aware Learning (PAL) | 5.1 | Structurally separating "seeing" from "liking" |
| Content cold start: CB2CF / MetaEmbedding | 5.2 | Letting new items borrow collaborative representations from content |
| User cold start: MeLU / POSO | 5.2 | Serving new users with meta-learning or segmented submodules |
| Semantic IDs / end-to-end generation | 5.3 | The core of generative recommendation: understand content, generate directly |
Prerequisites
- Finish the two paradigms and the three-stage funnel of Part 1 first
- Know the basic models of Part 2 Retrieval and Part 3 Ranking (collaborative filtering, two-tower, matrix factorization)
- Have basic deep-learning and sequence-model concepts (Transformer self-attention; meta-learning/MAML will help with 5.2/5.3)
This part is the bridge that closes out the fundamentals — it patches the defects of the pipeline from earlier parts while pointing toward the generative follow-up volume.
Tips for This Part
- Treat bias as an invisible opponent. While reading 5.1, keep asking: does this training sample really represent the user's true preference?
- The core of cold start is borrowing strength. A new item has no behavior, so borrow content; a new user has no history, so borrow meta-knowledge or population structure.
- 5.3 loops back to Part 1. It grounds "discriminative vs generative" in concrete models (HSTU, TIGER, OneRec).
Let's dive in! 🚀