Cold Start

📝 Before You Continue: Make sure you have read collaborative filtering and the two-tower model in Part 2 Retrieval, and the bias perspective in 5.1. Cold start is, at its core, a bias predicament: being asked to be accurate with no history.

A recommender system's most awkward moment is when a new item goes on shelf, or a new user signs up. Collaborative filtering learns preferences from user–item interactions, but at that moment the interactions are zero; content-based methods can handle new items, but often capture only surface similarity.

This is the cold-start problem — the system's core engine (behavioral data) has not fired up yet, but it must immediately output trustworthy recommendations. This chapter splits cold start into two faces: content cold start (new items lack interactions) and user cold start (new users lack history), with two representative solutions for each. Their shared wisdom is borrowing strength: a new item borrows from content, a new user borrows from meta-knowledge or population structure.

After reading this chapter, you will be able to:

• Distinguish the fundamental difference between content cold start and user cold start
• Explain how CB2CF maps content features to collaborative-filtering representations so new items get CF quality directly
• Write down MetaEmbedding's two-stage meta-loss and understand that it optimizes "learnability" rather than a fixed vector
• Explain MeLU's parameter separation and POSO's "personalization submergence" insight, and compare the two
• Work through 4 tiered practice problems to consolidate the engineering and math intuition for cold start

5.2.0 The Two Faces of Cold Start

Cold start is not one problem but two objects each "lacking history":

We take each in turn.

5.2.1 Content Cold Start: Letting New Items "Borrow" Collaborative Quality

Collaborative filtering uncovers complex implicit associations but is helpless with new items; content-based methods handle new items but often capture only surface similarity. The ideal is: new items also get collaborative-filtering-grade representations — exactly the goal of CB2CF and MetaEmbedding.

CB2CF: From Content Features to Collaborative Representations

The core idea of CB2CF (Content-Based to Collaborative Filtering) is to learn a mapping function $f:\mathcal{C}\to \mathcal{V}$ that maps an item's content features $c_i$ directly into the collaborative-filtering embedding space, yielding ${\hat{v}}_i=f(c_i)$.

For items that have both a content description and rich interactions, we hold both their content vector and their CF embedding. CB2CF uses a deep network to learn the nonlinear mapping between the two representations, so a new item obtains a semantically consistent CF representation from content alone. Its multi-view architecture has three modules:

• Content Encoder: encodes multimodal content (text, images, categories) into a unified content vector. CNNs for images, RNN/Transformer for text.
• Mapping Network: the core — stacked fully-connected layers that learn the nonlinear map from content space to CF embedding space, capturing complex content–preference associations.
• Constraint Optimization module: uses a cosine-similarity constraint to keep the mapped representation semantically consistent with the true CF embedding, guaranteeing the mapping is valid.

Where do collaborative vectors come from? For items with interactions, CF vectors can be produced in several ways: matrix factorization $R\approx U{V}^T$, where item $i$'s vector is row $i$ of $V$, $v_i$; the item-tower output of a two-tower retrieval model; or deep methods like NCF and autoencoders. Once CB2CF has learned $f$, a new item's content $c_i$ passes through $f$ to yield ${\hat{v}}_i$.

🧠 Mental Model: The Translator

Think of CB2CF as a translator. CF embeddings are the system's internal lingua franca; established items all speak it. A new item only speaks "content-ese" (text/images). The translator $f$ has learned to render content-ese into CF-ese, so even though the new item has never made friends (no interactions), the system understands it the moment it speaks and folds it into the collaborative network.

Analysis: CB2CF's strength is directness — one mapping and a new item instantly holds a CF-grade representation that plugs into existing retrieval/ranking. Its limits: the mapping's quality ceiling is bounded by how transferable "content → CF" is; if content correlates weakly with collaborative signal, the translation distorts. It also assumes existing items' CF vectors are trustworthy (you need a good CF model first).

MetaEmbedding: Meta-Learning "Smart" Initial Embeddings

CB2CF solves "new items can't get a CF representation", but another difficulty remains: even with an initial vector, traditional random initialization makes new items perform poorly early on and need lots of interactions to converge.

MetaEmbedding applies meta-learning to generate embeddings for new items that are both initially high-quality and quick to adapt. It optimizes the generator by simulating each item's full journey "from cold start to warmed up".

Algorithm inputs: a pretrained base model $f_{\theta }$, an item set $\mathcal{I}$, meta-loss weight $\alpha$, and step sizes $a,b$. For each sampled item $i$:

Initial embedding generation stage: the generator produces the initial vector

$${\varphi }_{[i]}^{\text{init}}=h_w({\mathbf{u}}_{[i]})$$

where ${\mathbf{u}}_{[i]}$ is item $i$'s features and $h_w$ is the generator with parameters $w$. Then sample two batches of $K$ samples each: ${\mathcal{D}}_{[i]}^a$ and ${\mathcal{D}}_{[i]}^b$.

Gradient adaptation and evaluation stage: compute the loss on the first batch and take one gradient-adaptation step, simulating "after a few interactions":

$${\varphi }_{[i]}^{\prime }={\varphi }_{[i]}^{\text{init}}-a\cdot \frac{\partial l_a({\varphi }_{[i]}^{\text{init}})}{\partial {\varphi }_{[i]}^{\text{init}}}$$

Then evaluate the adapted loss $l_b({\varphi }_{[i]}^{\prime })$ on the second batch.

The key is the meta-loss balancing two objectives:

$$l_{\text{meta},i}=\alpha l_a({\varphi }_{[i]}^{\text{init}})+(1-\alpha )l_b({\varphi }_{[i]}^{\prime })$$

Finally, update the generator with the meta-losses of all sampled items:

$$w\leftarrow w-b\sum _i\frac{\partial l_{\text{meta},i}}{\partial w}$$

💡 Key Insight: MetaEmbedding optimizes an embedding's "learnability", not the embedding itself. By repeatedly rehearsing "initialize → adapt → evaluate" on established items, it learns to give new items a "smart starting point" — one that converges to a high-quality representation after only a few real interactions.

🧠 Mental Model: Teaching "How to Learn" Instead of "Memorizing Answers"

MetaEmbedding is like a coach who doesn't hand a rookie the match answers, but trains him in "how to warm up before going on court and how to adjust through the first few plays". When the real match comes, he hits his stride after just a few real exchanges. $\alpha$ weighs "is the opening stance good" against "how strong is he after fine-tuning".

5.2.2 User Cold Start: Fast Personalization for New Users

A newly registered user has no interaction history, so collaborative filtering can only serve generic popularity-based recommendations. User cold start focuses on: how to capture personalized preferences quickly from a few behaviors. MeLU and POSO offer two approaches — meta-learning and segmentation architecture.

MeLU: Learning Each User as a Separate Task

MeLU (Meta-Learned User preference estimator) treats each user's preference learning as an independent task, and uses MAML (Model-Agnostic Meta-Learning) to train a model that adapts quickly to new users. MAML's essence is "learning how to learn" — rather than being optimal on one task, it learns a good initialization such that a few samples suffice to adapt to a new task.

MeLU uses two tiers of parameters:

• ${\theta }_1$ governs the embedding parameters for users and items (shared by all users)
• ${\theta }_2$ holds the parameters of the model's core decision network (adapts quickly to each individual)

Training strictly follows MAML's two loops:

1. Inner-loop adaptation: for each user $i$, compute gradients from their interaction history and update locally: ${\theta }_2^i\leftarrow {\theta }_2^i-\alpha {\nabla }_{{\theta }_2^i}{\mathcal{L}}_i^{\prime }(f_{{\theta }_1,{\theta }_2^i})$.
2. Outer-loop meta-update: using all users' adapted parameters, update both sets of global parameters simultaneously:

$${\theta }_1\leftarrow {\theta }_1-\beta \sum _{i\in B}{\nabla }_{{\theta }_1}{\mathcal{L}}_i^{\prime }(f_{{\theta }_1,{\theta }_2^i}),{\theta }_2\leftarrow {\theta }_2-\beta \sum _{i\in B}{\nabla }_{{\theta }_2}{\mathcal{L}}_i^{\prime }(f_{{\theta }_1,{\theta }_2^i})$$

MeLU's innovation is parameter separation: ${\theta }_1$ learns shared general representations while ${\theta }_2$ specializes in fast per-user adaptation. This retains representation capacity while personalizing quickly for new users. MeLU also proposes an evidence candidate selection strategy that picks the set of items most discriminative of user preferences for cold-start evaluation.

Analysis: MeLU's advantage is theoretical elegance — a new user gets personalized after a few gradient steps, no retraining from scratch. The costs: MAML's second-order gradients are computationally heavy, and it relies on the assumption that users' tasks are identically distributed; when new and old users' behavior distributions differ hugely, fast adaptation alone may not suffice.

POSO: Fighting "Personalization Submergence" with Segmented Submodules

POSO (Personalized cOld Start Modules) attacks from the architecture angle with a sharper insight: the root cause of user cold start is not just data scarcity, but the huge distributional gap between new and old users' behavior, plus the model's "submergence" when facing imbalanced distributions — when new users are far outnumbered by old ones, even with an "is new user" feature, training is dominated by the old-user majority. The model learns to ignore this heavily imbalanced feature, and the new users' personalization signal drowns.

POSO embeds into many module types; take the MLP as an example. The original MLP shares weights across all users, $y=\sigma (Wx+b)$; POSO introduces $K$ parallel submodules $f_i(x)=\sigma (W_{i}x+b_i)$, plus a personalized gating network (taking $x^{pc}$ such as is_new_user and activity level) that outputs weights $g_i=\text{softmax}({\text{MLP}}_{gate}(x^{pc}){)}_i$. The final output is the weighted combination:

$$\hat{y}=\sum _{i=1}^K{g}_i(x^{pc})\cdot f_i(x)$$

New users then rely mainly on "the submodule optimized for them" while old users use another set, effectively avoiding feature submergence. The idea extends to:

• POSO-MHA: extends to $K$ groups of attention heads, each with dedicated $Q/K/V$ transforms, concatenated and aggregated within each group; gating selects group weights by user features.
• POSO-MMoE: $E$ shared experts at the bottom + $K$ expert groups at the top ($M$ experts per group), stacking task gating and personalized gating for dual personalization at both the task level and the user-segment level.

🧠 Mental Model: Multiple Service Desks vs a Single Clerk

An ordinary model is like a single clerk serving all customers at once: biased toward the regulars' (old users') habitual requests, with newcomers' (new users') special needs drowned out. POSO is like opening $K$ dedicated desks: newcomers go to the "newcomer desk", regulars to the "regulars desk", and a greeter at the door (the gate) routes customers by type — newcomers' needs can never be shouted down by the regulars' volume.

Analysis: POSO and MeLU are complementary. MeLU assumes "all users are identically distributed; rely on fast adaptation" and suits scenarios with similar behavior patterns; POSO directly targets "imbalanced distributions causing feature submergence" and forces the split structurally — easier to integrate into off-the-shelf deep modules (MLP/MHA/MMoE) and free of meta-learning's heavy gradients. In practice they combine: use POSO's structure to keep cold start from being submerged, and meta-learning to further accelerate convergence.

⚠️ Common Mistakes in 5.2

Chapter Summary

📌 Key Takeaways

❓ FAQ

Q1: Do CB2CF and MetaEmbedding solve the same problem?

A: Not quite. CB2CF solves "new items cannot obtain a CF representation"; MetaEmbedding solves "even with an initial vector, random initialization converges slowly". They can chain: MetaEmbedding generates a good starting point, then a CB2CF-style mapping supplies CF quality.

Q2: Why is POSO more effective than just adding an is_new_user feature?

A: Because training is dominated by old users, a lone feature gets "submerged" — the model learns to ignore it. POSO uses $K$ dedicated submodules plus gating to structurally force new users through their own pathway, which cannot be ignored.

Q3: How to choose between MeLU and POSO?

A: If user behavior patterns are similar and few samples suffice to adapt → MeLU; if new/old user distributions differ greatly and features are easily submerged → POSO. They can also be combined.

Connections to Later Chapters

• 5.1 (debiasing): long-tail new items get little exposure and are easily drowned by popularity bias; cold start and debiasing must work in concert.
• 5.3 (generative): semantic IDs let new items be recommended without any behavior, easing content cold start at the representation level.
• Part 2 Retrieval (Ch2.x): the CF representations produced by CB2CF plug directly into two-tower/vector retrieval.

Practice Problems

Work through all problems in order — they get progressively harder. Each has a complete solution you can reveal after trying it yourself.

Problem 5.2.1 — Distinguishing Cold-Start Types 🟢 Easy

Is each scenario below content cold start or user cold start?

• (a) A newly launched documentary with no play records needs to be retrieved.
• (b) A freshly registered user has tapped only 3 videos, yet the system keeps recommending popular content.
• (c) A newly released song should go straight into personalized playlists, not just the "New Releases" list.

Problem 5.2.2 — Filling In the CB2CF Mapping 🟢 Easy

CB2CF learns a mapping function $f$ that maps a new item's content features $c_i$ into the collaborative-filtering space. Complete the output expression and explain the role of the constraint optimization module.

Problem 5.2.3 — Interpreting the MetaEmbedding Meta-Loss 🟡 Medium

MetaEmbedding's meta-loss is $l_{\text{meta},i}=\alpha l_a({\varphi }_{[i]}^{\text{init}})+(1-\alpha )l_b({\varphi }_{[i]}^{\prime })$. Explain: (1) what do the two terms each measure? (2) What happens if $\alpha =1$? (3) Why is it said to optimize "learnability" rather than the embedding itself?

Problem 5.2.4 — Designing a POSO Retrofit 🔴 Hard

You have a weight-shared MLP ranking model. Online, recommendations for new users perform far worse than for old users, even though an is_new_user feature has been added. Propose a retrofit following the POSO-MLP approach: write out the mathematical form of the submodules and the gate, and explain why this solves "feature submergence".

🏆 Challenge: A Cold-Start Combo

A short-video app faces both at once: new creators' content (content cold start) and newly registered users (user cold start). Write a plan of at most 200 words explaining how you would combine CB2CF / MetaEmbedding / POSO to address each, and identify which step depends most on "the quality of existing items' CF vectors".