Deep|META: Evolution of Recommender Model

Scaling Law, ROI, Generative Recommender Model 2.0

Since we began writing our reports, we've covered extensive ground on Ad Tech. However, we haven't systematically explained recommender algorithms and why AI and GPUs are fundamentally transforming them.

We've invested significant time understanding how META and other advertising companies evolve their recommender algorithms, frequently highlighting these contributions in our META Preview reports.

Before joining the hedge fund, I served as the recommender algorithm product manager for a short-video platform, witnessing the industry's complete transformation over the past decade. When I worked as a recommender algorithm PM, the short-video industry's content moderation and tagging systems were still nascent—I even participated in building some of the earliest moderation and tagging frameworks.

With earnings season now behind us, I'm dedicating two weeks to reviewing the technical evolution of recommender algorithms and explaining why LLMs and Agents are experiencing similar technological progressions.

This report will introduce recommender algorithm analysis approaches you've likely never encountered. I'll explain why AI and GPUs transformed the recommender algorithm industry without relying on technical jargon, and how AI has further revolutionized recommender algorithms in the past three months.

If 2023-2024 marked Generative Recommender Model 1.0, then the past three months have unlocked Generative Recommender Model 2.0.

• What Are Recommender Algorithms?

• What Is Generative Recommender Model (GR)?

• How GR and GPUs Transform Recommender Algorithm

• Wukong Model and Scaling Law

• What Did Generative Recommender Model 1.0 Actually Achieve?

• Does the ROI of Generative Recommender Model Make Sense?

• Generative Recommender Model 2.0

What Is Recommender Algorithm?

In the simplest terms, recommender algorithms filter millions of daily short videos on Meta Reels down to 10 videos loaded into a user's cache.

Before recommender algorithms, most content could only be displayed through lists. This was like having newspaper editors decide content choices for a billion people—we could only see the most popular/trending content, not what we actually wanted to see.

Below is a typical recommender algorithm workflow

• Candidate Corpus: Before large-scale GPU adoption, META Reels maintained a million-level recommendable candidate corpus. This content included not just daily videos—META used rule-based strategies to maintain content from shorter time windows in the content pool.

• Candidate Generation: Before large-scale GPU adoption, META Reels would retrieve thousands of videos from the Candidate Corpus each time.

• Pre-ranking: Conducts initial sorting of retrieved videos using simpler models or rules to quickly assess each video's potential value or relevance to users, further narrowing the recommender scope.

• Ranking: Before large-scale GPU adoption, META Reels' ranking models had tens to hundreds of billions of parameters. The goal was to build on pre-ranking using more complex, sophisticated models for in-depth analysis and sorting.

• Re-ranking: Adjusts ordering based on ranking results while adding optimization objectives.

What Is Generative Recommender Model (GR)?

Let's understand traditional recommender algorithms from a neural network architecture perspective, or more technically, Wide & Deep LRM.