x-algorithm

# X For You Feed Algorithm This repository contains the core code that determines which posts a viewer sees in the **For You** feed on X. It combines in-network content (from accounts the viewer follows) with out-of-network content (discovered through ML-based retrieval and other mechanisms), filters content based on a variety of inputs, and ranks posts using a transformer model. ## Table of Contents - [Notable Updates](#notable-updates) - [August 14th, 2026](#august-14th-2026) - [August 13th, 2026](#august-13th-2026) - [Overview](#overview) - [System Architecture](#system-architecture) - [Request Path](#request-path) - [Labeling Path](#labeling-path) - [Components](#components) - [How It Works](#how-it-works) - [Scoring and Ranking](#scoring-and-ranking) - [Filtering](#filtering) - [Experiments and Configuration](#experiments-and-configuration) - [What's not in this repo?](#whats-not-in-this-repo) - [Under the Hood Label Transparency Tool](#under-the-hood-label-transparency-tool) - [Key Design Decisions](#key-design-decisions) - [License](#license) --- ## Notable Updates ### August 14th, 2026 Notable updates: - **How weights work.** There's a common misconception about how weights related to actions (e.g. Like, Share, Block, Report, etc) work in ranking. The weights scale the predicted probabilities of such actions (or predicted continuous values, e.g. dwell time) — they do *not* scale the raw engagement counts, so e.g. it'd be incorrect to see that a report has 468 times higher weight than a like and conclude that e.g. "1 report cancels out 468 likes". The weights are a multiple on your own predicted probability of Liking, Reporting, etc, which is substantially driven by your own behavior. We've [added comments](home-mixer/params/param.rs) [to the code](home-mixer/scorers/ranking_scorer.rs) so that LLMs or people reading it are more likely to understand it correctly. - **Brazil 2026 Elections.** As [announced by X](https://x.com/XBR/status/2088341967864320507?s=20), in accordance with Brazilian electoral law, For You now runs `Brazil2026ElectionFilter`, which removes posts from accounts reported to Brazil's Electoral Court for the 2026 election, unless the viewer explicitly follows the account. *(Account list updated August 27, 2026.)* A benefit of open-source is that you can see that changes like this exist, and exactly how they work — take a [look at the code](home-mixer/filters/brazil_2026_election_filter.rs). ### August 13th, 2026 This release: - Adds key configuration parameters (including weights used to blend predicted action values into a score for a post) - Adds code for systems that impact whether a post is filtered from the For You feed - Replaces the Phoenix demonstration model with the code used to train the models the feed uses, as well as synthetic data generation code so one can run a proof-of-concept training run of Phoenix. Among new systems included are: 1. **Visibility filtering:** [`visibility-filtering/`](visibility-filtering/) determines whether to show a post, drop it, or show it behind an interstitial. 2. **The systems that produce labels that drive visibility filtering's responses:** rules that apply labels ([`botmaker/`](botmaker/), [`botmaker-rules/`](botmaker-rules/), [`scarecrow/`](scarecrow/)), models that score accounts on various dimensions ([`agatha/`](agatha/), [`bdsm/`](bdsm/), [`user-cred-v2/`](user-cred-v2/)), models that examine images and video ([`media-model-proxy/`](media-model-proxy/), [`clip/`](clip/)), and enforcement ([`abuse-enforcement-service/`](abuse-enforcement-service/)). 3. **Phoenix model code:** [`phoenix/`](phoenix/) now contains code that trains and runs the model, plus synthetic data generation. 4. **SimClusters:** [`simclusters/`](simclusters/), an additional source of posts from accounts the viewer does not follow that is called in retrieval alongside Thunder and Phoenix retrieval. This update is also paired with a new [**Under the Hood**](#under-the-hood-label-transparency-tool) transparency tool that allows people to see aggregate statistics about the labels on their account and posts that can limit visibility. --- ## Overview The For You feed is assembled per request. Posts come from two places: 1. **In-Network** — [`thunder/`](thunder/) keeps recent posts from the accounts a viewer follows in memory 2. **Out-of-Network** — [`phoenix/`](phoenix/) retrieval and [`simclusters/`](simclusters/) find posts from accounts the viewer does not follow Both are ranked together by the same model. **Phoenix** reads the viewer's recent engagement history and predicts, for each post, how likely the viewer is to take each action on it. Those predictions are combined into one score using weights held in the code — see [Scoring and Ranking](#scoring-and-ranking). Two pipelines do the work. The **Post Pipeline** finds, ranks and filters posts. The **Blending Pipeline** wraps it and adds what the model does not rank: ads, Who to Follow recommendations, prompts. Ranking sets the order. Whether a post can be shown at all is decided separately, by [`visibility-filtering/`](visibility-filtering/), from the viewer's own actions such as blocks and mutes and from labels that other systems here attach to posts and accounts. --- ## System Architecture ### Request Path

┌──────────────────────────────────────────────────────────────────────────────────────────┐
│                                   FOR YOU FEED REQUEST                                   │
└──────────────────────────────────────────────────────────────────────────────────────────┘
                                             ▼
┌──────────────────────────────── HOME MIXER   home-mixer/ ────────────────────────────────┐
│                                                                                          │
├───────────────────────  POST PIPELINE   PhoenixCandidatePipeline  ───────────────────────┤
│                                                                                          │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 1. QUERY HYDRATION                                                                 │  │
│  │    user action sequence — the viewer's recent engagements, and the                 │  │
│  │    main input to the model · following list · blocks and mutes · muted             │  │
│  │    keywords · posts already seen and served · followed topics, etc.                │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 2. CANDIDATE SOURCES — queried in parallel                                         │  │
│  │    ┌───────────────────────────────┐ ┌────────────────────────────────────────┐    │  │
│  │    │ IN-NETWORK                    │ │ OUT-OF-NETWORK                         │    │  │
│  │    │ Thunder                       │ │ Phoenix retrieval   retrieval model    │    │  │
│  │    │   recent posts from the       │ │ SimClusters         cluster similarity │    │  │
│  │    │   accounts the viewer follows │ │                                        │    │  │
│  │    └───────────────────────────────┘ └────────────────────────────────────────┘    │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 3. CANDIDATE HYDRATION                                                             │  │
│  │    post text and media · author details and account labels · quoted post ·         │  │
│  │    language · engagement counts · subscription status, etc.                        │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 4. PRE-SCORING FILTERS                                                             │  │
│  │    duplicates across sources · older than 48 hours · the viewer's own              │  │
│  │    posts · blocked and muted accounts · muted keywords · already seen              │  │
│  │    or served · subscriber-only posts the viewer cannot access, etc.                │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 5. SCORING                                                                         │  │
│  │    PhoenixScorer   a probability for each action the viewer might take             │  │
│  │    RankingScorer   weighted sum, then repeated-author decay, an                    │  │
│  │                    out-of-network discount, a new-author boost                     │  │
│  │    VMRanker        calls the reranking service in vm-ranker/                       │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 6. SELECTION — TopKScoreSelector                                                   │  │
│  │    sort by final score, keep the top K                                             │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 7. POST-SELECTION FILTERS — after the order is fixed                               │  │
│  │    VFCandidateHydrator  asks visibility-filtering/ per post and viewer             │  │
│  │    VFFilter             removes the posts it said to drop                          │  │
│  │    DedupConversationFilter  collapses branches of one conversation                 │  │
│  │                         ◄── these labels come from the Labeling Path               │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                                                                          │
├─────────────────────  BLENDING PIPELINE   ForYouCandidatePipeline  ──────────────────────┤
│                                                                                          │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ the ranked posts are one source here; the othe