x-algorithm

diving through the source: https://deepwiki.com/xai-org/x-algorithm and a codemap of the signal generation pipeline: https://deepwiki.com/search/make-a-map-of-all-the-signals_3d... - Phoenix (out of network) ranker seems to have all the interesting predictive ML work. it estimates P(favorite), P(reply), P(repost), P(quote), P(click), P(video_view), P(share), P(follow_author), P(not_interested), P(block_author), P(mute_author), P(report) independently and then the `WeightedScorer` combines them using configurable weights. there's an extra DiversityScore and OONScore to add some adjustments but again dont know the weights https://deepwiki.com/xai-org/x-algorithm/4.1-phoenix-candida... - other scores of interest: photo_expand_score, and dwell_score and dwell_time. share via copy, share, and share via dm are all obviously "super like" buttons. - Two-Tower retrieval uses dot product similarity between user features/engagement (User Tower) and normalized embeddings for all items (Candidate Tower). but when you look into the code and considering that this is probably the most important model for recommendations quality.... it's maybe a little disappointing that its a 2 layer MLP? https://deepwiki.com/search/what-models-are-used-for-user_98... - Grok-1 JAX transformer (https://github.com/xai-org/x-algorithm/blob/main/phoenix/REA...) uses special attention masking that prevents candidates from attending to each other during inference. Each candidate only attends to the user context (engagement history). This ensures a candidate's score is independent of which other candidates are in the batch, enabling score consistency and caching. nice image here https://github.com/xai-org/x-algorithm/blob/main/phoenix/REA... - kind of nice usage of Rust traits to create a type safe data pipeline. look at this beautiful flow chart https://deepwiki.com/xai-org/x-algorithm/3-candidate-pipelin... and the "Field Ownership pattern" https://deepwiki.com/xai-org/x-algorithm/3.6-scorer-trait#fi... - the ten pre-scoring filters are minorly interesting, nothing super surprising here apart from AgeFilter (https://deepwiki.com/xai-org/x-algorithm/4.6.1-agefilter) which I guess means beyond a certain max_age (1 day?) nothing ever shows up on For You. surprising to have a simple flat cutoff vs i guess the alternative of an exponential aging algorithm. - videoduration hydrator explicitly prioritizes video duration (https://deepwiki.com/xai-org/x-algorithm/4.5.6-videoduration...) but we dont know in what direction... do you recommend shorter or longer videos? and why a hydrator for what is presumably a pretty static property? open questions from me 1. how large is the production reranker? default param count is here https://deepwiki.com/search/how-many-params-is-the-transfo_c... but that gives no indication. the latency felt ultra high initially last year and seems to have come down some, what budget are we working with? 2. can we make the retrieval better? i dont have a tooon of confidence in the User Tower / Candidate Tower system - is this SOTA (it's probably not - see how youtube does codebook semantic id's https://www.youtube.com/watch?v=LxQsQ3vZDqo&list=PLcfpQ4tk2k... ) 3. no a/b testing / rollout infrastructure? 4. so many hydration subsystems - is this brittle?