24,625 karma · joined December 9, 2015
ai engineer pod/news: https://latent.space/
conference: https://ai.engineer
devrel/devtools: https://dx.tips/
personal blog: https://swyx.io/ideas
book: https://learninpublic.org/
but you can also email username [at] ai dot engineer
---
meet.hn/city/37.7792588,-122.4193286/San-Francisco
Socials:
- x.com/swyx - youtube.com/@swyxtv
"trust but verify" ofc . https://latent.space/p/artificialanalysis do api keys but also mystery shopper checks
yes you got the important thing!
generally think that there's a lot of fertile ground for smart generalist engineers to make a ton of progress here this year + it will probably be extremely financially + personally rewarding, so I broadly want to create a dedicated pod to highlight opportunities available for people who don't traditionally think of themselves as "in science" to cross over into the "ai for hard STEM" because it turns out that 1) they need you 2) you can fill in what you don't know 3) it will be impactful/challenging/rewarding 4) we've exhausted common knowledge frontiers and benchmarks anyway so the only* people left working on civilization-impacting/change-history-forever hard problems are basically at this frontier
*conscious exaggeration sorry
say more pls?
shudders in vietnam war flashbacks congrats on launch guys!!!
for those who want an independent third party endorsement, here's Brex CTO talking about Mastra in their AI engineering stack http://latent.space/p/brex
related meme i saw today:
“bro I spent all weekend in Claude Code it’s incredible”
“oh nice, what did you build?”
“dude my setup is crazy. i’ve got all the vercel skills, plus custom hooks for every project”
“sick, what are you building?”
“my setup is so optimized, i’m using like 5 instances at once”
looks like this is the "for you" feed, once again shared without weights so we only have so much visibility into the actual influence of each trait.
"We have eliminated every single hand-engineered feature and most heuristics from the system. The Grok-based transformer does all the heavy lifting by understanding your engagement history (what you liked, replied to, shared, etc.) and using that to determine what content is relevant to you." aka it's a black box now.
the README is actually pretty nice, would recommend reading this. it doesnt look too different form Elon's original code review tweet/picture https://x.com/elonmusk/status/1593899029531803649?lang=en
sharing additonal notes while diving through the source: https://deepwiki.com/xai-org/x-algorithm
and a codemap of the signal generation pipeline: https://deepwiki.com/search/make-a-map-of-all-the-signals_3d...
- Phoenix (out of network) ranker seems to have all the interesting predictive ML work. it estimates P(favorite), P(reply), P(repost), P(quote), P(click), P(video_view), P(share), P(follow_author), P(not_interested), P(block_author), P(mute_author), P(report) independently and then the `WeightedScorer` combines them using configurable weights. there's an extra DiversityScore and OONScore to add some adjustments but again dont know the weights https://deepwiki.com/xai-org/x-algorithm/4.1-phoenix-candida... - other scores of interest: photo_expand_score, and dwell_score and dwell_time. share via copy, share, and share via dm are all obviously "super like" buttons.
- Two-Tower retrieval uses dot product similarity between user features/engagement (User Tower) and normalized embeddings for all items (Candidate Tower). but when you look into the code and considering that this is probably the most important model for recommendations quality.... it's maybe a little disappointing that its a 2 layer MLP? https://deepwiki.com/search/what-models-are-used-for-user_98...
- Grok-1 JAX transformer (https://github.com/xai-org/x-algorithm/blob/main/phoenix/REA...) uses special attention masking that prevents candidates from attending to each other during inference. Each candidate only attends to the user context (engagement history). This ensures a candidate's score is independent of which other candidates are in the batch, enabling score consistency and caching. nice image here https://github.com/xai-org/x-algorithm/blob/main/phoenix/REA...
- kind of nice usage of Rust traits to create a type safe data pipeline. look at this beautiful flow chart https://deepwiki.com/xai-org/x-algorithm/3-candidate-pipelin... and the "Field Ownership pattern" https://deepwiki.com/xai-org/x-algorithm/3.6-scorer-trait#fi...
- the ten pre-scoring filters are minorly interesting, nothing super surprising here apart from AgeFilter (https://deepwiki.com/xai-org/x-algorithm/4.6.1-agefilter) which I guess means beyond a certain max_age (1 day?) nothing ever shows up on For You. surprising to have a simple flat cutoff vs i guess the alternative of an exponential aging algorithm.
- videoduration hydrator explicitly prioritizes video duration (https://deepwiki.com/xai-org/x-algorithm/4.5.6-videoduration...) but we dont know in what direction... do you recommend shorter or longer videos? and why a hydrator for what is presumably a pretty static property?
open questions from me
1. how large is the production reranker? default param count is here https://deepwiki.com/search/how-many-params-is-the-transfo_c... but that gives no indication. the latency felt ultra high initially last year and seems to have come down some, what budget are we working with?
2. can we make the retrieval better? i dont have a tooon of confidence in the User Tower / Candidate Tower system - is this SOTA (it's probably not - see how youtube does codebook semantic id's https://www.youtube.com/watch?v=LxQsQ3vZDqo&list=PLcfpQ4tk2k... )
3. no a/b testing / rollout infrastructure?
4. so many hydration subsystems - is this brittle?
LLM Code Generation - Unambiguous syntax reduces AI errors
Testing Discipline - Mandatory tests improve code quality
Simple & Fast - Minimal syntax, native performance
Design Philosophy:
Minimal syntax (18 keywords vs 32 in C)
One obvious way to do things
Tests are part of the language, not an afterthought
Transpile to C for maximum compatibility
ehh. i dont think the overhead of inventing a new language makes up for the lack of data around it. in fact if you're close enough to rust/c then llms are MORE likely to make up stuff from their training data and screw up your minimal language.
(pls argue against this, i want to be proven wrong)
mm i'd actually say its more moderated, lots of cynics/skeptics.
sorry what is this? exaggeration?
whats the sota on atproto encryption dan? just publish encrypted stuff with sha 256 and thats it?
yes: https://www.swyx.io/data-outlasts-code-but
all lasting work is done in files/data (can be parsed permissionlessly, still useful if partially corrupted), but economic incentives keep pushing us to keep things in code (brittle, dies basically when one of maintainer|buildtools|hardware substrate dies).
when standards emerge (forcing code to accept/emit data) that is worth so much to a civilization. a developer ecosystem tipping the incentive scales such that companies like the Googl/Msft/OpenAI/Anthropics of the world WANT to contribute/participate in data standards rather than keep things proprietary is one of the most powerful levers we as a developer community collectively hold.
(At the same time we shoudl also watch out for companies extending/embracing/extinguishing standards... although honestly outside of Chrome I struggle to think of a truly successful example)
always nice to see a database ceo be "one of us" and/or "write like a real human being".