HNHacker News
TopNewBestAskShowJobs

swyx

24,625 karma · joined December 9, 2015

Writer/Angel in devtools and smol AI enjoyer.

ai engineer pod/news: https://latent.space/

conference: https://ai.engineer

devrel/devtools: https://dx.tips/

personal blog: https://swyx.io/ideas

book: https://learninpublic.org/

but you can also email username [at] ai dot engineer

---

meet.hn/city/37.7792588,-122.4193286/San-Francisco

Socials:

- x.com/swyx - youtube.com/@swyxtv

submissionscomments
swyx··on Claude Fable is relentlessly proactive
> Zero tokens, just like in the old days!

because you zero rate your own human attention, which you should value

swyx··on Claude Fable is relentlessly proactive
> Having figured out all of these tricks Fable... hit some invisible guardrail and downgraded itself to Opus.

sigh

swyx··on Waymo Premier
> Waymo Premier costs $29.99 per month and will be initially offered to select riders in San Francisco, Los Angeles, and Phoenix.

> Waymo Premier is a new invite-only membership program built for those who rely on us most. For a monthly fee, members gain access to a suite of exclusive benefits designed to make their journey more seamless and rewarding:

Priority Pickups: Skip the line with prioritized matching

Ride Savings: Earn 10% Waymo Cash back on every trip, and even more during busy times.

Early Access: Be among the first to experience Waymo in new cities, as we expand.

Flexible Cancellations: Peace of mind with up to five free cancellations per month.

---

ok so just amazon prime for waymo. its alright but i feel like they had the chance to go REALLY high end with like a $300/month plan that people will still pay for because supply is so limited. instead they went mass consumer with a name like "Premier". eh.

(sorry waymo person reading this i know what its like to name a thing and regret it)

swyx··on Notes on DeepSeek
thanks. this really isnt that long, might as well paste in full here since OP deleted.

Notes on DeepSeek:

We visited the company HQ last Tuesday. It was founded in 2023 by Liang Wenfeng and operated out of his hedge fund, High-Flyer, until somewhat recently. The company released their R1 model in January 2025, so it was interesting to see what they’ve been doing

The company is located in an unmarked, 12-story building in Hangzhou. There is no DeepSeek branding visible from the street or lobby. I asked why this is, and the team demurred and said, “Well, there are many companies in this building, and we are not special.” They want to keep a low profile.

We met with their Head of Data and Head of Infrastructure. The company only has 300 employees. They are at least an order-of-magnitude smaller than Anthropic, and don’t care to scale further just yet. Their Head of Infrastructure, in particular, was young; maybe 30 years old and apparently one of the best AI buildout and energy experts in the country. (We briefly walked through the labs, and everybody seemed young. There was a lot of discussion; it felt like an exciting and energetic place.)

Lots of competition is coming from Alibaba (Qwen), ByteDance, and Moonshot (Kimi). People in China seem to mostly use Kimi or Deepseek. Young people use VPNs to access Claude, though Anthropic has blockers around usage in China and make it difficult. Poaching between groups is common, just like in the U.S. DeepSeek has a reputation as being really smart and “cool,” maybe similar to Anthropic. Big labs are mostly in Beijing, near Tsinghua and Peking University, with Hangzhou as the main exception (DeepSeek and Alibaba/Qwen are there).

The DeepSeek team reads western AI writers. They listen to Dwarkesh and read Gwern. The people we met with said they had never met with any employees from Anthropic. They were not at all concerned with some kind of hostile / AGI takeover scenario. They kept bringing up job loss (which is already high amongst youth in China) as their main concern. When we asked if they do red teaming on their models, they said no. In China, AI models are not regulated directly; the government instead has restrictions on how those models can be used in software, services, etc.

As a whole, China seems to treat AI as just another technology, rather than as some kind of singularity moment. National attention is still on basic needs and infrastructure buildouts, and on providing more medicines for people. The “dreams of singularity" seem like a luxury or distant consideration.

We asked the DeepSeek team: “What has the highlight been so far? What are your plans for an exit?” And they said that their highlight and great achievement was R1. They did not gesticulate at a future model or vision, but rather seemed proudest of what they’ve already done. They are content for now to remain ~6 months behind U.S. companies while maintaining a lower profile and team size.

swyx··on I Hate (Most) Keyboard 'Fn' Keys
> I Hate (Most) Keyboard 'Fn' Keys (danq.me)

> 11 points by speckx 13 minutes ago | flag | hide | 1 comment

this is currently #1 on HN gaining 11 points in 13 mins. never seen this before.

swyx··on Self Improving GTM Engineer Database
very nice adoption ladder: https://www.upside.tech/gtme/adoption-ladder

would you class any folks in your podcast along this ladder? i think it has potential.

swyx··on Claude Fable 5
jump in chart form https://x.com/swyx/status/2064414823748886591/photo/1
swyx··on FrontierCode
1 dimension is unfortunately all the mental bandwidth that talking heads have.
swyx··on FrontierCode
no disagreements. big fan of thinky
swyx··on Claude Fable 5
team member here - we had been working on frontiercode for ~6-7months. timing just lined up
swyx··on Claude Fable 5
he works on evals at canva
swyx··on FrontierCode
shared older model numbers here https://www.latent.space/p/ainews-frontiercode-benchmarking

tldr theres been broad progress despite your observed regressions

swyx··on FrontierCode
i think <third party evals platform> will help us do that best on their standardized model matrix. for frontiercode’s launch we were focused on.. the frontier models
swyx··on FrontierCode
to you it may do idk. note that if you scroll past fig 1 you get into a nice data explorer that breaks out pass@5 by reasoning level with token and $ and step cost visualized. i think some other commenters on this hn thread got very worked up about stuff we actually agree on.

internally ive charted everything and am satisfied that theres no meaningful rank bias introduced. weve sliced it every which way. in fact we have not even published the best looking charts for this story to be told, because we have further publishing plans on frontiercode

tldr “trust me bro” this isnt the issue and if anything we couldve done more to increase N as tedsanders below points out

swyx··on FrontierCode
ok i mean i agree, how is it a gaping hole when its literally the second (and third and fourth..) chart on the post? yes token cost and reasoning efficiency is important, hence the 2D pareto charts
swyx··on FrontierCode
no, single customer focus would be bad for a number of reasons.

but frontiercode-finance? thatd be cool…

swyx··on FrontierCode
i agree it would be interesting but apart from the fact that its be harder to measure and automate, theres real alpha in being the best truly async, hands off model/agent, which is what cog has been working on for 2 years now. its not that im opposed to steering or interaction mid task, its just that 1) it mostly Just Works, 2) it doesnt parallelize/scale well, 3) including on proactive agents (https://docs.devin.ai/product-guides/automations).

see my “semi async valley of death” post. people are pursuing both sides but per bitter lesson only one side scales indefinitely with compute

that said, multistage rollouts and synhetic rubrics (using grpo advantage? see dr tulu paper) somewhat approximate human intervention and interaction, so theres known ways to model that, its just not thaaaat valuable

swyx··on FrontierCode
this gives my non tired self a chance to fix the typo:

- “ ON TOP of that, 40+ hours of real human work to turn…”

+ “ ON TOP of that, 40+ hours of real human work PER TASK to turn…”

swyx··on FrontierCode
> Do you plan on releasing the tasks publicly?

yep

swyx··on FrontierCode
yes well aware :) numbers shown are on "house" harnesses eg codex with gpt and claude code with opus.

fwiw we have examples of each model doing better on NON-house harnesses too - speaking jsut for myself i think the "the labs are RLing on their own harnesses" narrative is kinda overstated if you think through wanting to have any meaningful api business (often eg the labs will give guidance on what is prefered and the agent labs can easily match tool contract to that, which is to say, the "home turf advantage" isnt as large as you think it is if you try a little bit)

swyx··on FrontierCode
thanks - credit to silas, eric, ben, and team for the depth of the evals, and the rest of the research team for doing the transcript reading parties lol

by nature of being based on open source, frontiercode public will saturate very very quickly. frontiercode main will be >80% in less than a year. hopefully diamond will last a bit longer. we can do annual refreshes, thats not my strategy for staying relevant - what i'm more excited to get funding for is private held out version of frontiercode based on repros of real enterprise customer problems. in an ideal agent lab (https://latent.space/p/agent-labs) you meticulously build up this domain understanding and that is essentially why both model labs and serious customers come to you.

swyx··on FrontierCode
*50 unique problems but 20-40 rubrics per problem (something I had to keep reminding people internally who were unimpressed with the N)

simple answer is our reporting was pass@5. feel like you'd need like 50+ runs to have reasonable confidence intervals, which somehow i dont see other people do, so i also didnt insist on it.

hoping to work with <prominent third party evals shop> to get this on their infra and evaluated along with whatever the industry standard is.

swyx··on FrontierCode
see Beyond Unit Tests and Novel Grading Methods in TFA.

i think something like ~60% llm as judge rubrics and the rest as described. every rubric validated by maintainer. 3000 rubrics

swyx··on FrontierCode: An eval to measure whether you would actually merge the code
:wave: i was on the team! AMA.

some headlines

- 3000 rubrics on code quality. First benchmark to measure: "would this code get actually merged?"

- 20+ expert open-source maintainer created tasks on their own repos to capture their opinion & taste.

- total 1000+ hours of real life software maintainer work captured in dataset. ON TOP of that, 40+ hours of real human work to turn that real life work into well validated and structured tasks with rubrics (even more work to turn tasks/prompts from devin-infra-specific to pluggable coding agent)

- results in 81% lower false positive rate than SWE-Bench Pro

- High quality bar: many QA stages & each task manually reviewed by Cognition researchers (examples in post)

Opus 4.8 scores 13% on FrontierCode Diamond.

one of my goals was also to datamine interesting stuff even on the easy tasks. for example, if you squint you can see the answer to "WTF Happened in late 2025" with coding models: https://x.com/swyx/status/2064081945567580323

swyx··on Biohub releases a world model of protein biology
and it is a capital N nonprofit https://latent.space/p/biohub
swyx··on Harness engineering: Leveraging Codex in an agent-first world
ya sorry we upgraded our thing after this
swyx··on Harness engineering: Leveraging Codex in an agent-first world
ha ok but why do that for a post you didnt write?
swyx··on Harness engineering: Leveraging Codex in an agent-first world
time of day also matters
swyx··on Harness engineering: Leveraging Codex in an agent-first world
we interviewed Ryan here: https://www.latent.space/p/harness-eng

and he gave a talk version of it in london: https://www.youtube.com/watch?v=am_oeAoUhew

swyx··on Biohub releases a world model of protein biology
we interviewed Alex Rives, cofounder of EvoScale and Head of Science at BioHub - here https://www.latent.space/p/esmfold2

also 3 paper coauthors walked thru it with us: https://youtu.be/4g1bURdKN0Q

all this is part of the new AI for Science effort we are spinning up at Latent Space - all guidance and support would be greatly appreciated as this is a much harder domain to cover than software

← PreviousPage 5 of 34Next →