ankit219

My conjecture is that their memory implementation is not aimed at building a user profile. I don't know if they would or would not serve ads in the future, but it's hard to see how the current implementation helps them in that regard.

The difference is implementation comes down to business goals more than anything.

There is a clear directionality for ChatGPT. At some point they will monetize by ads and affiliate links. Their memory implementation is aimed at creating a user profile.

Claude's memory implementation feels more oriented towards the long term goal of accessing abstractions and past interactions. It's very close to how humans access memories, albeit with a search feature. (they have not implemented it yet afaik), there is a clear path where they leverage their current implementation w RL posttraining such that claude "remembers" the mistakes you pointed out last time. It can in future iterations derive abstractions from a given conversation (eg: "user asked me to make xyz changes on this task last time, maybe the agent can proactively do it or this was the process last time the agent did it").

At the most basic level, ChatGPT wants to remember you as a person, while Claude cares about how your previous interactions were.

there is clearly overhype. Given the influx of people building here (and i am not aware if it happened previously too), in a bid to differentiate from other startups building the same thing, many just stripped the nuance out of any technical idea, and made it a simple marketing term. As an exec where ten startups are promising you "training on your data" to provide the best chatbot, it's hard to tell the difference between who would be actually training and who would be just tweaking prompts. This has happened to other concepts too (anecdotally, most famous is how everyone is offering deep research). yes, it helps growth, but comes at the cost of trust. There is this startup which promised "experience based learning" when all they were doing is adding memory to the prompt to get it to perform better. (you can look it up, recenty raised series A).

This does not mean ideas are not working. I personally think pretraining has done its job. We did not know what the job previously was, but now we do given the way RL works. Pretraining and test time compute enables models to develop a generalized prior they can use to solve any given problem (much like how humans solve such problems). Sometimes priors are lacking so you need to train more using RLVR, and still early days, but directionally I think we have another scaling curve here.

> your particular persuasive triggers through chatbot memory features, where they train and fine-tune based on your past conversations

Represents a fundamental misunderstanding of how training works or can work. Memory is more to do with retrieval. Finetuning on those memories would not be useful given the data is going to be minuscule to affect the probablity distribution in the right way.

While everyone is for privacy (and thats what makes these arguments hard to refute), this is clearly about using privacy as a way to argue against using conversational interfaces. Not just that, it's using the same playbook to use privacy as a marketing tactic. The argument starts from highly persuasive nature of chatbots, to somehow privacy preserving chatbots from DDG wont do it, to being safe with hackers stealing your info elsewhere and not on DDG. And then asking for regulation.

> We use a single agent architecture (as we found this reduces hallucinations)

Do you have a benchmark for this? in my experience, hallucinations have nothing to do with what framework you use.

I like the thesis, (and probably not a fully informed opinion here), it's easier for openai like router to go the affiliate model than ads model. Router can determine how much a query is worth (eg: help plan a vacation is worth $50 vs say what is the capital of Australia as $0). This further informs how much compute to use, and this can lead to how much they get on affiliate fees if a link is clicked. Ads here are counter to the proposition in the sense that results are to be trusted. Then, despite everything, while contextual ads are good, personalized ads are still going to be more effective (gut feel). To get there, openai needs the kind of infra Facebook and Google has. They can get the same value (perhaps more) from affiliate links - especially in a world where they are kind of gateway to the discovery - and don't have to do as much work on the infra side. This also aligns incentives for all three - companies, consumers, and middlemen, in a way it only happened with Google before this.

The problem for the judge seems to be that there is no alternative at this point. No other company can bid for or credibly pay Apple/Mozilla as much as Google did. Apple testified they would spend less on innovation if the payment goes away, Mozilla said they wont survive. So the alternative for the judge is to create a market in the next five years where people invest in search, there are more credible products that come up, and are competitive enough to justify the placement bids (ending dependency on google).

The nuclear option was DDG's hope. Google should share their entire data, so DDG can offer the same product without having to build out the thing themselves. The judge correctly identified (imo) where this sharing of index and search results would have meant a bunch of white labeled wrappers selling Google search and would have no incentive to innovate themselves in the short term. Somehow, DDG did not see that happening. At that goal, it's a great ruling, well considered.

In Jan, when deepseek launched, Dario Amodei had to disclose they spent about $10M to train the last generation of models (his arguments was deepseek was on the curve, not breaking it).

They earned $250M in May based on ARR, and about $400M in july. Model training is going to be amortized over multiple years anyway. I am not privy to how much they spent, not going to comment on that. GM was public news, and hence I got that.

Re Zitron's analysis, I don't find them to be reliable or compelling.

Re Chrome divesture:

> The remedy also extends beyond the conduct Plaintiffs seek to redress. It was Google’s control of the Chrome default, not its ownership of Chrome as a whole, that the court highlighted in its liability finding. See Google, 747 F. Supp. 3d at 120–21. Ordering Google to sell one of its most popular products, one that it has built “from the ground up” and in which it has invested (and continues to invest) billions of dollars, in the hope of opening a single channel of distribution to competition—and not even one that was unlawfully foreclosed by the challenged contracts—cannot reasonably be described as a remedy “tailored to fit the wrong creating the occasion for the remedy.” Microsoft III, 253 F.3d at 107; Rem. Tr. at 2466:23–2468:3 (Pichai); id. at 1634:23–1636:2 (Tabriz) (discussing PXR0215 at -257). Further, as a legal matter, the divestiture of Chrome exceeds the proper scope of relief. “All parties agree that the relevant geographic marketis the United States.” Google, 747 F. Supp. 3d at 107. Chrome, however, is not so geographically confined. The vast majority—over 80%—of its monthly active users are located outside the United States. Rem. Tr. at 1619:23–1620:6 (Tabriz). Plaintiffs do not try to make the case that a divestiture of Chrome to just U.S.-based users is feasible.

Their projections for ARR at the end of this year at a high of $9B[1] at the end of this year. And reported gross margins of 60% (-30% with cloud providers partnerships). All things considered, if this pans out, it's a 20x multiple. High yes, but not that crazy. Specially considering their growth rate and that too at a decent margin at gm level.

[1]: It was $3B at the end of May (so likely $250M in May alone), and $5B at end of july (so $400M that month).

Calling SEO content high quality is overestimating the nature and level of an SEO article. The other aspect that it implies is businesses are the reason we get so much high quality content. That is provably wrong. The moment people saw that some kinds of links got them high ranking on google, it resulted in an industry of producing SEO garbage where businesses would publish 1000s of words to answer something which could have been answered in a single word. (usual caveats exist)

People want signal and answers, not the 10 blue links as this post tries to argue.

The other thing is this: most high quality and valuable content can now be produced by individuals and finds distribution on social networks where they can occasionally charge for it as well. The drawback to google indexing those links was also that SEO-companies started targeting these mediums (eg: reddit, medium, forums). We needed an early regulation to minimize the needless hacking of SEO, but we let the market play it out, so it should still play out.

Oh nothing official. There are people who estimate the sizes based on tok/s, cost, benchmarks etc. The one that most go on is https://lifearchitect.substack.com/p/the-memo-special-editio.... This guy estimated Claude 3 opus to be 2T param model (given the pricing + speed). Opus 4 is 1.2T param according to him (but then I dont understand why the price remained the same.). Sonnet is estimated by various people to be around 100B-200B params.

[1]: https://docs.google.com/spreadsheets/d/1kc262HZSMAWI6FVsh0zJ...

A discounted Azure H100 will still be more than $2 per hour. Same goes for AWS. Trainium chips are new and not as effective (not saying they are bad) but still cost in the same range.

For inference, gross margins are exactly: (what companies charge per 1M tokens to the user) - (direct cost to produce that 1M tokens which is GPU costs).

This seems very very far off. From the latest reports, anthropic has a gross margin of 60%. It came out in their latest fundraising story. From that one The Information report, it estimated OpenAI's GM to be 50% including free users. These are gross margins so any amortization or model training cost would likely come after this.

Then, today almost every lab uses methods like speculative decoding and caching which reduce the cost and speed up things significantly.

The input numbers are far off. The assumption is 37B of active parameters. Sonnet 4 is supposedly a 100B-200B param model. Opus is about 2T params. Both of them (even if we assume MoE) wont have exactly these number of output params. Then there is a cost to hosting and activating params at inference time. (the article kind of assumes it would be the same constant 37B params).

They are still gating it by usecase (I presume). But this way, they are not limited to the creativity of what their self selected group of beta testers could come up with, and perhaps look at security against a more diverse set of usecases. (I am assuming the trusted testers who work on security etc would anyway be given access).

This is what they need for the next generation of models. The key line is:

> We view browser-using AI as inevitable: so much work happens in browsers that giving Claude the ability to see what you're looking at, click buttons, and fill forms will make it substantially more useful.

A lot of this can be done by building a bunch of custom environments at training time, but only a limited number of usecases can be handled that way. They don't need the entire data, they still need the kind of tasks real world users would ask them to do.

Hence, the press release pretty much saying that they think it's unsafe, they don't have any clue how to make it safe without trying it out, and they would only want a limited number of people to try it out. Give their stature, it's good to do it publicly instead of how Google does it with trusted testers or Openai does it with select customers.

I like the idea. I also think you may need a mechanism to detect adverse actions before they are executed. This becomes important because an email cannot be unsent, and if I dont review the text, the 1 in 100 chance of the email sounding weird would freak me out. There is also the basic corner cases against AI prompt injection and all those spam and phishing emails that are rampant and more and more plausible sounding. Wonder if you have ideas around how to deal with those?

Building with non deterministic systems isnt new. It does not take a scientist. Though people who have experience with these systems are fewer in number today. You saw the same thing with TCP/IP development where we ended up developing systems that assumed the randomness and made sure that isnt passed on to the next layer. For every game, given the latency involved in previous networks, there is no way on the network games were deterministic.

From Incognito window's note > Others who use this device won’t see your activity, so you can browse more privately. This won't change how data is collected by websites you visit and the services they use, including Google. Downloads, bookmarks and reading list items will be saved

Incognito does not hide your activity from Google. Especially when you googled in incognito and they likely use IP addresses as part of their targeting. I am also assuming it's different for different kinds of ads given you wont see ads if you look at something personal. They infact allow IP address targeting somehow. [1] Their privacy stance is more about 3rd party not having access to the data google has collected.

[1]: https://www.shopifreaks.com/google-to-allow-the-use-of-ip-ad...

Yes, we are getting there. I think compiler is a bigger problem than unit tests given most verticals don't even have that. With unit tests, there would be some reward hacking but would be controlled at the model level + tests. (this is one of the reason i dont believe in transformer based llm as a judge for a verifier)

The bottleneck for automation is verification. With human work, verification was fast(er) because you know where to look with certain assumptions that your upstream tasker would not have made trivial mistakes. For automation, AI needs to verify it's own work, review, and self correct to be able to automate any given work. Where this works, it will also change the abstraction layer compared to what it is today. The problem is same with every automation promise - it needs to work reliably at say 95% or 99% times and when it doesn't, there should be human contingency in terms of what to look for. Considering coding as the first example: it's already underway. AI generates the code, the test cases, and then verifies if the code works as intended. Code has a built in verification layer (both compiler and unit tests). High probablity the other domains move towards something similar too. I would also say the model needs to be intelligent to course correct when the output isn't validated[1].

Verification solves the human in the loop dependency both for AI and human tasks. All the places where we could automate in the past, there were clearly quality checks which ensured the machinery were working as expected. Same thing will be replicated with AI too.

Disclaimer: I have been working on building a universal verifier for AI tasks. The way it works is you give it a set of rules (policy) + AI output (could be human output too) and it outputs a scalar score + clause level citations. So I have been thinking about the problem space and might be over rating this. Would welcome contrarian ideas. (no, it's not llm as a judge)

[1]: Some people may call it environment based learning, but in ML terms i feel it's different. That woudl be another example of sv startups using technical terms to market themselves when they dont do what they say.

This is super cool. Usually you dont see effective models at 270M out in the wild. The architectural choices are new and interesting as well.

Would it be okay for you to divulge some more training information here? With 170M embedding parameters, how do you ensure no embedding collapse and keeping the embedding matrix stable at training time?

(i know i am asking too much, but just curious). There is a clear trade off for you with vocab / transformer layers. How did you arrive at the split of 170m/100m. Does this contribute to model's performance on task specific fine tuning? Any internal experiments you could share? or public info you could point us to? Anything would be amazing.

PS: I am sorry if this is rude, but this has so many decisions i am curious about. Not intending to undermine anything, this is amazing work, and thank you for the whole Gemma series.

No justification for a $100k number. For $100k a year or about $8k a month, you will end up using 1B tokens a month (that too a generous blended $8 per million input/output tokens including caching while the number is lower than that). Per person.

I think there is a case Claude did not reduce their pricing given that they have the best coding models out there. There recent fundraise had them disclose their Gross margins at 60% (and -30% with usage via bedrock etc). This way they can offer 2.5x more tokens at the same price than the vibe code companies and yet break even. The market movement where the assumption did not work out was about how we still only have claude which made vibe coding work and is the most tasteful when it comes to what users want. There are probably models better at thinking and logic, especially o3, but this signals the staying power of claude - having a lock in, it's popularity, and challenges the more fundamental assumption about language models being commodities.

(Speculating) Many companies woudl want to move away from claude but cant because users love the models.

This is speculated as the reason as blockbuster acquihires have risen:

https://www.bloomberg.com/opinion/articles/2025-07-17/meta-g...

https://natlawreview.com/article/rise-acquihiring-post-layof...

https://bowoftheseus.substack.com/p/update-the-gut-and-licen...

Meta's case is interesting. In the past, for what they want, I still feel they would have just acquired the company and be done with it. Now, they explored more paths, and ended up negotiating for Scale AI's stake.

Meta's case might be different, and hence i wrote it as scale would have been acquired in past times instead of an investment like they did today. For other companies, the hiring aspect avoids M&A filings, balance sheet consolidation, and still give the big tech companies access to IP and talent. Easier path to get what you want than say Wiz acquisition still waiting to be completed. The stance may have recently changed, but seems like big tech companies are not testing it. (it's also not just US, but other jurisdictions have a say too. Meta had to sell Giphy due to issue raised by UK/Europe)

Re Microsoft and Windsurf, this[1] article claims Windsurf did not want MS accessing it's intellectual property, a default condition for Microsoft OpenAI deal.

> Windsurf didn’t want Microsoft to have access to its intellectual property — a condition that OpenAI was unsuccessful in getting Microsoft’s agreement on, people familiar said. That was one of several sticking points in Microsoft and OpenAI’s ongoing talks about the AI company’s effort to restructure into a commercial entity. Microsoft’s existing agreement with OpenAI says the software giant is entitled to access the startup’s technology.

This is different from Microsoft not wanting openai to compete with Github copilot, given thats what they do with Codex anyway.

> Knowledge is extremely valuable when it comes to AI/ML We agree on that, but even in the past most acquisitions would value teams at a certain cost too. Without the team that built it, any company would struggle to get a good multiple. OP's claim was that lower multiple was cos of margins only, and i disagreed there.

[1]: https://www.bloomberg.com/news/articles/2025-07-11/openai-s-...

They arent public and pitch in front of sophisticated guys who can probably tell this apart.

This is a highly speculative post, with conjectures presented as facts. Some things that irked me:

- Cursor did not hire Anthropic's "researchers". It hired the guys who built Claude code (PM and dev). Who then promptly went back to Anthropic in 14 days. A researcher for Cursor need not come from Anthropic either. One high profile recruit for them was Jack Gallagher (Midjourney) who is probably one of the best at RL.

- Google's deal with Windsurf is structured that way because they likely could not directly acquire, or were not confident that it would have gotten past the antitrust. A signal for that is such deal increased in last few years after FTC refused to allow any deal over $100M or so. Microsoft has done such deals too. Meta would have acquired scale ai in older times. Not sure with Openai, but they arent as scrutinized as Google for such deals. To imply that this means Google did not care about ARR is not justified. and then google licensed Windsurf's IP too.

- Openai's agreements with Microsoft is more probable than they did not complete the acquisition because of negative gross margins.

- Plus, the old adage about how a growing startup is worth more because of a stellar team. You strip a team away and still get 2x multiple is sure enough valuing the current ARR highly.

I thought the userbase is valuable. A sale at this point made sense because they might not have been to get the money if they waited a year. Reasons laid out in the article are not why I think so.

Anthropic in the latest fundraise announce their gross margins are around 60%. Typically in the industry now, the gross margins are reported as revenue - cost of only serving paid users (free users etc. go in either cac or r&d). The thing is, with this margin, Anthropic can serve 2.5x more tokens than say cursor without making a loss for the same price compared to cursor. This reality is likely going to hit other startups based on the same assumption too. Anthropic has not reduced its pricing in almost a year now.

I think CLI is a good idea for now. Next abstraction seems to be Github PRs where someone (likely me) files an issue/feature, then I click a button, and the agent fixes the issue/feature. Github has talked about something similar, but surely it were a pain to figure out if it was GA and I had access to it given so many different variations they have called gh copilot. (PS: it exists, but not as smooth as I described: https://docs.github.com/en/copilot/how-tos/use-copilot-agent... )

This was a version where they wanted to collect data. The next version where gemini cli/jules harness is likely part of the RL training environment and the model would work a lot better. Thats the trajectory of improvement.