williamtrask

"You are not at the beginning of the curve like you were in the"

Note: on an exponential, you can always say this. People will look back on today with the same "aww cute" eyes.

The scandals/year page has a little more umph to it than the main page https://ycombinator.fyi/timeline

Yeah... probably right. I do hold out hope that this is mostly a timeframe thing. Like, the library, printing press, etc. all had their moments of centralization. But eventually they federated.

fwiw - i think the design looks good.

I wonder if a popularization moment for local AI will ultimately be the pin-prick that pops the AI bubble. Like the deepseek or openclaw moments but bigger/next.

A non-profit to deconcentrate power over AI through better infrastructure for external auditing/oversight, and better infrastructure for local/federated inference/training https://openmined.org/

Also, we're hiring engineers and PMs (the eng position is about to be up). https://openmined.org/careers/#brxe-zgsziy

i like your comment better than mine. more please.

i'm not sure it's productive to think this way. senators could be making more money on prediction markets. they took a nice step which will lead them to make no money on prediction markets (less money overall). it also sets a precedent which could easily be applied to the stock market.

what you're saying is probably on the mind of at least one Senator, but all things considered, this feels like a net-positive move which they didn't have to do.

Sometimes I find joy in noticing the importance of comma placement:

Everything the incumbents ship, in an open codebase your firm owns.

vs

Everything the incumbents ship in an open codebase, your firm owns.

<3 - have ambitious plans to release a new video/blogpost each week, but that's pretty ambitious we'll see

...with a price :)

"Conclusion Our monorepo isn't about following a trend. It's about removing friction between things that naturally belong together, something that is critical when related context is everything.

When a feature touches the backend API, the frontend component, the documentation, and the marketing site—why should that be four repositories, four PRs, four merge coordination meetings?

The monorepo isn't a constraint. It's a force multiplier."

Thank you Claude :)

tried searching for "noodlesUK" and didn't find anything meaningful

Nit: regarding (2), Phil Blunsom did (same Blunsom from the article, and who was leading language modeling at DeepMind for about 7-8 years). He would often opine at Oxford (where he taught) that solving next word prediction is a viable meta path to AGI. Almost nobody agreed at the time. He also called out early that scaling and better data were the key, and they did end up being, although Google wasn’t as “risk on” as OpenAI on gathering the data for GPT-1/2. Had they been history could easily have been different. People forget the position OAI was in at the time. Elon/funding had left, key talent had left. Risk appetite was high for that kind of thing… and it paid off.

"This is not the reason, the reason is that this data is private. LLMs do not just learn from data, they can often reproduce it verbatim, you cannot give medical records or bank records of real people, that will put them at a very real risk."

(OP) You make great points. I think we're actually more in agreement than might be obvious. Part of the reason you need to "give" data to an LLM is because of the way LLMs are constructed... which creates the privacy risk.

The principle of attribution-based control suggested in this article would break that principle, enabling each data owner to control which AI predictions they make more intelligent (as opposed to only controlling which IA models they help train).

So to your point... this is a very rigorous privacy protection. Another way to TLDR the article is "if we get really good at privacy... there's a LOT more data out there... so let's start really caring about privacy"

Anyway... I agree with everything in your comment. Just thought I'd drop by and try to lend clarity to how the article agrees with you (sounds like there's room for improvement on how to describe attribution-based control though).

With you on this one. I do think ABC is a step in the right direction to improve things. <3

"The claim that humans need petabytes of data to develop their mind seems completely indefensible to me."

And yet every human you know is using petabytes of data to develop their mind. :)

I'm relatively close to publishing my PhD thesis which is broadly a survey paper of what you're describing. Will share (almost done with revisions).

I think this is the right question to ask. I think it depends on the task. For example, if you want to predict whether someone has cancer, then access to avast amounts of medical information would be important.

This article is meant for a policy audience, so that does keep the technical depth pretty thin. It's rooted in more rigorous deep learning work. Happy to send your way if interested.

I agree with you in a way - that it seems likely that new data will be incorproated in more inference-like ways. RAG is a little extreme... but i think there's going to be middle grounds betweeen full pre-training and RAG. Git-rebasin, MoE, etc.

Yeah Zama's stuff is great.

Agree with you on the nuance.

(OP) YOLO

(OP) fwiw I fully agree with the privacywashing you're describing here, and this piece is advocating for a more rigorous standard than input privacy (homomorphic encryption), which is insufficient to enable data owners to actually retain control over their data (but is a useful ingredient).

This is the right question. If full attribution-based control is achieved, then this would be impossible. And the ingredient you've suggested could be a useful way to help achieve it.

This is the magic :)

(OP) the scaling laws / bitter lesson would disagree, but I tend to agree with you with some hedging.

If you get copies of the same data, it doesn't help. In a similar fashion, going from 100 TBs of data scraped from the internet to 200TBs of data scraped from the internet... does it tell you much more? Unclear.

But there are large categories of data which aren't represented at all in LLMs. Most of the world's data just isn't on the internet. AI for Health is perhaps the most obvious example.

The piece advocates for the opposite of this. Attrbution-based control keeps data holders in control.

Well, we're opining about a statement about the world. Is the universe only 200 terabytes of information?

"Biological lifeforms seem to be able to train/develop general intelligence from much, much less."

This statement is hard to defend. The brain takes in 125 MB / second, and lives for 80 years, taking in about 300+ petabytes over our lifetime.

But that's not the real kicker. It's pretty unfair to say that humans learn everything they know from birth -> death. A lot of that learning bias was worked out through evolution... which takes that 300+ petabytes and multiplies it by... many lifetimes.