williamtrask

Fwiw - this post doesn't advocate for trust. It advocates for an enforcement mechanism (attribution-based control).

(OP) 100% and this piece advocates for an enforcement mechanism for that kind of payment (attribution-based control)

(OP Here) This is a fair point. Internal datasets can be deceitful just as public ones can. That said, most propaganda lives in the public domain. :)

(OP here) Homomorphic addition (e.g. aggregation) is very performant, including for the Federated Averaging algorithm used in Federated Learning. Not hand-waivey.

(OP here) — with you on that analysis. This was in an effort to make the piece legible for a (primarily) non-technical, policy audience. Rigorous numbers are in other parts of the piece (and in the sources behind them).

(OP here) I agree with this in spirit, but also it's hard to imagine the world can be fully described with 200 terabytes of data. There's a lot more good stuff out there.

But to your point, a crucial question in AI right now is: how much quality data is still out there?

As far as the impracticality, it's a great point. I disagree and have spent about 10 years working in the area. But that can be a post for another day. I understand and appreciate the skepticism.

IMO - this paper is right about a major contributing factor to hallucinations but wrong about the cause

LLM hallucinations are closer to a cache miss.

https://x.com/iamtrask/status/1964403351116009671

Nice comment! I had forgotten about metaphorical truth. Sent me on a nice rabbit hole.

I think 'metaphorical truth' is correct but slightly too narrow. Pragmatic truth includes metaphorical truth but is slightly wider.

And while I agree with your assertion in the short run, I'm inclined to doubt its correctness in the long run. Most things eventually have consequences.

If truth is defined as beliefs which lead one to make decisions that cause you/your society to thrive, this is a good thing (that the Old Testament has similarities to other major works). Implies a kind of evolutionary algorithm for truth. Likely implies these stories are more true because they’re more tried and tested. Societies who believed them became strong.

If truth is about repeated experimentation or journalistic records (a very new concept in history of writing - less than 500 years), then perhaps this is of concern.

I accept both definitions, but when they’re in conflict, the former tends to be more end-to-end, while the latter tends to overfit to the moment. Mostly because data is scarce and life is a very complex distributed system. On the other hand, the former changes slowly while the latter perhaps keeps up with the pace of change.

Except the point of life is probably to thrive more than to collect a list of facts. So when in conflict, I lean towards the former. Personal choice tho. I expect most of HN leans the other way.

Nit: the author says that supervised fine tuning is a type of RL, but it is not. RL is about delayed reward. Supervised fine tuning is not in any way about delayed reward.

hugged to death?

Claude is down.

EDIT: for the moment... it supports 0 tokens of context xD

Breakthroughs usually require a step-function change in data or compute. All the firms have proportional amounts. Next big jump in data is probably private data (either via de-siloing or robotics or both). Next big jump in compute is probably either analog computing or quantum. Until then... here we are.

Well... unless you know how to turn your brain completely off while being awake, you're probably always giving your attention to something. Consequently, attention and time both spend at very similar rates.

If this isn’t jumping the shark it’s darn close.

Feels like a portal to the alternate universe we could have with modern technology but no mass media.

> So yes, totally. Turn it off and go read a book.

For what its worth, 500 years ago people were just as worried about books as we are today about newsfeeds. But it took a long time for books to ultimately decentralize enough to become a more egalitarian, community knowledge. But even that's not entirely the case now. Books can be propaganda just like everything else.

Freedom of choice isn't that freeing if you don't have time/money to actually permute through all the options to find what you like.

This is the actual reason why the door is continually held open for propaganda and centralized control. Decentralizing everything struggles with inefficiency problems.

The first person to really discover and popularize this was Edward Bernays — who invented Public Relations to help corporations and politicians weaponize this inefficiency. He kicked off the "Mad Men" of 20th century New York.

The introduction to Bernays's book "Propaganda" lays this out very clearly:

https://www.voltairenet.org/IMG/pdf/Bernays_Propaganda_in_en...

Or if you don't like reading... another overview of Bernays is Adam Curtis's "Century of the Self"

https://www.youtube.com/watch?v=eJ3RzGoQC4s

We will keep giving up control to centralized forces until we can share information freely and efficiently about what choices lead to better/worse outcomes in a decentralized way.

Or just address the core element of advertising that creates the perverse incentive — the ability for an auction to determine what you see. Paying to be a part of a digital phonebook is fine. Recommending things is fine. But skewing your recommender against the highest bidder maybe not.

I love homomorphic encryption, but why can't they just do a neural search locally?

- iOS Photos -> Vectors

- Search Query "Dog photos" -> Vectors

- Result (Cosine Similarity): Look some dog photos!

iPhones have plenty of local storage and compute power for doing this kind of thing when the phone is idle. And cosine similarity can work quickly at runtime.

Forgive me, but with machine translation becoming nearly a solved problem — why would kids spend years of their lives learning new languages anymore? By the time they grow up, won't that be a rather useless skill — except perhaps in very nuanced contract negotiations?

I lead an open source nonprofit which deploys things like this. Feel free to shoot me a DM on Twitter. Handle is @iamtrask

> What's to be done?

Our society would need to re-think its norms around moving away in order to move upward, which runs into some pretty big philosophical ideas around geography, culture, and values. For technologists, this cultural movement is still in a bit of a philosophical stage, but it's starting to make its way into some early experiments. https://www.plurality.net/ is probably the best written work on it so far.

Yes but investing in new, long-term friendships is a bit of an oxymoron. In order to do that, you have to pick one set of friends and then stick with them (cuz opportunity cost — the day is only so long). And once you've done so, you then have to stop investing in new friends and stick with those old ones.

Cities aren't very conducive to this in part because... people in cities tend to be more transient and their relationships more transactional. You have so much more choice, and the people you know have more choice, and people come and go.

In order to invest in long-term friendships, you need to be in a location where both you and the people you're making friends with are actually going to stay, and where both sides of the relationship are invested in the long-term nature of it.

This is much more likely to happen in a rural area than a city. I'd wager that constraining choice creates intimacy far moreso than alignment of interests. It's not about how much you like each other — it's about high-enough switching costs to hanging out with someone else that keep you together over a very long period of time.

In a city, even if you find the most compatible friend ever... there's also going to be 50 people almost as compatible (and vice versa for the people you're hanging out with).

In a rural area, you might not find someone you're perfectly compatible with... but they're far more likely to be the only person in the area with interests that are that aligned... and that creates intimacy... which will only make you more like them and them more like you.

Having lived in both rural and urban settings, I'm not sure the isolation you describe would be as correlated as you think. People in rural areas still hang out with one another (and even... alot) and people in cities still isolate themselves.

To follow that line further, I'd argue that the fact that rural living gives you less choice in company increases the degree to which you invest in the relationships that happen to be around. Consequently, rural living can create a greater sense of intimacy and companionship than the bustle of a city where there's always a new face around the corner (if you get bored of the old faces).

For a nice book on the tech-philosophy of this question (recently endorsed by the Dalai Lama, you might enjoy: https://www.plurality.net/

"a larger model with RAG etc is still better than a small one"

This paper from DeepMind a few years ago offers a counter example to this claim.

https://arxiv.org/abs/2112.04426

Fwiw, I used to think this way too but LLMs are more RAG-like internally than we initially realised. Attention is all you need ~= RAG is a big attention mechanism. Models have reverse curse, memorisation issues etc. I personally think of LLMs as a kind of decomposed RAG. Check out DeepMind’s RETRO paper for an even closer integration.

> Belief in a higher power is not a substitute for finding goodness in what you do.

But it can be, namely because it’s one path to finding motivation to do good. Not saying it ends up there for everyone, but it certainly can be for some.

Note: netik’s bio indicates they used to work at Twitter.