williamtrask

But they don’t work very well at temp 0. Better to seed your random numbers.

Food for thought: randomness is in the eye of the beholder. It’s only random to you if you don’t know how to predict it. So can a mind ever truly be stochastic? Perhaps to the minds of others, but never to itself.

Fake isn’t the right word here. The synthetic images contain real structure useful for learning.

Lol. Probably worried it would reproduce and become a paperclip maximiser.

> there are no physical or info-theoretical limits that superhuman intelligence must require a megawatt-sized compute cluster and all the data on the internet (which obviously no human could ever see).

Much of your "intelligence" is a function of natural selection. This is billions of years X bajillions of creatures in parallel, each processing tons of data at a crazy fast sampling rate in an insanely large/expensive environment (the real world). Humanity's algorithm is evolution moreso than the brain. Humans learn for a little while, start unlearning, and then die — which is an important inner for loop in the overall learning process.

Taken together, there is some evidence to suggest that superhuman intelligence must require megawatt-sized compute cluster and all the data on the internet (and a lot... LOT more)

> “Motivated by this limit, we investigate scaling language models in data-constrained regimes. Specifically, we run a large set of experiments varying the extent of data repetition and compute budget, ranging up to 900 billion training tokens and 9 billion parameter models. We find that with constrained data for a fixed compute budget, training with up to 4 epochs of repeated data yields negligible changes to loss compared to having unique data. However, with more repetition, the value of adding compute eventually decays to zero.”

Which goes to show you one of the best strategies in ML isn’t necessarily thinking about learning theory, it’s thinking about what our fastest computers are most capable of doing.

But this does run out of steam (or rather…data) eventually. The largest LLM was ibm’s tangora model back in the 1980s. 8 trillion params.

We’ll go through the cycle again with a new paradigm.

Paper you’re looking for is called “Attention is all you need”

One more step towards digital ID

I believe so - Narrow AI. It seems to be much easier to build than generalist models. Think all the protein folding, game playing, image classifying, machine translating, image captioning, super-intelligent AIs of the last decade. It’s not clear we really need super general models. Even LLMs can be topic specific.

Fun fact - a good percentage of YouTube and Spotify views are fake.

Fwiw - I’m not sure it’s wise to ground one’s moral philosophy from mass murderers. The invisible hand in economics seems both more causal to technology’s advancement and less dubious.

Also there’s a fantastic BBC docuseries called “Connections” by James Burke that makes the case that nothing in tech could have happened sooner or later than it did, and that each progression was an inevitable next step from the previous. It has interesting implications.

> Also it's not like people are making a choice between whether they meet friends or install Calm.

Neither meeting friends or using an app are binary events. The more you spend time on one, the less you spend time doing anything else — in the limit including the other. There's considerable evidence to suggest app and/or media usage encroaching on in-person time. It seems plausible that people who can band-aid their feelings in an app may be less inclined to take the leap and talk about personal matters with friends.

But these are complex things - there would need to be a study to know for sure.

Fwiw most of the academic labs working on AI safety also work on these threats. Future of Humanity Institute, Center for the Study of Existential Risk, etc.

Although in general there’s a pretty big information asymmetry between a multi billion dollar car company and a boomer dad w/ a Tesla. So whether the latter is usually better off may be debatable.

For those who don’t know about the author, he is most well known a founder of the field of virtual reality. As a full time philosopher in the CTO’s office at MSFT, and the originator of concepts like data dignity, he’s uniquely qualified to write this kind of piece.

https://en.m.wikipedia.org/wiki/Jaron_Lanier

This ideology of “replace corrupt institutions with tech” is at the center of SV ideology going back to the 70s. It’s as flawed then as it is now, but it occasionally has wins.

There’s a great documentary on the long history of this idea in tech, both wins and losses, called “all watched over by machines of loving grace”. Brilliant documentary.

There's no free lunch when it comes to learning signal. The right comparison here is between training on synthetic data and training on the data used to create the synthetic data generator.

Synthetic data doesn't create more learning signal, it just creates more data repeating the same learning signal with some bias assumptions that might as well be in your final model.

The difference is moreso in training times for retraining new models. If you've taken highly disparate data and compressed it into a particularly "juicy" dataset which is smaller, that could bring down training times.

That is to say, it's related to curriculum learning moreso than breaking information theory.

What you're referring to is society's attempt to re-build itself based on rationality and individualism as a replacement to political and religious institutions. A canonical response to your point would be that we're seeking to build a society where the rational action would be for the head of NASA further the aims of NASA. This follows from an ideology set out by Ayn Rand. There are a collection of documentaries by Adam Curtis covering the arc of this philosophy: "All watched over by machines of loving grace." is probably the main one here, but "Century of the Self", and "Can't get you out of my head" are also very good.

Naturally, the problem with society's attempt to do this is that rationality is not sufficient for the head of NASA to not go against her duty as NASA's head. A utopia of radical individuals free from any ideology or institution hasn't (yet?) been realized. It's probably not a good idea, but it's the central dogma of silicon valley, the blockchain movement, the AGI movement (especially the sentience-heavy ones), much of the modern scientific community, and (as it happens) hacker news. So here we are.

Also, your logic holds that swearing on a Bible is — in the limit — more likely to lead to moral behavior than swearing on Carl Sagan's work. Nobody believes that Carl will punish wrongdoers (he might even reward them for being rational, or face questions on why he's pushing morality on others). However, people do believe that God punishes. So regardless of whether a deity exists, swearing on religious texts is more likely to alter behavior than on Sagan's.

The first image I loaded looked just like Jack Nicholson, the second image I loaded showed me a picture of Justin Bieber.

Can confirm that PySyft is currently in the process of integrating with Flower. Best of both worlds.

I can’t speak for Flower’s core dev roadmap, but PySyft is in the process of integrating Flower and some Secure Enclave options which would let you do this.

Congrats on the launch Flower team!

Copy paste has been cheap for a long time.

Detecting whether something is written by an AI is a waste of time. Either someone will sign the statement as their own or they won't (and it should be treated as nonsense).

People lie. People tell the truth. Machines lie. Machines tell the truth. I bet our ability to detect when a person is lieing isn't any better than 50% either.

What matters is accountability, not method of generation.

There are plenty of products where people choose marketing even though demo and explanation are possible. Most.

EY is not rare, but he is famous by proximity to powerful/wealthy actors. There are many philosophers, likely more than ever in history. Aside from professional philosophers, there’s a large pipeline of PhDs “doctor of philosophy” who work at a high enough level of abstraction in their own field to be considered a philosopher.

Forgive me but I think this may be a contradiction. “didn’t adequately account for duration risk” == “risk taking that did them in”

Transparency of algorithms is more enforceable than one might think. Check out “Structured Transparency” tooling.

I find it intriguing that this kind of project will plausibly be the true legacy of the crypto community’s efforts. This is both ironic and tragic as their intentions were the opposite in the wake of the 2008 crash.

I think it’s also related to the lack of trained political scientists in the crypto movement. A first year undergrad is taught that real political power comes from whomever has a monopoly on violence. In this light crypto was always doomed to fail in this way. It had little to no affect on the underlying real power. It only worked 1 later up (monetary supply / taxation / etc.).

Something worth thinking on. It’s oversimplifying to say that this is merely a case of bureaucratic greed for power (which seems to be the main theme amongst comments).

Kids are being harmed and are harming others online. Parents (imperfect as they may be) have far less ability to intervene than they do in physical spaces. Technologies (and technologists) are a major cause of this autonomy.

If we build technology that leads to harm, it increases the chances that a law like this is passed. Doesn’t matter if we meant to. The primary role we have to prevent legislation like this is to think creatively on how what we build can help create a better world in many aspects that are largely overlooked. Otherwise we’re feeding the political capital of bills like this.

Want to preserve the current internet? Don’t settle for the current societal impacts. They’re unacceptable and politicians will have to respond, and will do so the only way they know how.