peterlk

Reminds me a bit of the coolest talk I ever got to see in person: https://youtu.be/FITJMJjASUs?si=Fx4hmo77A62zHqzy

It’s a derivation of the Y combinator from ruby lambdas

It’s because they’re playing with monopoly money. If you leverage all the money on itself repeatedly, you can make the numbers look insane. Then, you hand pieces of the leveraged money (stock options) to a bunch of executives who will be watched very closely. You don’t sell stock that’s going to go up in value, so they have to be very careful about when and how much they sell. If they sell too much, then the facade can break, and the leverage evaporates.

To be clear, there is real, underlying value & revenue. But there’s a lot of froth right now

My favorites are the micrograd series by Andrej Karpathy on youtube [0], and “Why Deep Learning Works Unreasonably Well” [1]

The greats on youtube are also worth watching: 3B1B, numberphile, etc.

[0] https://youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9Gv... [1] https://youtu.be/qx7hirqgfuU?si=8zmrbazuvnz379gk

Attention is all you need…?

The short answer, as far as I’m aware, is that no one really knows. The longer answer is that we have a lot of partial answers that, in my mind, basically boil down to: model architectures draw a walk through the high dimensional vector space of concepts, and we’ve tuned them to land on the right answer. The fact that they do so consistently says something about how we encode logic in language and the effectiveness of these embedding/latent spaces.

Modern AI is a miracle. The math that makes it work is beautiful and really impressive. For example, if you wanted to map all knowledge on earth, how would you do it? AI answers that question by building a high dimensional vector space of embeddings, and traversing that space moves you through a topology of basically every concept that humans have.

Or another thought; why is it that a stochastic parrot can solve logic puzzles consistently and accurately? It might not be 100%, but it’s still much better than what you might expect from a markov model of ngrams.

Openclaw is only sort of interesting. How to vibe code your first product is uninteresting. Claims about productivity increase from model usage are speculative and uninteresting. Endless think pieces on the effects of AI slop are uninteresting. There’s a lot of hype and grift and bullshit that is downstream of this very interesting technology, and basically none of that is interesting. The cool parts are when you actually open the models up and try to figure out what’s going on.

So no, I’m not bored of talking about AI. I’m not sure I ever will be. My suspicion is that those who are bored of it aren’t digging deep enough. With that said, that will likely only be interesting to people who think math is fun and cool. On the whole, AI is unlikely to affect our lives in proportion to the ink spilled by influencers.

I don’t think this problem is that hard to solve, it just requires political will that doesn’t exist. The solution is to make it the platform’s problem. If the platform doesn’t want to deal with fraud, they don’t get to operate in that jurisdiction. Sue them into submission. If they don’t care about that geography, then there is now a gap in the market for a more local business to fill.

You have to read it all the way through. It’s a pretty hefty investment, but the series is truly a masterpiece. I had to read the whole series twice to feel like I was actually starting to understand some of the symbolism. I don’t blame people for not being able to get into it; it’s dense. But it’s so epic and there is so much symbolism and philosophy packed in.

I have quite a bit of family in Germany, and have had several friends move from the US to Europe. Europe absolutely knows that they have an opportunity to capture a ton of talent right now. If you have skills that are in demand, basically any country in the Schengen zone will find a way to get you a visa. For example, if you’re a trans researcher, you will find open arms at academic institutions in Europe.

You could also lie and claim your address as a US address, and then just live in another country. This is obviously illegal, but I’ve met a few people who made it work for a while. But I’m also speaking abstractly on the internet, so maybe I’m just making all this up.

Yes. Other humans are generally accepting of mistakes below some frequency threshold, and frontier models are very robust in my experience

I think this article conflates (at least) two problems.

The first is that very few people (especially rich people) are anonymous. A motivated person who has had a psychotic break can be very dangerous, and if you’re even a little bit famous, the probability of that happening to you goes up substantially.

The second issue is the one that everyone is getting riled up about - wealth inequality.

These are distinct issues, and I think it does harm - in the form of polarization - to not explicitly call them out.

Shameless plug. We’re working on something like this. thismachine.ai. It’s still early, but interested to get feedback. The slack/chat part is still behind a feature flag. Let me know if you want to use it

The solution is parents! Stop making your bad parenting my problem!

I have been having this conversation more and more with friends. As a research topic, modern AI is a miracle, and I absolutely love learning about it. As an economic endeavor, it just feels insane. How many hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories, etc. could we build with the money we’re spending on pretraining models that we throw away next quarter?

I see value here. Firstly, it’s a fun toy. This isn’t that great if you care about being productive at work, but I don’t think fun should be so heavily discounted. Second, the possibility of me _finally_ having a single interface that can deal with message/notification overload is a life-changing opportunity. For a long time, I have wanted a single message interface with everything. Matrix bridges kind of got close, but didn’t actually work that well. Now, I get pretty good functionality plus summarization and prioritization. Whether it “actually works” (like matrix bridges did not) is yet to be seen.

With all that said, I haven’t mentioned anything about the economics, and like much of the AI industry, those might be overstated. But running a local language model on my macbook that helps me with messaging productivity is a compelling idea.

My dad has some stories of working in Burkina Faso (and Mali, and other countries) with a drone, and having to appease locals about his witch-bird. A lot if places in Africa still prosecute witchcraft.

This argument does not make sense to me. If we push aside the philosophical debates of “understanding” for a moment, a reasoning model will absolutely use some (usually reasonable) definition of “user harm”. That definition will make its way into the final output, so in that respect “user harm” has been considered. The quality of response is one of degree, the same way we would judge a human response.

This is a reductionist perspective that is unhelpful. Does buying a water cooler in the office increase profit margins? What about a coffee machine? Across a wide portfolio of decisions, a business does need to be profitable. However measuring the individual impact of single vendors is often a very difficult task.

How do you measure developer productivity? Code quality? Developer happiness? As far as I know, no one in the industry can put concrete numbers to these things. This makes it basically impossible to answer the question you pose.

The link is highly relevant to the executive order because this executive order attempts to place limitations on what laws US states can create.

I’m working my way through “The End of the World is Just the Beginning”, and the main thesis is that everyone is preparing for demographic collapse. Global populations are declining almost everywhere, and this breaks the current global order. For example, what does the Chinese economy look like when all the people subject to the one child policy retire? What are the knock on effects of labor becoming more expensive almost everywhere? Can immigration solve this problem? What about the cultural friction of mass immigration? What happens to the places that everyone emigrates from?

The book basically argues that a significant amount of the world is headed for destabilization, and a destabilized world involves a lot less trust.

Side note: personally, I find the writing style and general tone to be hyperbolic, but some of the analysis is interesting.

I am so, so, so tired of hearing this argument. At a minimum, AI provides efficiency gains. Skilled engineers can now produce more code. This puts downward pressure on jobs. We’re not going to eliminate every software engineering job, but the options are to build more software or to hire fewer engineers. I am not convinced that software has a growing market (it’s already everywhere), so that implies downward pressure. The same is true for customer support, photography, video production (ads), paralegal work, pharma, and basically any job that involves filing paperwork.

Eliminating jobs has absolutely happened. How many jobs exist today for newspaper printing? Photograph development? Film development? Call switchboard operation? Technology absolutely eats jobs. There have been more jobs created over time, but the current economic situation makes large scale jobs adjustment work less well.

LLMs can now detect garbage much more cheaply than humans can. This might increase cost slightly for the companies that own the AIs, but it almost certainly will not result in hiring human reviewers

You’re absolutely right!

Does this target any particular area? Or does it make people more hairy everywhere? If it targets a particular area, can anyone explain to me how scientists discover such specific pathways?

Counterargument: I can learn things at record speed (for me) because I can learn things in the order that makes sense to me. I find it much more motivating to start with: “why is AI bad at playing video games?”, than to start with “what is the chain rule?”

I will certainly need to learn about the chain rule eventually, but I find that I get lost in the details (and unmotivated to continue) without an end goal that is interesting to me.

AI loves to make vibe charts (sometimes with lots of extra steps), but that’s part of the process. It is nontrivial to wrangle LLMs through larger projects, and people need to learn that too

Of course! We still have computers the size of mainframes that ran on vacuum tubes. They are just built with vastly more powerful hardware and are used for specialized tasks that supercomputing facilities care about.

But it has the potential to alter the economics of AI quite dramatically

Because, similar to the US, they have authoritarian tendencies - strong nationalism and anti-immigration. How are you going to round up the bad people if you don't have surveillance everywhere?

> We also discovered an extra-pair-paternity event in Beethoven’s paternal line

I’m not a Beethoven scholar. Is this a novel discovery?

This is not true any more. The vaccine has been shown to lower cancer risk for those who already carry the virus, so it is recommended even for people who are HPV positive

As a side note, I find it really interesting and a bit disheartening to follow the history of “web 3”. The original “web 3” was a beautiful vision including the semantic web and its cousins. The term was hijacked by cryptobros through what could have been a genuine or malicious misunderstanding of the original web 3