simianwords

Yeah this I agree because even personally all my rubrics break when a new model is released. Things were stable while I was using 5.6 GPT. There ought to be some room to explore and understand this intuition and yes one must account for this bugdet.

Disagree with this because we have ways to steer price use per task.

1. choose a good model

2. choose the appropriate reasoning effort

3. choose a prompt to nudge it even further

Then it comes down to understanding the intuition of what kind of task deserves what effort?

what is the most evil corporation? Microsoft?

Lots of EA criticisms come from literal extremists who consider the mere aspect of earning money in free market itself to be evil. EA is fine and has issues but it needs critic from more normal people.

Interesting point. There need to be more smart people who take ai seriously while also not taking x risks seriously.

What terrible things has he done and what accusations? I'm going assume I won't get any reply to my comment

Here we go again, the industry largely gave on up RAG. In fact I have hardly seen any case where grep doesn't work as well as RAG.

> I cannot even deign to imagine someone trusting that company with an iota of their personal information, or that it would do anything constructive with it.

?

On the other part of your comment: well, agree to disagree. Its probably just a good product and not everyone's hyper sensitive to privacy concerns.

come on man have a bit of theory of mind. people just don't care that much about privacy and Meta has a half decent product. Personal agents are here to stay and they are the future. It works and obviously its going to permeate.

I see. Gary Marcus said that AI won’t be able to make a coffee in any arbitrary home kitchen.

I think it’s a good test and I think LLMs will reach it in 3 years. Current benchmarks maybe slightly incorrect.

I’m happy to make a 4:1 bet in my favour that I’m correct about the kitchen bet.

Sure the climate change space is as diverse as the x risk space. Both have a continuum of complete extinction and tail risks and complete collapse vs significant collapse.

For example significant climate change activists argue that collapse is imminent if we don’t do something about emissions.

Yes I do think tail risks are mostly not true for climate change. And I think it’s the same for AI risks.

I’m not sure what you are trying to say

I don’t understand your point. You say that tools are the only things that are required. But then you say that the quality of tools don’t matter.

I think what you are trying to say is that “give the LLMs access to information and it can figure out how to use it. Don’t get fancy about how to structure the data”.

If that’s the case it has largely turned out to be true. RAGs have fallen out of fashion.

Where does that put AGENTS.md though? Is it worth spending time to structure it or just dump it.

I actually agree with you. Verifiable domains will have better performance.

But there’s nothing specially bad about LLMs that don’t allow it to work outside of its training set. It’s just that biology has to verify itself in physical realm and it’s a bit slower.

So yeah, I also don’t think some bad actor will find the secret to manufacturing a bio weapon using LLMs. But maybe these people think it’s possible. I’m skeptical but I’m going to also listen to the people who know it best.

What’s very scary is that the type of people to downplay x risks are the exact people who have the absolute WORST track record like Timnit and Bender and Gary Marcus and Yann LeCun. At least Yanns only mistake was to not be bullish on language models. Timnit and Bender are not just wrong, they are fundamentally bad faith people.

The people who got predictions right on the other hand are people like Yud and Nate Soares who think x risks are real.

So here you have people with a good track record, who know how things work and they seem to suggest x risks are real. What do we take from this?

Humans are addicted to it. It’s just that last 30 years the mode was climate change and Thunbergesque rhetoric. Now it is AI xrisk. We humans love a crisis story.

There are lots of juicy parallels between climate risk and AI x risk. Both claim non trivial chance of human extinction and pause economic activity for it.

This take belongs in 2024. It has been falsified multiple times but it never seems to go away.

we have benchmarks proving it can do spatial reasoning.

hmm my conviction that labs have high margins

Extremely funny that GPT 6.1 sol is not only cheaper but also more efficient and faster and more performant and therefore better for environment. So they give up on all this just for data sovereignty.

not beating the allegations

Yeah but are you willing to pay additional cost for marginal improvement in accuracy that only pays off sometimes? If so, take the judgement and use a better model.

If consumers are required to pay: nooo i want free stuff

If the free stuff is subsidised by ads: noo advertisements are bad!!

what happens to the aistudio.google.com thing? :)

Looks like Google Maps said the location was open but there was some notice that said it would be closed and Google Maps didn't account for it? You'd have made the same mistake anyway

My take was that we didn’t have these 4 things

1. Someone with enough conviction of end goal which is agi

2. That someone with enough capital or agency to pursue it

3. Enough capital to explore all options - at that time we didn’t know about transformers

4. Enough smartness to know what ideas to not purse and which ones to prioritise

As things got cheaper, it allowed more people to explore.

Yeah the fork in our disagreement is on trusting human judgement vs metrics.

My position is that if you don’t trust a human to use metrics in good spirit, then you also may not trust them without metrics. But agree to disagree I guess. I do buy your point that metrics give the excuse of “hey that’s what you asked!”

Commodify the complement?

No yes, I agree with you but frankly I'm tired of the cliche that you can't measure PRs or LOC when in reality it is very much correlated with whatever you want to optimise. I do think it can be gamed, but relying on vibes and human judgement can also be gamed (which is what I tried to point out).

You are completely right that the VP can measure user satisfaction but here's the thing: that feedback loop has a much longer time period. You could also measure your company by revenue or its stock valuation. But the point is to have metrics that have a shorter latency. How can you achieve it? Its a hard problem to solve.

What’s the alternative to this? Without having a fallback to human driven judgement that itself can be gamed.

Maybe you’d suggest that the VP must ask all their direct reports to verify it manually. Then you rely on each person below you to have good judgement and also act in good faith. The director asks the managers who asks the leads who may or may not give accurate reports.

It’s not clear that’s any better?