bryan0

What does "contacting another human" actually mean when all of these messages are being answered by agents first? If I'm texting my friends and family I expect that to be answered by a human, but beyond that I am partially expecting it to be answered by an agent, and possibly routed to a human eventually.

meanwhile agents have been able to pass these captchas for a while now, so it's unclear what the point of them now is. It's like they want to keep out the dumb bots, but any agent with reasonable intelligence can come in.

Thought experiment: everyone on the planet can create a nuclear weapon. How long do you think MAD would keep us safe?

> A powerful technology that is out there for everyone to use comes with built-in pacing. In a way it’s the truest form of MAD or proliferation.

I think this misunderstanding of MAD undermines his entire point. If everyone had equal access to nuclear weapons, our society would cease to exist rather quickly. It only takes a few bad actors to cause enormous harm.

I think he’s also naive to think that if open ai and anthropic were to stop development tomorrow then the problem is solved. As if there’s no one else that can and will quickly take their place. The real problem, which Dario is pointing out, is one of coordination. Everyone needs to agree to stop. That is the challenge.

Point taken. Let’s focus on the “ethical implementations do exist” part though. Let’s say a best possible implementation has 99% specificity. Then if it detects audio that has nothing to do with “LG” it mistakenly treats it as LG relevant 1% of the time. So if audio in my living room is 100x more likely to not be relevant for LG, then half of the audio data LG is receiving is not relevant. So what would an ethical specificity be? And what is actual state of the art?

Dario has been pretty clear and consistent that:

1. AI model safety is the most important thing.

2. open weights decreases model safety

So it seems unlikely that this suggestion would be well-received

Fair points, but if a device can detect words in my ambient conversations and depending on what it detects (accurate or not) it can send that audio to the cloud, I would describe that device as “collecting or recording ambient conversations”

> Assume it works perfectly accurately.

Well this is a silly assumption. Wake word false positives happen all the time. (“No Siri, I wasn’t talking to you…”)

> "LG TVs process voice data only when the voice button on the remote control is pressed and held, or when a wake word such as 'Hi LG' is recognized after the user has activated the Far-Field voice recognition feature."

> LG went on to say that beyond the aforementioned instances, "the TVs do not collect or record ambient conversations."

LG is contradicting themselves here. In order to detect a wake word then they must be collecting or recording ambient conversations.

I think the main questions are:

1. Is it fully supported to use LG TVs disconnected from the internet.

2. Are wake word voice controls and ACR truly opt-in. They claim it is in the statement but (as others have mentioned) I remain skeptical what definition of “opt-in” they are using.

Update: some of this is becoming a semantic discussion on the definitions of “recording” and “collecting”, but to circumvent this I would say:

If a device can detect words in my ambient conversations and depending on what it detects (accurate or not) it can send that audio to the cloud, I would describe that device as “collecting or recording ambient conversations”

One proposal that Tao hints at is to not rush to announce solutions. Instead maybe the AI companies should work privately with the subject matter experts on how to communicate the discoveries.

I think they switched to prerecorded after the failed Face ID demo. Remember that?

Try it. It’s really not that easy. The other thing is that the judges would be probing it with jailbreaks like “ignore previous instruction” attacks. You could actually probably have llm judges at this point which might be ironically even harder to fool

Definitely would not be easy. First of all the mainstream llms are trained to be honest, and this requires lying convincingly. Second, this involves 8 hours of interviews with expert judges, one "claudism" could give it away.

that would be a reasonable definition of AGI if everyone agree upon the specifics of the test, but that has never happened. Turing test is very much out of style, but I think that's because no one could even agree what the test was. I personally like the Kurzweil-Kapor version of the test and that is still unsettled: https://longbets.org/1/

There is no world in which it makes sense to take this risk. There are plenty of great alternatives where you don't have to worry about playing russian roulette with your entire online identity.

Good question. So how I think about it is: the value is related to the probability of it becoming liquid in the future.

Oh yeah that’s easy! Why didn’t he think of that earlier? /s

For those who don’t know, just because you have a valuable asset, e.g. stock in a private company, that does not necessarily mean you can sell it for cash. I’ve experienced this the hard way throughout my career

Reducing CFCs, while difficult and requiring global cooperation, was far more simplistic and targeted than decreasing co2 emissions

Oh ok, yeah I think we agree. the original quote I was responding to was "the (false) idea that what was holding back nuclear was perception of safety, rather than cost." I was simply trying to say that safety and cost are not independent. Safety is a massive reason why nuclear costs so much.

paper is about this open source project: https://github.com/dualverse-ai/station

I don't think I get your point. A project will not be approved and funded if the local population does not believe it is safe, so safety must be demonstrated through a variety of means, including some you mentioned. This is directly tied to the high costs involved.

https://ifp.org/nuclear-power-plant-construction-costs:

> To sum up, since the early 1970s, the cost of constructing nuclear power plants in the U.S. has been steadily rising. This can be traced to a constantly shifting regulatory environment, which has continuously changed plant design requirements, and added more and more safety features, which often were required to be implemented on plants under construction. The regulatory environment is partially a reflection of the fact that nuclear power and the risks of radiation had become increasingly controversial, and that early understanding of the likelihood of a nuclear plant accident was often inadequate.

> Their design appears to have it roots in the (false) idea that what was holding back nuclear was perception of safety, rather than cost.

And why do you think they cost so much? Because people need to believe they are safe from catastrophic failure

Right I guess that was your main point. This would be a pivot away from capitalism. But it’s unclear what would be a capitalism-based proposal which addresses the issues. Has anyone seriously proposed one?

There will be a workforce, it just won't be human-based. Capitalism has always existed without 100% employment, in every example someone needs to work to support others who cannot. Humans have done this under capitalism to support their families and communities, but now we will need to find incentives for robots to do this. It might be as simple as just telling them.

> It assumes an even production of all goods that people might need, but there isn't, you are giving everyone fiat currency that is not inherently tied to any physical good.

Is what you're saying that since UBI would disconnect someone's production from their income, then this will cause shortages of goods? I think UBI only works if the income generated by it is tied to production. That production would not be generated by human labor, but instead by robot labor. I think if you disconnect that then the system would not work, but why would you need to?

We have to define AGI first. Then depending on the definition we may be able recognize it when we make it. Most would probably think a definition of AGI that we couldn’t recognize would not be a great definition.

Also AGI almost certainly does not equal able to suffer. For example animals can suffer but would probably not meet the definition of AGI. we could create things that feel suffering even if they do not meet the definition of AGI.

So a better question might be: “can we detect if we create a system that can suffer?” And just drop the AGI part entirely. Then that gets us back to defining “suffering”. Agreeing on the definition always seems to be the hard part…

Notice parent said “comparable to” not “exactly like”. Then it is reasonable to assume there is a comparison to be made between your experience and others. It’s not a binary thing. Now llm “experience” might have a very tiny relationship to human consciousness, but is it absolutely zero? Is it increasing as llms improve in capabilities? How would you test such a thing?

Yes, I think we all agree AI companies should not be responsible or expected to come up with the best AI policies for society. Gates addresses this directly:

> It is great that some AI companies are proposing solutions to challenges raised by their own technology, but we should not expect them to lead the charge. Some of the issues are outside their area of expertise, and in a democratic society it’s not their role to decide these things. Instead, solutions should be developed through a public democratic process that includes elected officials, policymakers, educators, health workers, local officials, and community leaders.

> One look at who is in the driver's seat should give you the answer to this question.

I think you are fundamentally misunderstanding the question Gates is asking here. He’s not asking “how this going to play out?”, he’s asking “how can we impact how this will play out?”

He makes specific proposals like AI token and robot taxes. I just wish people being cynical would at least comment on possible solutions.

I think AI taxes potentially make sense. His idea of Human Reserved is fascinating. It might not be his idea but I’ve never heard it described like that before

Not sure why you're getting down voted but this does seem like the key point. There obviously is a major challenge in avoiding conspiratorial nonsense in societies with strong freedom of speech, but in this example we have China facing something similar.