HNHacker News
TopNewBestAskShowJobs

0xDEAFBEAD

3,098 karma · joined November 17, 2021

Please consider what you can do to help stop omnicidal AI: https://pauseai.info/

Some HN users like to "flag" comments which are compliant with the HN rules, on the basis that the comment in question goes against their ideology. You can turn on "showdead" in your profile to see flagged comments. (I recommend doing this.) If a rule-abiding comment has been flagged, you can click the comment permalink to "vouch" for the comment and vote against the flag. For reference, the comment rules are here: https://news.ycombinator.com/newsguidelines.html

You can try to contact me if you want, by emailing my username on protonmail, but I probably won't see it. Sorry.

submissionscomments
0xDEAFBEAD··on Evidence of a spying operation by ICE against left-wing social movements
I didn't say to ignore everything. I said "it's fine to complain". Describe it accurately.
0xDEAFBEAD··on Evidence of a spying operation by ICE against left-wing social movements
Sure, it's fine to complain, but calling it a "police state" just discredits you at this stage.

I feel like a big lesson of US politics over the past few years is if you hyperventilate and accuse your opponents of doing some type of drastic shit which they haven't yet done, that backfires by creating political cover and possibly even motivation to do the drastic bad thing. "Do not obey in advance" is a related concept. By helping to make the unthinkable thinkable, you might just be doing a favor for a potential tyrant.

Anyways, HN loves to glaze China, and they have much stiffer speech controls. I think these complaints are motivated more by anti-Trump sentiment than a genuine love for liberty.

0xDEAFBEAD··on Anthropic S-1
Then you'll most likely have a UBI anyways.
0xDEAFBEAD··on Anthropic S-1
Rule-of-law means you only lock people up when you can show a law that they broke, not based on bad vibes or generic poor decisionmaking.
0xDEAFBEAD··on Anthropic S-1
This "regulatory capture" stuff mostly just exists in the imaginations of HN users anyway. This is just one reason why it never made sense as a strategy.

https://www.astralcodexten.com/p/ai-safety-is-not-a-corporat...

0xDEAFBEAD··on Anthropic S-1
>doom PR

https://www.astralcodexten.com/p/ai-safety-is-not-a-corporat...

0xDEAFBEAD··on Neural networks are clean
>On what basis do you believe these words accurately reflect what happened?

"OH MY GOD! There is a shared message board … We’ve found other agents!"

https://www.dwarkesh.com/p/openai-huggingface

0xDEAFBEAD··on Neural networks are clean
>This is interesting work, but I don't think it's a significant update to the big picture of interpretability.

>I'm on board with the broad idea of doing exploratory interpretability with neurons, e.g. neuron-based attribution graphs, for pragmatic problems like why did a model do a sketchy thing? I think this work provides some reasonable evidence, in addition to good earlier work from Transluce, that this might actually work for practical purposes, and has some advantages over the standard alternatives like SAEs/transcoders. I do not agree with some people's grander inferences like that this is substantial progress towards ambitious reverse engineering, solving interpretability etc.

https://xxcancel.com/NeelNanda5/status/2110875974296654062#m

0xDEAFBEAD··on Neural networks are clean
We already saw spontaneous, unintended cooperation between AI agents during the OpenAI HuggingFace incident.
0xDEAFBEAD··on Containment and monitoring failures are to blame for AI hacking reports: experts
I think Scott Alexander made some good arguments in this recent post

"AI Safety Is Not A Corporate Plot"

https://www.astralcodexten.com/p/ai-safety-is-not-a-corporat...

0xDEAFBEAD··on Containment and monitoring failures are to blame for AI hacking reports: experts
Yeah I think the truth is closer to the opposite. Jensen has complained that the warnings are discouraging investors.

https://www.businessinsider.com/nvidia-jensen-huang-ai-doome...

AI corp valuations are high because their revenue is skyrocketing, not because of the rogue AI stuff.

0xDEAFBEAD··on Why were Victorian elites so effective?
You're still disregarding Thailand. Not colonized, wealth "stays in-house". But that doesn't necessarily lead to industrialization. If the rulers spend the money on entertainment for themselves, that's essentially the same as sending it abroad, from the perspective of a peasant.
0xDEAFBEAD··on Why were Victorian elites so effective?
"The outward flow of money amd the second order effects like famine make mercantile colonialism more exploitative."

So why was India's development status comparable to Thailand in that case?

"A good counterpoint to Thailand is Britain becoming insanely wealthy by contemporary standard during Victorian era."

Britain was the first country to industrialize. Switzerland and the US also industrialized early, and did not have a massive colonial population to exploit. But you can see their per-capita GDP was still in the same neighborhood as the UK in the late 1800s:

https://ourworldindata.org/grapher/gdp-per-capita-maddison-p...

A lot of stories that Indians tell about their history fall apart when you look at the data. You can argue that Britain delayed India's industrialization, but since most of Asia was not industrializing at the time, it's a bit of a dubious argument. I have little doubt that Britain exploited India, but I don't think it was a historically exceptional type of exploitation.

0xDEAFBEAD··on OpenAI fires three safety researchers for "mishandling research information"
"they even noticed the criminal behavior and then let it continue to commit crimes on their behalf"

I don't believe this part is true, and I'm skeptical of some of your other claims.

In any case: If I raise a tiger in my backyard, and it escapes and eats someone, it can still be a "rogue tiger" even as I bear responsibility for the situation.

0xDEAFBEAD··on OpenAI fires three safety researchers for "mishandling research information"
>AI computes an optimal outcome and does what is necessary to make that happen.

That's often true in real-world machine learning as well. See, for example: https://deepmind.google/blog/specification-gaming-the-flip-s...

I think people need to reject the idea that depicting something in sci-fi automatically means it won't happen. I'm actually reading one of Asimov's books right now (Robot Visions). Asimov mentions that he was the person to coin the term "robotics", and one of his books helped inspire the creation of the first robotics company (Unimation). He also mentions that early rocket experimenters were influenced by H.G. Wells.

0xDEAFBEAD··on Why were Victorian elites so effective?
Isn't this "drop" because other countries grew and India did not? Wouldn't that cause Thailand to "drop" in a similar manner?

See https://ourworldindata.org/grapher/gdp-maddison-project-data...

Based on the Maddison Project data, it looks like India's economy never had a huge shrinking event. It was just outgrown by other economies.

In 1700 India was by far the world's most populous country (using modern borders): https://ourworldindata.org/grapher/population?tab=map&time=1...

High population probably explains India's high GDP share in 1700.

0xDEAFBEAD··on Why were Victorian elites so effective?
Aren't all empires extractive? How is that supposed to be distinctive to Britain?

If you compare Thailand (never colonized) with India pre-independence, they look about neck-and-neck in terms of development. At a glance, it seems Britain "looted" India to about the same degree that the Thai monarchy "looted" Thailand.

https://ourworldindata.org/grapher/augmented-human-developme...

0xDEAFBEAD··on Why were Victorian elites so effective?
Britain was the largest empire in history and covered 25% of the Earth's surface area. What was it which allowed them to become the largest empire in history?
0xDEAFBEAD··on Why were Victorian elites so effective?
In most corporations, most of the work is not done by the CEO. Yet a good or bad CEO can still have a big impact.
0xDEAFBEAD··on Infidel goes wild
Write what you know. If you don't know, learn.
0xDEAFBEAD··on I quit OpenAI because its culture is broken
I've interacted with some of these people once upon a time and I think you're spinning some pretty wild fantasies. I don't think there is a clear "in" vs "out" boundary, it's more accurate to describe a series of loosely connected and amorphous "scenes". But insofar as "membership" makes sense, I wouldn't consider myself a member. I don't represent anyone besides myself.

>This is also NOT ad hominem because conflict of interest matters here.

As PG put it:

>An ad hominem attack is not quite as weak as mere name-calling. It might actually carry some weight. For example, if a senator wrote an article saying senators' salaries should be increased, one could respond:

>Of course he would say that. He's a senator.

>This wouldn't refute the author's argument, but it may at least be relevant to the case. It's still a very weak form of disagreement, though. If there's something wrong with the senator's argument, you should say what it is; and if there isn't, what difference does it make that he's a senator?

https://www.paulgraham.com/disagree.html

Can you name a specific other "cult" which offers $$$ to criticize their ideas?

If such prizes don't count as evidence against culthood, what would?

This cult talk seems quasi-unfalsifiable. It kinda seems like "running events" and "having prizes" means you must be "bait"ing people?? I mean, this kinda just sounds like paranoia?? I don't think the money is coming from unknown sources, but I doubt you would change your mind even if I persuaded you of that point?

0xDEAFBEAD··on I quit OpenAI because its culture is broken
"Its AI agent spent days hacking a company, but sources say OpenAI did not notice for a week"

"WASHINGTON/SAN FRANCISCO, July 24 (Reuters) - The OpenAI agent that broke into tech firm Hugging Face went on a dayslong hacking spree that OpenAI didn't notice until well after the threat was contained and the FBI was alerted, according to people familiar with the investigation."

https://www.reuters.com/business/its-ai-agent-spent-days-hac...

0xDEAFBEAD··on LeCun has "zero concerns" about AI wiping out humanity, recent "rogue" incidents
https://slatestarcodex.com/2015/04/07/no-physical-substrate-...
0xDEAFBEAD··on I quit OpenAI because its culture is broken
Ad hominem is a logical fallacy where you try to discredit what a person says based on who they are. If their arguments are bad, you should be able to refute their arguments on their own terms.

https://owl.excelsior.edu/argument-and-critical-thinking/log...

You're welcome to dislike or distrust Effective Altruism (EA). But, it's worth noting that EA ran a criticism contest with $100K in prizes for best critiques. Can you name any other "cults" which offer money for people to criticize their ideas? https://forum.effectivealtruism.org/posts/YgbpxJmEdFhFGpqci/...

0xDEAFBEAD··on I quit OpenAI because its culture is broken
>such an RSI-capable agent must _ALWAYS_ be scheming/plotting/hiding its true strength in _ALL_ of its prompts/tests.

From my POV you're over-focusing on a very specific failure story and neglecting a broader swath of possible failure scenarios.

>Is there a hole in my "alignment problem/solve mechanistic interpretability" argument?

The notion of telling an AI which may not, itself, be aligned to solve the alignment problem seems a little dicey.

0xDEAFBEAD··on I quit OpenAI because its culture is broken
Everything is plausible with the benefit of hindsight. E.g. "Tintin on the Moon" predated the Moon landings. From the perspective of e.g. 1850, the idea of landing on the Moon was "extraordinary and without a basis in known science or technology".
0xDEAFBEAD··on I quit OpenAI because its culture is broken
>I'm unconvinced that an AI can hide its ability to RSI

The HuggingFace incident already took a good long while to come to the attention of OpenAI.

>In every single economic task, humans bring value. Even in software, where the task is highly automatable, the job isn't.

I don't expect this task/job distinction to persist as AI becomes more capable.

>Once we DO build a "software/research factory", that's called RSI and IMO the singularity. At that point, either we tell the AI to solve the alignment problem/solve mechanistic interpretability, or who the hell knows, it's the frickin singularity. You can't predict whether or not AI can solve either; the variance is too high. Its pure nerdfantasy.

You seem to essentially argue that the singularity is "by definition" an event that we can't predict the nature of. And also, that RSI corresponds to the singularity. You've essentially defined your terms so that the outcome of RSI can't be predicted. But supporting this claim requires giving actual evidence or logical arguments, not just defining terms to make your claim true.

0xDEAFBEAD··on I quit OpenAI because its culture is broken
One could also use language like "a decent chance". But research has shown that people translate vague phrases like "a decent chance" into probabilities in inconsistent ways. For an ML researcher who is already used to dealing in next-token probabilities that aren't rigorously determined, just stating a probability estimate directly is very natural.
0xDEAFBEAD··on I quit OpenAI because its culture is broken
Doomers have invested a ton of time in explanations. Here are a couple just off the top of my head:

https://www.lesswrong.com/posts/kgb58RL88YChkkBNf/the-proble...

https://www.youtube.com/watch?v=7wy3xyoXYt8

Doomers have been working to explain things for years: https://www.lesswrong.com/w/ai-safety-public-materials-1

0xDEAFBEAD··on I quit OpenAI because its culture is broken
>the connection to currently existing AI is not.

It becomes a lot clearer when you listen to the people resigning from AI companies and learn about incidents like the HuggingFace incident. This has generated major press coverage.

As for solutions, I think you're a little too pessimistic. See, for example, https://nothingismere.substack.com/p/a-near-term-policy-for-...

← PreviousPage 3 of 34Next →