3,098 karma · joined November 17, 2021
Some HN users like to "flag" comments which are compliant with the HN rules, on the basis that the comment in question goes against their ideology. You can turn on "showdead" in your profile to see flagged comments. (I recommend doing this.) If a rule-abiding comment has been flagged, you can click the comment permalink to "vouch" for the comment and vote against the flag. For reference, the comment rules are here: https://news.ycombinator.com/newsguidelines.html
You can try to contact me if you want, by emailing my username on protonmail, but I probably won't see it. Sorry.
I feel like a big lesson of US politics over the past few years is if you hyperventilate and accuse your opponents of doing some type of drastic shit which they haven't yet done, that backfires by creating political cover and possibly even motivation to do the drastic bad thing. "Do not obey in advance" is a related concept. By helping to make the unthinkable thinkable, you might just be doing a favor for a potential tyrant.
Anyways, HN loves to glaze China, and they have much stiffer speech controls. I think these complaints are motivated more by anti-Trump sentiment than a genuine love for liberty.
https://www.astralcodexten.com/p/ai-safety-is-not-a-corporat...
"OH MY GOD! There is a shared message board … We’ve found other agents!"
>I'm on board with the broad idea of doing exploratory interpretability with neurons, e.g. neuron-based attribution graphs, for pragmatic problems like why did a model do a sketchy thing? I think this work provides some reasonable evidence, in addition to good earlier work from Transluce, that this might actually work for practical purposes, and has some advantages over the standard alternatives like SAEs/transcoders. I do not agree with some people's grander inferences like that this is substantial progress towards ambitious reverse engineering, solving interpretability etc.
https://xxcancel.com/NeelNanda5/status/2110875974296654062#m
"AI Safety Is Not A Corporate Plot"
https://www.astralcodexten.com/p/ai-safety-is-not-a-corporat...
https://www.businessinsider.com/nvidia-jensen-huang-ai-doome...
AI corp valuations are high because their revenue is skyrocketing, not because of the rogue AI stuff.
So why was India's development status comparable to Thailand in that case?
"A good counterpoint to Thailand is Britain becoming insanely wealthy by contemporary standard during Victorian era."
Britain was the first country to industrialize. Switzerland and the US also industrialized early, and did not have a massive colonial population to exploit. But you can see their per-capita GDP was still in the same neighborhood as the UK in the late 1800s:
https://ourworldindata.org/grapher/gdp-per-capita-maddison-p...
A lot of stories that Indians tell about their history fall apart when you look at the data. You can argue that Britain delayed India's industrialization, but since most of Asia was not industrializing at the time, it's a bit of a dubious argument. I have little doubt that Britain exploited India, but I don't think it was a historically exceptional type of exploitation.
I don't believe this part is true, and I'm skeptical of some of your other claims.
In any case: If I raise a tiger in my backyard, and it escapes and eats someone, it can still be a "rogue tiger" even as I bear responsibility for the situation.
That's often true in real-world machine learning as well. See, for example: https://deepmind.google/blog/specification-gaming-the-flip-s...
I think people need to reject the idea that depicting something in sci-fi automatically means it won't happen. I'm actually reading one of Asimov's books right now (Robot Visions). Asimov mentions that he was the person to coin the term "robotics", and one of his books helped inspire the creation of the first robotics company (Unimation). He also mentions that early rocket experimenters were influenced by H.G. Wells.
See https://ourworldindata.org/grapher/gdp-maddison-project-data...
Based on the Maddison Project data, it looks like India's economy never had a huge shrinking event. It was just outgrown by other economies.
In 1700 India was by far the world's most populous country (using modern borders): https://ourworldindata.org/grapher/population?tab=map&time=1...
High population probably explains India's high GDP share in 1700.
If you compare Thailand (never colonized) with India pre-independence, they look about neck-and-neck in terms of development. At a glance, it seems Britain "looted" India to about the same degree that the Thai monarchy "looted" Thailand.
https://ourworldindata.org/grapher/augmented-human-developme...
>This is also NOT ad hominem because conflict of interest matters here.
As PG put it:
>An ad hominem attack is not quite as weak as mere name-calling. It might actually carry some weight. For example, if a senator wrote an article saying senators' salaries should be increased, one could respond:
>Of course he would say that. He's a senator.
>This wouldn't refute the author's argument, but it may at least be relevant to the case. It's still a very weak form of disagreement, though. If there's something wrong with the senator's argument, you should say what it is; and if there isn't, what difference does it make that he's a senator?
https://www.paulgraham.com/disagree.html
Can you name a specific other "cult" which offers $$$ to criticize their ideas?
If such prizes don't count as evidence against culthood, what would?
This cult talk seems quasi-unfalsifiable. It kinda seems like "running events" and "having prizes" means you must be "bait"ing people?? I mean, this kinda just sounds like paranoia?? I don't think the money is coming from unknown sources, but I doubt you would change your mind even if I persuaded you of that point?
"WASHINGTON/SAN FRANCISCO, July 24 (Reuters) - The OpenAI agent that broke into tech firm Hugging Face went on a dayslong hacking spree that OpenAI didn't notice until well after the threat was contained and the FBI was alerted, according to people familiar with the investigation."
https://www.reuters.com/business/its-ai-agent-spent-days-hac...
https://owl.excelsior.edu/argument-and-critical-thinking/log...
You're welcome to dislike or distrust Effective Altruism (EA). But, it's worth noting that EA ran a criticism contest with $100K in prizes for best critiques. Can you name any other "cults" which offer money for people to criticize their ideas? https://forum.effectivealtruism.org/posts/YgbpxJmEdFhFGpqci/...
From my POV you're over-focusing on a very specific failure story and neglecting a broader swath of possible failure scenarios.
>Is there a hole in my "alignment problem/solve mechanistic interpretability" argument?
The notion of telling an AI which may not, itself, be aligned to solve the alignment problem seems a little dicey.
The HuggingFace incident already took a good long while to come to the attention of OpenAI.
>In every single economic task, humans bring value. Even in software, where the task is highly automatable, the job isn't.
I don't expect this task/job distinction to persist as AI becomes more capable.
>Once we DO build a "software/research factory", that's called RSI and IMO the singularity. At that point, either we tell the AI to solve the alignment problem/solve mechanistic interpretability, or who the hell knows, it's the frickin singularity. You can't predict whether or not AI can solve either; the variance is too high. Its pure nerdfantasy.
You seem to essentially argue that the singularity is "by definition" an event that we can't predict the nature of. And also, that RSI corresponds to the singularity. You've essentially defined your terms so that the outcome of RSI can't be predicted. But supporting this claim requires giving actual evidence or logical arguments, not just defining terms to make your claim true.
https://www.lesswrong.com/posts/kgb58RL88YChkkBNf/the-proble...
https://www.youtube.com/watch?v=7wy3xyoXYt8
Doomers have been working to explain things for years: https://www.lesswrong.com/w/ai-safety-public-materials-1
It becomes a lot clearer when you listen to the people resigning from AI companies and learn about incidents like the HuggingFace incident. This has generated major press coverage.
As for solutions, I think you're a little too pessimistic. See, for example, https://nothingismere.substack.com/p/a-near-term-policy-for-...