snaking0776

I think there’s really only a few solutions to it which all involve some kind of divestiture, it just depends on if you want to target corporations or individuals:

1) a new wave of anti-trust ala Lina Khan and breaking up most major tech companies.

2) passing laws limiting the behavior of, and allowing for the punishment of corporations and group agents as a whole. There’s a lot of philosophical work about how to hold a group accountable for their actions, try searching List and Pettit Group Agency. There’s been a lot of development since then but I’m less aware of it.

3. Caps on decision making power (potentially measured via some weighted stock ownership, or wealth more generally) like suggested above. I agree there would be a lot of second order effects here which are beyond the scope of this comment but I think we’ve clearly gone too far in one direction where single individuals are now essentially funding national party midterms, have increasing power over real world conflicts, and any other of the myriad of corruption issues you can find out in the wild.

1. It has people in it who are career obsessed and who are willing to ruthlessly go after any opportunity to improve their standing/stock valuation. Using a 2 week old model to snipe a millenium prize for PR is in line with that.

2. There are many researchers and even executives at the company who genuinely think we are speedrunning the end of the world. I don’t know anyone in this field who honestly argues that if we build ASI soon it doesn’t lead to extinction. This group of people can output warnings about the state of research and fears for the future while pushing for regulation out of genuine fear of what they’re building. I tend to agree with them.

You can’t view these companies as a monolith. They’re actions won’t be consistent because it’s built of many people with conflicting beliefs. Please look at arguments regarding AI risk and the current pace of progress and value them as it relates to the argument itself, not who said it. We are in a dangerous place and no one is sure how quickly we’ll get to a bad spot.

Almost nobody (I have to say almost) in the world including Xi, the ceo of deep seek, Dario, and even Sam Altman want AI to kill everyone. I really think we just have to

A. Get stakeholders to believe this is a bad idea.

B. show that we’re willing to do it first to prevent this stupid race from continuing.

1. Robotics begin rolling out more broadly across the world.

2. Labs start automating more and more of the physical process of running science as expectations of natural science advances begin to mount.

3. Economic pressure between the labs continues to ramp up and the pressure to continuously improve forces quicker and quicker model releases than a team of human scientists can effectively evaluate outside of automated means.

No one knows what pre-conditions are for us to hit the point of no return nor how quickly it will come. If all is required is a sufficiently advanced cyber model we may not be far off. If it requires incredibly complex biological knowledge and access to certain lab supplies we likely have a bit longer. Yes this is guess work and we need more evidence of the dangers but at the same time we need evidence of safety. While you may disagree with the risk level, I think it is easy to see the consequence if these labs achieve their stated goal. At this point it seems a political solution is the only way to enforce caution.

It’s not apparent to me that current LLM’s have the right biases for a trivial unit of “more LLM” = smarter. It seems apparent that LLM’s are under many metrics more “intelligent” than the average person but they still don’t rival our corporations or collectives in terms of intelligence and I don’t see them being plug in replacements for humans and maintaining the benefits of the collective structures yet. Not impossible but definitely not trivial in my mind. We and our cultures evolved to work together and the modern world is the result of the emergent structure that resulted.

As a plug my favorite field relating to this is called stigmergy and basically describes how individual units like ants build intelligent collectives that are far greater than the sum of their parts.

Not sure what you’re talking about here. We can’t replace all energy needs with solar but it’s clearly one of the cheapest energy sources and with the added benefit of low capital expense to get started so you can set it up in distributed grids without the massive expenditure to support nuclear installations.

Generally you can take a geometric view of this where certain features in a stimulus covary with neural activations in the same way they will with RNN “activations” which is at the real core of why people model things this way. The general idea being a dot product in an RNN can tell you something about what features are relevant for a task and we can look for hints of the same information being encoded in neural data. Certainly not everyone is in agreement on this though.

Anthropic has successfully made their vibe the bringers of the end of the world so please stop us. ChatGPT is in some weird middle ground where they’re no longer the evil company of a year ago and are trying to act like the mature one in the business. XAI is the vanity project of a jealous kid who wants to ignore anyone elses opinion but is falling behind. Gemini is viewed as the less cool cousin and no one pays attention to since google has generally avoided the end of the world talk (coincidentally I think gemini has the best non-coding model but no one seems to talk about that)

I think each company has been treated in the wider discourse based on their vibe and no one is operating on facts here since it’s all pretty obfuscated intentionally. I could see some things going very bad as a result of these models but I also don’t believe anyone who’s just pattern matching to their favorite scifi book here and we’re all playing into it. May as well go outside for a bit and enjoy the sun.

https://doi.org/10.1037/a0026699

https://doi.org/10.1037/bul0000131

Past research has a eugenicist bias because early statisticians were eugenicists seeking evidence for the ideas. I would argue that’s why the social determiners research is valuable to help offset that.

Where it fails me though is exactly when I’m doing something novel like developing a new model or trying to develop some new method to process data. I’ve tried many times to one shot these ideas with detailed descriptions of what I want, how I’d like to generate abstractions, etc and it almost always ends up changing what I want to what I can only describe as something which better matches its training data. It often quietly changes key details that means that I have to delete the whole thing and start over. Just today this happened. On this level of task I’ve found that my workflow and pace of iteration hasn’t really changed at all in the last year. I still have to go and explain in detail on a function by function level what I want in much the same way I did a year ago. While that’s obviously a harder task, it seems to me like the task this whole long term exponential argument hinges on. I obviously could be wrong and maybe LLM with eval loop will do all of this for us but it seems still quite bad at anything without a clear definition of “good”.

I’m personally much more concerned about autonomous weapons, surveillance, and people plugging these things into places they don’t belong to avoid responsibility than I am the general possibility of these models being smarter than me in every way but obviously I could be wrong on this and am just using it incorrectly, hence the question.

I would argue the best way to safeguard against long-term threats is to start by focusing on the issues that already exist. If you can offset the health risks of local datacenters or issues of unequal distribution of wealth by creating a more equal society right now then you’re already on the path to handling these long term issues. To me, this distant focus only distracts from the already present issues and conceals effective policy in this moment. We do need to safeguard against AI risk and it’s already here. Don’t even get me started on the havoc which recommendation systems have caused in society in the last 15 years which we still don’t call AI because it doesn’t speak.

Tl;Dr: These essays can feel disingenuous because:

1) AI risk is already present without exponential growth. The exponential growth argument often feels like a distraction from the fixes we could put in to fix the current issues that are already here.

2) The people stating this argument often have billions of dollars to gain if it comes true. While they may be altruistic, I also don’t see them doing all that much to fix the issues that people are already claiming exist and instead continue onwards on their path by justifying that they are the only ones responsible enough to handle it if the super intelligence does arrive. By continuing down that path, if that day ever does arrive they’ll have ensured the existence of a system which is unable to handle it.

I see this move as an attempt to stifle 2 by hiding 3. It seems like a bet that if we don’t focus on the global issues (like a collapsing current) then much of the political power behind climate change disappears since we are frankly quite lucky in the US to have a beautiful, and largely clean environment/air. Grassroots movements and politically powerful messages are likely an annoyance for those who want to keep doing business as usual since it benefits them to do so. Not trying to make a strong claim that our current response to climate change or what anyone in particular is doing is right or wrong but it seems clear that climate change is a powerful message that pushes for much grander scale changes than most people are comfortable with especially those with strong private interests in ignoring it.