metalcrow

How is this specific to open source models?

> a weight file you can’t trust is still a black box

Important to note, LLMs are still black boxes even if you use a more open source model, I.e open training data or even if you train it yourself!

Making a language that an LLM can't use is one challenge, but making one that at the same time humans CAN use is a whole other issue. This language seems quite annoying to use as a human!

What, exactly, is the point of this post, and especially the point of it on HN? It appears to be a rant directed at 2 people on the internet that has nothing to do with tech.

I've been going to anthrocon for 10 years and a furry for longer. Ever year there are multiple posts about it, both on Twitter and in telegram. In fact once a friend of mine was drugged there.

Also as a furry I dunno if I would say the community has a culture for looking out for each other.

i honestly don't think that's true actually. especially since a lot more people go to anthrocon, but even per-capita i bet AC is higher

I gotta say I'm confused about the entire reason this is important. Any furry convention has just as many IT professionals (and probably a few AI devs), but people aren't posting about Anthrocon and talking about how many people get drugged and assaulted there in the context of IT. Why is this different?

> If you use RSS for feeds outside of a community you moderate, there is no replacement. RSS has been a beloved part of the open web

wow. Nothing else to say about this. This is pretty evil.

I don't really get the metaphor here. What is the reasoning for the name Universal Bone Dissolver? Is it trying to imply the only purpose is harmful and negative?

Screaming fire in a movie theater is legal. https://en.wikipedia.org/wiki/Shouting_fire_in_a_crowded_the...

This is a reason AI regulations definitely need to be done at the government level, with teeth. Regulations need forced transparency, independent audits, and teeth to ensure that, to continue the metaphor, Germany actually moves it's troops off the border.

This piece kinda just reads like a smear campaign that says "these people are associated with some other people who i think are weird and icky, so it's better to let the government (i.e Trump) control AI". I think that's a pretty bad take.

> Why are you treating those as if they're separate things?

Because they are? You haven't explained at all your belief that they are the same.

> This is just completely wrong. Don't mistake the path by which something happened (or appeared to an outsider to have happened) for a fundamental truth.

So it's completely wrong but it's only what has happened so far? You're repeating yourself but you're not listening to my responding to your exact points. I literally responded to your claim that "would you have us destroy all chip fabs and ban all biomedical research?" In the last post.

> Nukes don’t get “worse” because of physics, not because of laws.

Yes, that's my point, that's why the metaphor is wrong

> Criminals using LLMs does not make the LLM situation bad

If you believe that bad things are not bad then there's nothing left to say. A billion people are going to die because you don't understand what is bad.

The point is the risk is in counties being gifted or assisted in making it, not them building it on their own.

Given what's happened since then with propaganda and wide spread scams, I think the original concerns about that have proven quite accurate

> Presumably performance on the level of the current frontier of AI would be achieved in short order and the ceiling would only continue to advance from there

That's an enormous presumption! You're saying that even in the theoretical case that frontier research is halted but efficiency isn't, we could do better then the frontier and reach world-changing AI in 30b parameters at home-scale labs?! If that's true then we can just give up now: the world as you know it is going to end in around a decade and billions are going to die, there's nothing we can do. But I don't think that's true. Advancing the frontier seems to take a massive amount of compute, data, and parameters: miniaturization only happens afterwards.

If you're right then I concede. It doesn't matter what we do, regulations or not. But if I'm right then regulation can do something and in theory help bring a better future.

We only know about the stuff in https://www.anthropic.com/threat-intelligence-report-septemb... because it's not open source. I assure you the open source and Chinese models are being used in this same way.

North Korea got it with help from China, Israel got it with help from France. They didn't do it on their own.

> Not in the way you seem to be thinking, no. Nuclear weapons can be banned because they are prohibitively expensive to research and build. You can't reliably hide a nuclear program.

Isn't that true of AI as well? Data centers are very big and very expensive.

> Bit of a tangent but we do, actually

Sure but not like the movies, these don't destroy cities. But the metaphor isn't accurate anyway: nukes don't get worse. The actual idea here is preventing the frontier of AI from advancing.

Sure, it's possible for regulations to make things worse, i will agree. But without regulations it's pretty clear things are going to end up VERY bad, and the only knob we have to make it not bad is regulations. So it's important we try something, and work towards doing a good kind of regulation, or any one of the bad futures you imagine is pretty likely to come to pass.

I don't think giving them free reign is going to do anything positive for it either.

To further the metaphor, didn't the ban on nuclear weapons research work? We don't have pocket nukes, and there's very little risk of random countries acquiring their own due to the amount of work involved. To be concrete to AI: if we stop work on the frontier, the best we can do is make the current frontier easier to get to and more accessible. And while current frontier is strong, it's not world-changing so. The only way for the frontier to get to that level is to do research on it, and stopping that research gives time to help develop plans to not make it world-changing when we get to it.

That's a valid concern, but some of that is outright impossible. Banning chinese models and killing open source training is not happening without massive unified international cooperation, and that sort of level of action would require the international counties decide to allow the US aligned companies to just, win. Which would be pretty against their own interests.

Honestly, fair point. There's not much truth to go around these days. But still, doing nothing is also a massive power grab. We're facing a technology with no comparison in the history of humanity: the creation of a new mind. Doing nothing is like doing nothing about nukes. There are a few smart people who have come up with ideas that will not require that much trust.

The regulations don't have to address the end goal, but the process of development itself. Even just a pause on frontier research is a good start

> the positives are subjective, potential, and still not convincingly verifiable.

Saying this post- millennium prize problem solving is laughable.

A lot of people in this comments section seem to be against this, i genuinely don't get it? Why? Do you think the current state of affairs is GOOD? That if we let companies create a mind that is, AS OF TODAY, able to solve problems no human in history has solved, with no regulation, things will end up good for us? We need some sort of regulations, some sort of method to help ensure the thing we are creating ends up good, instead of just running headfirst into it blindly. I assure you, any sort of regulation at all, including ones that actually hurt all leading companies, would be met with celebrations from these voices. https://www.seangoedecke.com/they-really-do-think-ai-might-k...

Then what's the reason behind this decision?