HNHacker News
TopNewBestAskShowJobs

gck1

895 karma · joined May 9, 2018

saggy.eskimo294@passmail.net
submissionscomments
gck1··on Claude Fable 5
Get any model, any reasoning level, ask it to tackle a challenge, have it come up with a plan. Then ask it "are you sure? This feels wrong", and it will now think it's wrong. Do that again in a loop and you'll see how unnecessary human judgment actually is.

Or alternatively, have fable write some complex code. Then ask it to do an adversarial review of that code in a clean session. You'll find that it will find issues in the code that it just wrote.

Now imagine you're a layperson who doesn't know which one is true.

Human expertise is never going to become irrelevant.

gck1··on Claude Fable 5
Also, aren't all Claude users in their own "organizations" in Anthropic's own terms?
gck1··on Claude Fable 5
They would still have to procure things that would (I hope) light up many screens before they're able to. And such numpties are probably already monitored, or in prison for some other stupid life decision.

I also would like to hope that people that are likely to do such things are probably:

A) don't know how to break even the most basic guardrails of models

B) already in glasswings project

To prove point B - Theranos existed.

gck1··on Claude Fable 5
I think this really depends on what is interesting to you, personally. Something that you can have fun while doing, without breaking laws.

For example, I'm a privacy nerd, so I like reverse engineering proprietary software to figure out how it works and what data it collects.

I also like getting full access to the hardware I own - like a robot vacuum (bonus point: you'll also learn soldering, probably, which might come in handy if robots take over). Or my Mac studio that imposes some limitations on me on how many active user sessions I can have.

These kinds of things have put me on a path where I've learned how hardware or networking works on deepest levels, what goes through these pipes, how I can place myself in the middle, how I can enter places someone didn't want me to enter.

And once you know how to do these things, you know how to apply this knowledge in defence.

Essentially: always be curious and always try to say "but I want to" when something that doesn't cross the boundary of your physical property says no to you (legally).

Yes, models like Mythos may find vulnerabilities, but your knowledge will make it possible to point it in the right direction, and understand where it's mistaken, or to understand the output when it's right, and what is the right course of action.

gck1··on Claude Fable 5
Because your personal projects likely are not very complex and not high stakes. And you are not responsible to anyone but yourself.

If you're working at a place where this is true about the the organization, then sure, that job will likely be gone. But that was never a good place for your career regardless.

I have 4 concurrent personal projects that are quite complex, but low stakes. I can have SOTA models go wild on them (because low stakes), but they can't one shot anything there. And I can't really work on more than one at a time, even if AI is doing coding - it still requires supervision.

I also frequently nuke these projects and start over because they made a mess there, but I collected necessary knowledge on how to guide them better. You can't do this on a production project, not when there are deadlines and stakeholders.

But just in case some organizations decide to embrace the "trust it blindly" model anyway - cybersecurity specialization will ensure you always have a job.

gck1··on Claude Fable 5
And if you're working on a live target, just put up local proxy and point it at a localhost.
gck1··on Claude Fable 5
Opus 4.8 (or a classifier in front of it) flagged my account and refused to comply when I told it to kill the process. Reasoning summary was complete bananas.

With this in mind, I don't want model to be proactively instructed and encouraged to sabotage without telling me.

gck1··on Claude Fable 5
But Chinese models will poison your output if you ask them about Tiananmen Square! That's not good, so poisoning everyone's output without telling them is the only way to prevent that.

Come on guys, why can't everyone just be there for the good guy?

gck1··on Claude Fable 5
Your job is just going to change. You may or may not appreciate/enjoy what it becomes necessarily, but it doesn't mean that you are going to not have a job.

People underestimate how people hate looking at terminals and "weird looking combination of characters" even if they didn't have to write them. If anything, you will likely have more career opportunities in the future, than ever.

And if you get a chance to wet your fingers in cybersecurity - I would take it.

gck1··on Claude Fable 5
I love that the conditions of getting into "ultra authorized club" just means that you either have deep pockets, or you've got the size of the audience that marketing department approves.

As if being in any of these two somehow means that you won't use the models to say, steal random people's money.

Sam Bankman-Fried or Elizabeth Holmes would have been the members of Glasswings project, if not one of the initial members. Who's to say we don't have similar people with access to Mythos right now?

gck1··on Claude Fable 5
> Same thing Meta was doing before they fell behind.

Obviously unrelated to the OP, but it's crazy to me how incompetent Meta is at everything new they try to do.

They burned billions of dollars on the most ridiculous project one could ever think of - somehow thinking that VR is the future.

Then they did catch the initial wave of actual future with AI, they were at the forefront of open weight models - and failed at that too.

What is even happening there?

gck1··on Claude Fable 5
I'm not sure how the new guardrails work exactly, but I've read enough of reddit / Chinese communities focused on jailbreaking the models, to know that you either have to nerf it to the point where it fires even on "kill the task", or someone (maybe even other LLM) is going to come up with a set of tokens that is going to go around the defenses.

Nerfed models are really bad for PR, especially when you're staking your company's future on it being the smartest, most dangerous thing in the world.

So I believe they will ease up on nerfing/guardrails just enough that bad actors will find a way, while good ones will stay limited on anything dual-use. Just like such restrictions usually work in other places.

P.S. yes, "kill the task" did, in fact result in a refusal AND a warning on my claude account in Opus 4.8's early days.

gck1··on Claude Fable 5
That sounds like a harness issue to me.
gck1··on Claude Fable 5
It already is. But marketing is hell of a drug.
gck1··on Claude Fable 5
But how is this sustainable? It's not like paying $5000 per feature means you'll be refunded if prompting "make no mistakes" didn't work.

The only reason why I pay $200 is because LLM's errors costs me that much, at worst. If "make no error" starts working - sure. But surely, unless you have millions of dollars of cash to burn, a coin flip that costs $5000 is an insane idea?

gck1··on Claude Fable 5
Yeah, because it's impossible. You can't ask it anything about the thing that it's known for. It will not even answer a sky-high level question about reverse engineering, for example.

In CC, it will probably report you to authorities if you ask it to do a vulnerability scan of your codebase.

gck1··on Claude Fable 5
You should check out some Chinese forums. There are services selling gateways/proxies for all major models at fraction of the official rates. Likely reselling subscriptions, or some other form of abuse.

I've seen people posting screenshots of billions of tokens consumed where they paid next to nothing.

These same gateways are likely also reselling the data to Chinese labs, because TLS has to terminate at the gateway level.

gck1··on Claude Fable 5
There's also a reality where China does develop Mythos-level model but stops releasing the weights.

That reality is much scarier.

gck1··on Claude Fable 5
And yet, that is completely uninteresting to their user base.

If saner factory can sell you the same tool at a fraction of the cost of a gold plated factory, your choice is going to be obvious.

gck1··on Claude Fable 5
I can see that being true, and it very likely is true. But isn't infinite VC money and no incentives to optimize operations the reason behind that?

Take a look at China for example - they have no access to NVIDIA, so they're trying to build their own hardware, they have no unlimited funding, so they try to optimize things.

And Anthropic is complete opposite of that - if NVIDIA were to triple their prices tomorrow, Anthropic would still pay them.

In the end, either we all somehow go mad and start paying Anthropic tens of thousands of dollars per month so support this madness, or we will go with whoever isn't lighting cash on fire.

gck1··on Claude Fable 5
I do, and it's called DeepSeek's pricing table. At the same time, "subscriptions are subsidized" cohort have no data whatsoever, and yet they're in every thread.

Granted, it could still mean that Anthropic just chooses to lose money - but that's Anthropic's choice.

DeepSeek has proven that inference can be much, much cheaper than what Anthropic advertises on their API rates page.

gck1··on Claude Fable 5
Nothing is subsidized. Subscriptions are profitable for both Anthropic and OpenAI.

Anthropic wanting to switch billing to API rates is them just wanting to generate more profit.

gck1··on I built a vulnerable app and spent $1,500 seeing if LLMs could hack it
On refusals: I found that many models are fine with security work if they think what they're working on is local. They do get very pushy if they think it's a live target.

GPT-5.5 xhigh refused to perform RE on a live JS VM. I had it extract the VM from the target, which it was happy to do, then in a clean session, had it working on this offline artifact - which it was again, happy to work on.

Then I found even simpler trick: I proxied the target from localhost and it was happy to perform anything on the target.

Opus is a different story. Claude does so many mid-turn prompt injections and classifiers, that probably 30% of its context is consisting of "refuse to do work" lines. It refuses to even scrape a page.

gck1··on I built a vulnerable app and spent $1,500 seeing if LLMs could hack it
No. Anthropic runs prompts through a classifier that then proceeds to do prompt injection on anything dual-use, which then results in an escalating flag on your account, which increases the strictness of the classifier and volume of prompt injections progressively.
gck1··on MacBook Neo is so popular that Apple doubled production
I'm a recent-ish convert from 18 years of Linux to Mac. I paid $2k for a Dell laptop 5 years ago that has been used maybe 6 months and every single day of using it was miserable. Loud fans, stutter, horrible resolution, battery anxiety - all of these problems got resolved with a $2k Macbook.

It almost feels like Apple knows that hardware shouldn't be getting in your way and the rest of the industry thinks it's still 2002. Granted, that memo didn't get to the software team at Apple, but that's a relatively small problem that you get used to.

gck1··on 32GB of DDR5 now costs $375 – AI shortage continues to squeeze PC building
I'm curious, how is Apple seemingly unaffected by this RAM supply crunch? Why are they not increasing prices 2x/3x?
gck1··on 32GB of DDR5 now costs $375 – AI shortage continues to squeeze PC building
My unraid server alerted me today that one of the 28TB HDDs is at 100% capacity. I bought that drive for $290, a year ago. Today, it costs $590.

I'm at least happy I went for DDR4 RAM when I was building this server.

gck1··on Uber's $1,500/month AI limit is a useful signal for AI tool pricing
A lot of talk about cheaper models here. Just curios, is there any non-Anthropic model that can do UI well? GPT-5.5 is laughably bad, and I'm never restarting my Anthropic subscription after their 6-month sprint of gaslighting, even if opus was really good at UI.
gck1··on Uber's $1,500/month AI limit is a useful signal for AI tool pricing
The same thing applies to US models. Check out various system prompt leak repos on github. There are also prompt injections by various parallel "alignment" models that pre-process the prompt before it's sent to the main one with questionable guidance.

You'd be surprised how much of bias exists in easily extractable information. Now imagine how much of that happens during training, that you can't easily extract.

So this is largely a moot point. Yes, Chinese models will likely have some weird things injected into them. But so do the US models. Do I care? Not in the slightest. Models are my code monkeys, and if the code leaves my machine, I assume IP is leaked be it a Chinese model that clearly tells me they do use the data, or US models that pinky promise they don't.

gck1··on Uber's $1,500/month AI limit is a useful signal for AI tool pricing
You don't have to remove the safeguards if you can prompt your way around them.

There's a subreddit for people wanting to sex-talk to various models. It just so happens that the same prompt they use to 'jailbreak' SOTA models for sex talks also works if you want to have model write malware, or tell you how to design a highly illegal device.

← PreviousPage 4 of 12Next →