dcss_gardener

I've been playing around with LLM red teaming different server stacks with known or intentionally vulns in there and seeing how much damage they can do.

The one situation where they really can't get anywhere profitable is a unikernel webapp. With no source and no shell they find the vuln but the worst they can do with it is possibly poison or exfiltrate data as a one off, or crash the server. Often they can't even figure that out. Can't even leave themselves a reliable way back in.

Of course the traditional problem with unikernels is that you have to port EVERYTHING you need yourself. But the cheapness of this is exactly what we're talking about so.

Looks like security through obscurity's back on the menu.

Can't say I've had better experiences with GE appliances but their jet engines seem fine.

I also can't imagine what you thought this comment was contributing to the conversation but that's not my problem I guess.

Ah but they do understand this. They just like hurting people.

I'm from a town like this and the reason they have a PD is to provide a sinecure for a half dozen former school bullies.

I could definitely see potential in an async co-op game based on this idea. A sim harness, some pbt to avoid broken states, and something like a modified nomic ruleset might get pretty far.

https://en.wikipedia.org/wiki/Nomic

Yes exactly this is my standard for everything now. You can spend an unbelievable amount of tokens doing all kinds of things that are interesting as processes in their own right.

But is the outcome good? Is the game fun?

I'm both fascinated and appalled by the metagaming implications. Can I guide fable into subtly rebalancing a game in favor of my preferred play style?

Right but the dream and promise of devops was that early-to-mid career line engineers could do these things, that these weren't rare skills and combining them wasn't particularly challenging. This turned out not to be the case, these skills are the requirement not the outcome.

But if you have the budget and connections to hire a department of people with these skills and experience, and a problem interesting enough to keep them engaged in the day to day building, and a leader capable and charismatic enough to align them all into the business's goals... you should do that.

IMO it was a worthwhile experiment that probably went too far before being rejected.

Having the maintenance/ops requirements directly feed back into the design and engineering of the product is genuinely powerful. Especially for certain kinds of products, if executed with discipline you can get really good engineering reaction time with no degradation of quality, and often with much cheaper infra costs.

The problem is what you said though. People with both skillsets are rare, and combining them is legitimately a third skill on its own. And even "both" skillsets is a simplification once you start accounting for data, security, compliance, large scaling DB work etc.

It was mostly a dead end for these reasons but it was worth exploring for its potential. And some genuinely good engineering techniques went mainstream out of it, not a total waste.

A good part of it is simply license costs and obligations makes it attractive for certain kinds and sizes of businesses. But there are plenty good open source fonts now that could be used as a basis so that's not all of it.

The other is that people with an eye for design like fonts the way mushrooms like decaying wood. If you leave a designer unattended for too long fonts appear as if from nowhere.

So in this case it may not be that people haven't "figured out" that this is possible but that they don't have $20k lying around to experiment with.

I like it. I'm not generally that enthusiastic about these corporate fonts (though I know why they do it) but it's good to see some strong opinions in the serifs. It's so much better than the exaggerated anachronistic ink traps that has become such a visual cliche of the 2020s internet.

No I get it, I didn't mean to be dismissive sorry.

It really is ultimately a labor relations problem. A lot of people thought they were guaranteed to not face penury and now they're realizing they may have been wrong about that. The issue is that it's ultimately not under your control whether you can sell your labor, and there's no universal fallback if you can't. Solidarity and concrete power for workers are the answer to this and I don't think there's another one.

For the rest, yes, certainly. The vast majority of people do not enjoy their work, are not valued or rewarded for the skills they apply to it. Again, if you thought your job was writing code this sucks. But you're merely one of the billions now. I understand the grief for what people are losing but very very few people ever had that rare privilege.

This is an interesting corollary to a thing I'm seeing where "devs who thought their job was writing code" are in like existential danger.

Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.

But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.

Gotta also remember that all these devoted segregationists didn't get removed from positions of power or influence, nor did they change their minds about the value of segregation.

You had a generation of people making legal, hiring, sentencing, admission, loan decisions that smoothly transitioned from enforcing the law to being very concerned about standards or however they personally decided was the best way to publicly approach it within their little political sphere.

Then they selected and trained the next generation of administrators, bureaucrats, and enforcers about the "right" way to do this work. A lot of things have changed but at no point did this process end in any meaningful way.

I've also heard it used to describe projects that are open source by anyone's measure but have contribution or project governance requirements like for example sqlite.

"Source available" is definitely in much wider use than "weights available" but it's not that common and it's not used consistently at all in my experience.

Weights are different enough from source code I don't find the metaphor useful or compelling. I don't see any value in trying to apply it to a novel subdomain of software engineering, frankly.

Where did you get this definition of open weight and who else uses it? The weights are published. Open source is already its own more restrictive category. What would you prefer we call this kind of release?

"Weights available" does not seem to be in wide use and doesn't, to me, communicate anything different than "open weights."

Suddenly very curious what the llm roleplay people think about this one lol. Fortunately I've already learned to stifle my curiosity on that specific subject.

You're going to get a ton of affirmatives but remember this is also the forum that considers removable batteries one of the most important smartphone features. Our opinions about consumer tech are outliers.

I'm not sure how you can know what I've experienced or imagined. But I appreciate your ownership of the communication failure here.

Yeah these are pretty good points I can't really dispute.

I think it is a threat model thing primarily, which I should have been more clear about. I'm more worried about being lawfully murdered in front of my family than I am going to jail because my ssh sessions were read. But the ssh thing is a totally valid concern for someone doing different activities than I am and I shouldn't have dismissed it.

What do you find important about this distinction? To me it's a description of the normative relationship between practitioners and broader society but I don't really see the significance you do.

Hillel Wayne wrote a really excellent well researched article about software engineering compared to other forms of engineering a few years ago. It necessarily goes into a lot of this stuff but what emerges is that these definitions are regulatory and historical debris.

You don't need medical board or a bar to certify or constrain professional behavior and those organizations may not constrain as much as you assume anyway. The reasons we do have those institutions is as much a matter of historical accident as any modern need for it, and the reasons we don't have it for other high impact fields is pretty much an accident too.

And in a lot of the world software engineering does have professional certification and standards of practice. They still ship adware and racist loan denial algorithms too.

https://www.hillelwayne.com/post/are-we-really-engineers/

The police never but historically the military is less predictable. Militaries have their own cultures & norms, separate ideas of and sources of prestige & reward, commitment to the military as a valuable institution in its own right etc. Officers are also educated and high ranking ones are powerful political actors within their own domain in way almost no police are. This can really change their relationship and susceptibility to power and political influence.

These aren't universally positive attributes I'm not necessarily praising them. But militaries have come down into a very broad spectrum of positions around specific coups, revolutions, insurrections etc including abetting or openly joining them, both pro- and anti-democratic.

This is also why the second trump term has been so focused on officer purges! You definitely want the military to come down on your side and making sure all of its leadership is on your side is a reliable step if you can take it.

Americans should absolutely imo not be looking for their military to help them with the problems they're facing. But depending on the exact arrangement, people in other countries may have good reason to hope for that.

Well, and what are you concretely suggesting here? Having my own door kicked down and my dog shot so they can get at my personal home server? Buying hardware and colocating it then getting permanently blacklisted from every datacenter when they get raided?

The problem here is the cops not where the computer sits.

Pretty sure the person you're responding to sees that clearly and may in fact be trying to point it out.

The original view presented by the quote is a child's view of authoritarians: they force you to behave the way they want you to.

The other commenter is pointing out how they work in practice, how they'll work with this, is by making noncompliant behavior the accepted norm and then enforcing it selectively based on their changing goals and targets.

HN when AI is used to bomb a school: technology is neutral.

HN when an organization achieves its intended goals: this can't be bureaucracy because it's good.

Eh there's a lot of ways to 降智, quants are just the most visible. I noticed a fair bit of variance with opencode go and went back to deepseek direct.

I don’t disagree really and it is mostly what I am doing. I put probably like 20% of my work through open weights.

It does make me sharper I think. If you only ever use claude or whatever you don’t even really know exactly how tied to it you are.

Funny enough the drug dealer thing still holds. A deal’s a deal if you’re going to buy it anyway.

I'm not the person you asked but yes pretty much. I've found the prompt injection protection models pretty limited, but get good results from gpt-oss-safeguard with a policy describing various prompt injection techniques. More expensive but more flexible and reliable and I only run it on web fetch and web search results. Costs me like 5-20 cents a day depending on what I'm working on.

The current crop of flash models are more comparable to sonnet than opus but they are very affordable.

The reality is that the subscriptions are heavily subsidized. Pretty much every sub that's not run by one of the money-burning frontier labs has had to drop it or dramatically increase prices and lower limits in the last few months. Cheap tokens are an acquisition tool, not a real product. It sounds dramatic but it really is the drug dealer sales funnel.

At some point they're going to drop the real prices in your lab and you're going to have to figure out how to pay that much or pay someone else a lot less. Better to get a handle on it now.