grayhatter

> This is itself alarming to me, but no one else seems to find this to be quite damning for the AI services being offered, preferring instanced to be wowed by the convenience and speed at which they can be delivered unreviewed and unproven information.

Welcome to the club? This new awareness you've found over the true quality of LLM based GenAI output has been what "all the haters" have been mad about for-ever. That the output of LLMs are clearly defective, and merely have found a cute trick towards making humans think they're less defective than they are actually measured to be.

And the corresponding anger and frustration to push the risks of genai output out onto others, while also aggressively pushing it as a feature you should be using already. You're behind don't you know, and whatever other lie I have to tell to trick you into enough FOMO to pay me 200USD/mo so I can sell FOSS back to you.

An LLM can only output the mean next likely token, and then add a bunch of extra noise on top of that so it feels interesting and not repetitive. None of this is new, the problem is, 50% of humans are below the mean, but have no idea. So when an LLM tells them some lie: well, it sounds so helpful! It's impossible for someone who sounds this helpful to lie to me, liars never sound confident! It must be PERFECT! I'm gonna tell everyone how perfect it is. so the bottom 0-33% think LLMs are fantastic tools that make nearly 0 mistakes in comparison to the bottom 33%. 33-66%-ish aren't sure, some times it's great, but it will make that random mistake sometimes, but I can catch most (or all of them depending on ego). and the 66%+ are angry about how many people are getting tricked by something so obviously low quality, or are lucky enough to not have to care.

No, it doesn't, or at the very least, that wasn't my read and I don't think that's a reasonable interpretation. The comment starts with the topic, shock at the lack of curiosity from a group happy to comment in the theme of news interesting to hackers, and concludes with an argument that someone sharing their life/experience is something valuable.

IMO, the only reasonable argument one could take, is that roughly believes hackers should be curious, and that if you want to treat humans with the respect they deserve as individuals, you should default to trying to believe what they say, listening when they try to communicate, and avoid ignoring what they're trying to communicate just so you can interject something unrelated about yourself.

I not only agree, but I'm glad someone took a moment to encourage treating others with respect.

> someone telling you about their lived experience of the world is a gift.

I'm not sure about that, but to your higher point, HN hasn't taken pride in it's nominative determinism nature, nor does it appear to be a desired trait from the majority. But the continued enshittifcation aside, "it doesn't happen to me" is still a useful observation. That shouldn't be read as a refutation, because I already agree with your point. The intent of most of the comments your objecting to, likely does come from a narcissistic compulsion, to turn the topic to something about them.

But I could easily say "it doesn't happen to me" while (poorly) trying to convey a message of encouragement towards self-confidence, and self-worth. Rather, it doesn't happen to me, because I haven't been gaslit into the shared and common delusion that: if you get fired, it'll because of something you did, and not because your manager felt like it. "Employee didn't answer the call/message at 10pm" is the reason they'll invent to fire you after they've decided to fire you. It won't be the root cause. You can just turn your messages off, and nothing bad will happen because you didn't respond.

Are there dysfunctional companies where something like that will get you fired? Absolutely! I would have hoped that existing wrongful termination laws would have already prevented this kinda thing, (if they don't that's a much larger problem) but I have no objections to making this an explicit law to compel the behavior of the sub-human group that would rather mistreat their coworkers. But given that within places that behave like this, this law would only fix a small subset of the pervasive human rights abuses inflicted during non-working hours. I feel like something more expansive should be done to protect those people from clearly abusive behavior.

I still expect that the vast majority of the people that would benefit from this, could simply just turn their phone off, and no one would notice... because while it's a problem, that does happen to some people, one that needs to be fixed! It doesn't happen to me, and probably doesn't really happen to you (most people) either.

So you like the law, but don't like how it didn't actually solve the problem it was trying to solve?

I assume you're pretty well read up on matters of privacy, right? So you have a better awareness and understanding. But do you believe the average person does? Or would you assume that the average person has either been trained to ignore the banner, automatically consent to more invasive tracking, or is generally more confused about why the banner exists, or what it does?

The cookie consent law is the dumbest application of an attempt to improve privacy. It's made the internet worse, and is being used to train people into consenting to giving away their privacy without thinking... because: "clicking accept is what you have to do to use the page" -- every normal person casually browsing any site.

No implementation for cookie based consent can be done correctly.

Personally, I'd love to see a law that makes any/all dark patterns a crime, and empowers state prosecutors via grand jury to bring charges for them against both the company, and individual authors of the specific commits as jointly responsible. I don't want statutory laws, I want a trial jury to look at it, and decide if any technological measure, pattern, tactic, procedure, design, or measurement was used to encourage one decision over the other instead of a fair choice.

I don't want a set of rules that given enough funding any company is able to win as a negative sum game. I want a jury, not a trailing clause, to decide if the company is clearly acting in good faith or worthy of apocalyptic fines.

No thank you.

I meant it when I said it intentionally. I still run BB programs the same way, and expect others to behave similarly. Funny enough I was just talking to that friend this week, about the BB program. Nothing has changed so given my friends still follow the same pattern at that company... We is more accurate. Sorry it bothers you, but not everyone is you.

Your willingness to try to play fair to advertisers (a group with an earned reputation of going out of their way to avoid anything resembling good faith) is admirable, but won't show up in metrics.

Protect your attention (and frustration levels) by running adblock full time. If you want to influence their behavior so everyone stops running annoying ads. You have to give the ad sellers a reason to re-evaluate what ads they're willing to show. The only way you can do that is increasing the number of users who don't view ads. Starting with in off, makes you a user who views ads, not the kind making a point.

It's unlikely that you're meaningfully able to influence the system. You're much more likely to respond and adapt to the system, so it's more likely that youtube has trained you to watch ads.

I say we, intentionally not naming the company, because 1) doing so tends to turn off people's brains and they default assume everything $company does is the correct way, but if I say something stupid I'd rather someone tell me, instead of assume someone at $company must know or couldn't possibly know. 2) I say we, because I'm speaking for myself, (and maybe a tiny bit) for my 2 friends still running the BB program at what possibly should describe as my former company, but then I've always exclusively been speaking for me, not about them...

So I'm still not gonna name them, it wouldn't be hard to figure out who they were, with a likely-trivial amount of effort if feel the need to know... but if you'd rather, I'd encourage you to imagine I work at the worst company you can name or imagine, so you can use that to discard anything I've said. Because I'd rather be judged on my argument, not who hired me that one time.

> No one is doing 16 step exploits unless you're a huge target in some way. 0.0000001% of companies fit that bill. And even then, ok, what did they get? An account login? What are they doing to do? Read email?

Account take over of a user account. I'm pretty sure I could sell access to the DMs of a few popular people for 100x what we paid out for that report.

But also, I'm pretty confident that this researcher delivered this exploit because I'd said that there was no way he could use it maliciously, not because he wanted to be paid. Then, once I made that critical error in judgement by questioning (rejecting) his assertion in his report. He, like most hackers, being insulted by the idea, was then required to restore his name and reputation. There are the people who only go after targets that they can confidently make money off targeting... some of us care more about reputation than money, and will die on any hill when our reputation/work is questioned/doubted.

> Security "teams" are a bunch of fucking busybodies with nothing to do. Pay for a competent admin team and the security dept is completely redundant and useless.

Lmao, tell me you don't really understand what goes into getting functional systems/corp security without telling me. I don't even disagree with the point you were trying to make. You're absolutely correct! If you have a competent admin team, you don't need a dedicated security team. Unfortunately, as I live in the real world, where most people are incompetent, it does help to have a dedicated security team. Especially considering if you were an admin who is competent, you could make 2x as a security engineer, which normally keeps all the competent people out of admin, and thus requiring a dedicated security team.

I don't know why you're mad, or why you're arguing it at me. I'm pretty sure I already agree with most of your points... the only one I might disagree with, and only then because you're arguing at me for some reason, and that makes me think you probably disagree, with the important point which is, we're all on the planet together, you're not required to help me do my day job, but as an industry, both security engineers and security researchers, we need to remember that we're actually on the same side, and we need to aggressively resist returning to the us vs them mentality that we're just barely starting to escape from. Case in point, it appears to me that you think complaining about how security people are useless and CVEs don't matter, as a much more important point, than complaining about obviously irresponsible disclosure.

I said "doesn't matter" to someone once... the resulting lesson came in the form of a reply from the whitehat researcher (waves, hi brian!) a 16step exploit chain resulting in a one click full account takeover.

I'm equally annoyed and over the alarmist takes. But I don't think it's fair to group mine into it. I'm annoyed at seeing discard respect for others into the same void everyone is happy to toss quality.

Do these tiny things matter? No, not to the default-panic-level everyone adopts when they see 0day, or CVE... but duh, I'm now just repeating exactly what you already said. That no, for the record is mostly because I don't use any of these, not just because they're boring exploits. While I always look, I default assume anything CVE is boring/pointless. But I still read them.

But then, I'm not trying to convince the owner of the repo. I'm trying to discourage the theme among researchers that "no one cares", because I have seen researchers disclose bugs publicly, that we'd be eager to pay out on, because they disagreed with the decision on their last report.

I've fixed bugs being actively exploited against our users, that was found/fixed only after a whitehat report for something adjacent (we pay on those btw, and you should too). I don't wanna live in the world where it's easier for the bad guys, the only way we get there is once "everyone knows", you gotta report the all bugs that you can turn into an exploit. I don't want "the whitehat researcher culture" to move towards, who cares' dump the PoC on github, screw anyone that could be hurt by the bad guys, they deserve to be punished for the incompetence of others. SWE's are shit at security, security researchers are shit at SWE, the only way we get the good outcome, is if they're willing (and encouraged) to work together.

If you believe this, then why did you say?

> starting to think security through obscurity might not be a bad thing

> I don't know what methods where used to find these exploits but I am starting to think security through obscurity might not be a bad thing in this day and age, where someone can just let bots loose on your codebase.

I'd love to hear why you think obscurity is bad, if you now think maybe it's good in the LLM age?

I'd also be interested if you could describe exactly what or how you think security through obscurity works, or doesn't?

I've been thinking a lot about how to better teach this concept, so I'm looking to understand exactly how everyone thinks/understands how it currently works, or should work, or what it should do. I don't care about the "correct" answer, (I have ddg too :P) I'm interested in general expectations from SWE's that I might teach at work, instead of opinions of security eng speaking about theory.

> At the time I post these, none have been reported. Feel free to report them yourself and take credit for the CVE if handed out lulz. I do this so to allure people into the field, and I've always found this is the most efficient way.

I've been a skiddy, he would have believed this. Thankfully, I've grown a bit, and can see this for the transparent, "I'm angry and want to hurt others so I will feel a little less alone", it actually is.

I'm sorry you're so angry dude (me too), but as someone who's joined the blue side, we'd appreciate it if you gave us some kind of heads up, the bad guys generally have a lot more time to scroll for new payloads than I do. Not all of us deserve the kindness of a heads up, but every single one of our users deserve it. Don't punish them because you're mad at someone else.

You can flex on the idiots you're trying to flex on, without hurting people. Even an email to security@[that_project_domain] saying "hey, I've published these" would move you from the group of people I see making the world worse, into the group making it better. (You don't have to, obviously, but making the whole world worse wont make you less angry.)

> I am thoroughly seduced by a power I've never felt, even if I see it as poison.

Meditate on the idea of the negative sum game the people who seek power prefer, and then about what you'd rather see them, or yourself do with that power. Because of the things I actually care about, I find that random fantastical/idealistic desire for power to be hollow, something much easier to see in comparison. I don't care about power, for powers sake (the best way, perhaps only way, to obtain power itself). All my power fantasies involve some sort of stopping people from using their power to abuse and take from others.

There's nothing wrong being seduced by power, if you're worried about how it might corrupt your ethical principals, just don't be foolish enough to copy the small minded power seekers (humans do love to emulate the people the see around them). You can seek and hold power, and then use it to do good things. Is that harder? Probably, but I can't articulate a single reason it would be harder than doing good things without power, which most people already don't do. So don't be tricked into power being the thing that corrupts. Most people are just shitty, and very few have meaningful power; sample bias can be a bitch.

> It seems to incentivize the contributor to try [to do something unethical]

I find the argument that the act of enforcing rules to protect the things that you want to protect, as the thing that incentivizes someone else to lie, stupid.

If I tell you no, I'm not encouraging you to try to figure out how to get around what I want. Why do you feel saying no counts as encouragement to ignore or evade it?

> Incentives drives the outcome. What incentives does this give people?

I understand this argument to be, if you stop people from doing something you don't want them to do to you, you're only incentivizing them to lie to you, before the do that exact same thing to you.

Is that the argument you're trying to make? Because I don't think the solution to wanting to exclude LLM codegen, is to ... not reject LLM generated patches because it might force other people to lie to get around the exclusion.

But, just to be clear, I think the argument that enforcing rules would induce someone to lie, is an insane argument to try to make.

> but he never said he was willing to lie, he was offering a hypothetical.

I disagree with your assessment.

> First of all, I could’ve hidden the fact of LLM usage, and yet decided to declare it explicitly. By being truthful I already lost my footing. This alone makes the policy stupid. If admittance is punished it’s better to push submissions without admitting. It punishes integrity, not usage per se.

The author values getting what he wants over interacting fairly and honestly with others. Literally saying "it’s better to push submissions without admitting [the truth]".

I think your predictions are inaccurate, but will gladly acknowledge the facts do show the author decided not lie this time (perhaps because it was too late he already admitted and would have lied if he knew about the policy. Which does seem more inline with the recommendations in post). Unfortunately he was still willing to advocate and argue that honesty was the root cause reason his patch was rejected. I think generally speaking, you wouldn't willingly encourage others to lie if you weren't willing to lie yourself, would you?

Neither matter. The maintainer isn't required to adopt how they want to protect their repo, and lower the amount of value they perceive it to have. They can reject any patch for any reason, or no reason at all. No one is owed any control over their repo. If you want your code in the repo, you fork the repo.

The hurdle you have to clear as the devils advocate, is what rules is the maintainer allowed to enforce about their repo, and what are those limits. The maintainer doesn't want to introduce LLM generated code. Until you solve for that, nothing else matters.

I have an exceptionally strong, visceral, negative reaction to people who aren't offended by the arguments the author makes in this post.

Your patch was rejected because the maintainer objects to the source and tooling used to generate the patch. If you agree with the maintainers opinions or object because you wanna do it your way, does not matter.

Honesty didn't get your patch rejected, root cause analysis shows the origin of the rejection was the patch was LLM generated. If the author had decided to lie, but the maintainer still knew it was LLM generated, it would still have been rejected. Honesty isn't implicated at all, and framing it as such is also dishonest.

The title of the post can only exist if the author would gladly lie to get what he wanted ignoring the others involved in the process. That behavior is extremely disgusting.

> I don't care about what you want, so I'll gladly lie to you about my submission so that I get what I want... what you care about, and what you want don't matter!

-- Przemysław Alexander Kamiński, presumably?

I'm embarrassed by proxy that the author^ was willing to write this, and then publish it on the internet. Because this kinda behavior makes all of us working in and around software look bad. Please, adopt some personal ethics that include consideration and respect for others, and expend even a basic about of thought into if you're treating other humans with said respect. Because reading this, you're obviously not.

Your example is incorrect. @ptrCast has the same (similar, if you want to be pedantic to the exclusion of good faith) result rules. If you need @as to @ptrcast, you'd need it to @bitCast as well.

> It is significantly worse to take address and deref afterwards.

How are you measuring worse? Because my understanding from the article is that's exactly the behavior @bitCast used to have. So, instead of worse, it'd be exactly the same?

If you mean it's simply more things that you have to type... You're describing a core language feature as "worse". For all the builtins, some of them can help the compiler emit better code, but can for some doesn't mean will for all. As an example

    const thing: f64 = @floatFromInt(int_ish);
    const result = thing + other_float;
    return @intFromFloat(result);

Could zig auto convert between these types? Yes, absolutely. But it doesn't as a design decision. On some arch, converting between float and int can be very expensive. A competent engineer will ensure they're type converting in a reasonable order. Zig requires this painfully verbose syntax it order to make it painful. Are there times where it's is actually the only reasonable option? yes, but even if there wasn't it'd still need to exist because I'm not rewriting my whole program to avoid a single float conversion. But because it's a bit painful, I will rewrite this one function to make it less painful.

And, yes having already made that exact mistake... I now write better code from the start because there's no way I'm gonna ruin all my beautiful code with a bunch of ugly, annoying, hard to read, casts.

I used to complain about unused variable errors, unhandled enum branch, var unmodified (hint: use const) errors, hell even result ignored or error ignored when I'm trying to test some unrelated single line of code. But now that I'm used to them, I emit better code without thinking. It's made me a better programmer. Is it annoying? abso-fucking-lutely but I'm better now than I used to be, so: worth it; and: thankyou sir can I have another. :D

BAH! and I forgot to say the most important part.

Much more important, thanks for not just the devlog, and explaining the changes. But also; thanks for fixing/improving this!

I appreciate all the work you've put in, I really enjoy watching the the language I like constantly improve.

How confident are you? I ask because I'm a zig zealot, and am constantly shilling for it. But I disagree with a number of ark's positions, and think of him as a bit of a shitter... So I don't think "cult of personality" accounts for it, despite how easy it would be for someone to be able disregard zig if was just a personality cult.

I can only answer for me, and while I do think it's more significant a metric for me, I equally assume it probably has some influence on others as well.

C3 uses :: for namespaces, that makes it a competitor with C++ more than C. Equally Odin's syntax is more at home among python, not systems programming.

The appeal of Zig is it feels like C. To many people, this is a downside. C is very very scary to them. But for people who feel at home in C, it's not a downside.

Additionally, the selling point for both are "c replacement" where the selling point of Zig is "good systems programming language" C is only mentioned by it's users as a heuristic.

If 2 groups are trying to replace a language that people are running away from, and that's their best selling point... I'd assume they're less likely to be as successful as a different language just trying to be as good as it can be.

I've even stopped comparing Zig to C, IMO, it does a disservice to both. And I say that as someone who likes C.

Full disclosure, I need to spend a bit more time with both odin and c3 to know exactly how this compares. But the reason I keep writing Zig, and still love it, is how simple it is. Zig is aggressively insistant on simplicity at the expense of functionally or comfort. The only other high level language I know of that is as aggressive about it's design simplicity is infact C. While I assume it's an accident when C does it, it's definitely not an accident in Zig.

oh, I think it's mostly frustration over how eager everyone is to delegate their thinking to literally anything else, accelerated by [gestures at reality]. Is frustration with apathy really pretentious?

??

> Quite long devlog coming up, apologies—I got a little carried away with this one!

mlugg, please don't apologize for creating something I actually want to read. I'm drowning in low effort garbage, the in depth technical explanation is a refreshing breath of fresh air.

Might as well apologize for creating a language without a garbage collector, sure most people are unwilling to think, but some of us like nice things and are actually willing to apply effort.

Oh no, someone is profiting of the work of others?!

anyways...

It's really impressive how you could apparently argue so strongly for Fender's defense: but the message that I took away was Fender is obviously the bad guy in this story, and I want nothing to do with them... and I haven't even clicked on the new tab for the story yet.

> Eh, it depends on the workflow. Especially if you have certain stack based workflows.

I would normally assume there's 0 percent chance that `git` (the binary) is a significant impact on LLM based devel. The same applies to git, the protocol/format/tree.

I'd love to hear about what makes the workflow you have, where any part of git becomes a noticable proportion of the process? Unless you mean your LLM just can't figure out how to use git?

The way you define the evaluation criteria seems very problematic[^1].

I don't understand the point of describing it as 3 possible outcomes. I objected to it because the only reason I would do something like that would be to obscure the severity of the model defects. I'm sure I'm missing something, but the reason I suspect that's how it's done, is to [intentionally] obscure the actual meaningful metrics.

I would expect any engineer to evaluate any model using accuracy, (error rate), and usefulness (definitive answer rate), as strictly independent metrics. Did it answer, and if it answered, did emit incorrect or misleading information and how many quantifiable bits of each.

The false negative rate (model confirmed to contain the requested output/information via other method but was unable to for the given test) is significant, but given a non-definitive answer is significantly different from a definitive and incorrect answer. Why would you want to group hallucinations?

Number/rate of useful answers (correct and incorrect) and error rate (given any answer how often will that answer be defective in some way).

To be clear, I'm differentiating hallucination rate from eagerness to answer, even though they're obviously linked because I believe presenting 20 correct answers, 20 incorrect answers, and 60 abstentions as a hallucination rate of 25% as obviously malicious. If I give you 40 answers, 20 correct and 20 incorrect. the error rate is 50% and if it refused an additional 60 times, it's usefulness rate would be 40%... arguably 20% depending on how strict you choose to be about the definition of useful. The matrix we should be using is a 2x2 true positive, false positive, true negative, false negative. But being that honest that might make the model look bad!

[^1]: just in case it's unclear, I'm using you exclusively rhetorically. I don't think you personally are being misleading, only that you're explaining how it's done... but that's the problem isn't it.

That's weird, why do you do that?

When someone asks a question, if I don't know the answer; I say I don't know.

System 1 vs 2 doesn't really matter... I won't use an LLM that's willing to make up random shit. Equally I also won't work with a human who does that. Trust and confidence a system will function correctly is an important quality, in both humans and genai