HNHacker News
TopNewBestAskShowJobs

grayhatter

2,386 karma · joined January 13, 2017

https://gr.ht

[ my public key: https://keybase.io/grayhatter; my proof: https://keybase.io/grayhatter/sigs/j8azbrA5Q7W1ecOwaYaZfMwmWh95MA06i5YpVRIhmno ]

submissionscomments
grayhatter··on Zig quits GitHub, says Microsoft's AI obsession has ruined the service
> I was saddened to see how they ganged up to bully the author of the Zig book. The book author, as far as I could tell, seems like a possibly immature teenager. But to have a whole community gang up on you with pitch forks because they have a suspicion you might use AI... that was gross to watch.

Your assumption is woefully incorrect. People were annoyed, when the explicit and repeated lie that the AI generated site he released which was mostly written by AI, was claimed to be AI free. But annoyed isn't why he was met with the condemnation he received.

In addition to the repeated lies, there's the long history of this account of typosquatting various groups, many, many crypto projects, the number of cursor/getcursor accounts, the license violation and copying code without credit from an existing community group (with a reputation for expending a lot of effort, just to help other zig users), the abusive and personal attack editing the PR asking, for nothing but crediting the source of the code he tried to steal. All the while asking for donations for the work he copied from others.

All of that punctuated by the the fact he seems to have plans to typo squat Zig users given he controls the `zigglang` account on github. None of this can reasonable be considered just a simple mistake on a bad day. This is premeditated malicious behavior from someone looking to leach off the work of other people.

People are mad because the guy is a selfish asshole, who has a clear history of coping from others, being directly abusive, and demonstrated intent to attempt to impersonate the core ziglang team/org... not because he dared to use AI.

grayhatter··on Zig quits GitHub, says Microsoft's AI obsession has ruined the service
> I agree with you. It is shitty behavior to say it is not AI written when it clearly is.

> But I also think we at this point should just assume that everything is partially written using AI.

Using "but" here implies your 2nd line is a partial refutation to the first. No one would have been angry if he'd posted it without clearly lying. Using AI isn't what pissed anyone off, being directly lied to (presumably to get around the strict "made by humans" rules across all the various Zig communities). Then there was the abusive PR edits attacking someone that seems to have gotten him banned. And his history of typosquatting, both various crypto surfaces, and cursor, and the typosquatting account for zigglang. People are mad because the guy is a selfish asshole, not because he dared to use AI.

Nothing I've written has been assisted by AI in any way, and I know a number of people who do and demand the same. I don't think it's a reasonable default assumption.

grayhatter··on Zig quits GitHub, says Microsoft's AI obsession has ruined the service
If you want to measure the language used by the productivity of the desired outcome. I'd encourage you to survey the ratio of comments talking about the problems with github's very broken CI and UX, with how many people expressed an objection to the language and words used in the announcement. Failure to convey ideas with tact and respect, is demonstrably more counter productive.

I assume you'll choose to dismiss those who object as fragile birds... but then you don't really care about the productivity towards the goal then do you? You just want to be ok with being mean because it doesn't bother you.

grayhatter··on Zig quits GitHub, says Microsoft's AI obsession has ruined the service
Intellectual work requires a bit of creativity (across all the domains I can think of), abuse, of any kind increases stress, stress decreases creativity, ability to problem solve, and resilience (or the ability to endure the difficulty of solving hard problems).

But even if that wasn't true. There's a significant difference between confronting the harshness of reality. And behaving in a way that makes reality suck more. Every human deserves to be treated with dignity, and a base level of respect.

Suggesting that someone is fragile and weak, because they object to being insulted, or object to the careless and needless stripping of dignity and humanity from people is a wild take.

grayhatter··on How to Attend Meetings
> As someone who leads meetings and sets agendas, I’ve basically accepted there is no perfect meeting for everyone. I regularly get oscillating feedback. One day I’ll hear “We spent too long on a couple of topics and didn’t get to enough topics” then if we try to start limiting topics to a certain amount of time I’ll hear “we never actually _solve_ anything, it’s all just too high level!”.

Are you sure it's not that you're just bad at leading meetings?

I ask that tongue in cheek, but my thesis is the same. How are you measuring the positive criteria for a meeting? How would you know if some variable has a meaningful impact on said metrics? Are the metrics you're tracking the same metrics others are using, and if not how do you translate them?

Most meetings lack a clearly defined success criteria. Most attendees couldn't describe this criteria, even if pressed. Given my experience, that's the root problem.

People who are trying, often use "this meeting has an agenda" as criteria for if a meeting is likely to be useful. But this is a heuristic detached from what is actually important. Meetings are about, obtaining consensus, or uncovering some truth*. If your success criteria doesn't reflect either of these. It's much more accurately described as a waste of time, rather than a meeting.

Pretend meetings aren't a thing, you're requesting a significant amount of time from a number of people. Now, on top of that, add in the cost of context switching. You're proposing a completely novel approach to solving a specific problem. Define that problem.

Most of the meetings I've attended evaporate under that criteria.

Meetings without a doubt, solve real problems. But most meetings aren't solving any problems. They're checking boxes, because that's what people expect. Which results in the pattern you describe. it's either a waste of time, or a waste of time, in the other direction.

> The best I can do when I lead these is gauge the importance of each topic myself, which is not a perfect science, and allow time to run over for important topics.

It still sounds like to me, you're gauging the quality of a meeting, based mostly on the time cost. That shouldn't be considered*. Instead, assume you have infinite time. In this magical world, every sits in this room until you've arrived at [objective]. Pretend that amount of time might as well be infinite. Is the objective worth infinite time? Or can you still not describe the objective outside of the time cost?

> Sometimes, things just work out super well. We touch on everything people want to touch on, we fly through it and everyone leaves the meeting happy. If I’m honest though, the biggest predictor of that outcome seems to be the mood of people coming into the meeting. Meetings after long weekends get above-average reviews.

There's a nugget of truth, or more accurately reality behind this observation. Meetings that are rated positively, correlate strongly with context alignment. What concrete meaning have you taken from this critical observation?

If you have already have clear alignment, what impact should that have on the next meeting?

Imagine "next meeting" sounded like an absolutely ridiculous question, why would you ever consider having a "next meeting"? What a stupid question for some rando on the Internet to ask?!

Once you shift your thinking into being able to answer, why on earth, there would be a follow up meeting... You'll understand how to extract value from meetings.

Also, do note... sometimes meetings are just to hang out and shoot the shit. This might be more important than [average meeting] so don't undervalue the real benefits of spending time with coworkers! I've lost count of the number of difficult problems I've solved by casually ranting to a friend who asks smart questions. (which I take to mean, don't step on important conversations, for the sake of some bullshit agenda... try asking people if they felt like the conversations in a meeting were friendly and welcoming, see what that question does for your success criteria long term)

grayhatter··on Writing a good Claude.md
Can you cite a section from this very long page that might convince me no one at the time understood how turning water into steam worked to create pressure?

If this is your industry, shouldn't you have a more reputable citation, maybe something published more formally? Something expected to stand up to peer review, instead of just a page on the internet?

> We should not be making Luddite calls to halt progress simply because our analytic capabilities haven’t caught up to our progress in engineering.

You've misunderstood my argument. I'm not making a luddite call to halt progress, I'm objecting to my industry which should behave as one made up of professionals, willingly sacrifice intellectual control over the things they are responsible for, and advocate others should do the same. Especially not at the expense of users, which I see happening.

Anything that results in sacrificing the understanding over exactly how the thing you built works is bad should be avoided. The source, either AI or something different, doesn't matter as much as the result.

grayhatter··on Writing a good Claude.md
> It took 100 years to explain the steam engine. That didn’t stop us from making factories and railroads.

You keep saying this, why do you believe it so strongly? Because I don't believe this is true. Why do you?

And then, even assuming it's completely true exactly as stated; shouldn't we have higher standards than that when dealing with things that people interact with? Boiler explosions are bad right? And we should do everything we can to prove stuff works the way we want and expect? Do you think AI, as it's currently commonly used, helps do that?

grayhatter··on Writing a good Claude.md
> As a matter of fact, commercial passenger service started almost immediately as the tech was out of the fiction phase. The airship were large, highly experimental, barely controllable, hydrogen-filled death traps that were marketed as luxurious and safe.

And here, you've stumbled onto the exact thing I'm objecting to. I think the Hindenburg disaster was a bad thing, and software engineering shouldn't repeat those mistakes.

> Very well, then Mr Tinkleberry also passes the bar because it's exactly such a trick. That it irks you as a cheap hack that lacks rigor (which it does) is another matter.

Yes, this is what I said.

> there's a critical difference between engineering research, and experimenting on unwitting people.

I object to watching developers do, exactly that.

grayhatter··on Writing a good Claude.md
> Really for those of us who actually work in critical systems (emergency services in my case) - of course we're not going to start patching the core applications with vibe code.

I used to believe that no one would seriously consider this too... but I don't believe that this is a safe assumption anymore. You might be the exception, but there are many more people who don't consider the implications of turning over said intellectual control.

> But yeah, that frankenstein reporting script that half a dozen amateur hackers made a mess of over 20 years instead of refactoring and redesigning? That's prime fodder for this stuff. NOBODY wants to clean that stuff up by hand.

It's horrible, no one currently understands it, so let the AI do it, so that still, no one will understand it, but at least this one bug will be harder to trigger.

I don't agree that harder to trigger bugs are better than easy to trigger bugs. And from my view, the argument that "it's currently broken now, and hard to fix!" Isn't exactly an argument I find compelling for leaving it that way.

grayhatter··on Writing a good Claude.md
> We used the steam engine for 100 years before we had a firm understanding of why it worked. We still don’t understand how ice skating works. We don’t have a physical understanding of semi-fluid flow in grain silos, but we’ve been using them since prehistory.

I don't think you and I are using the same definition for "firm understanding" or "how it works".

> I could go on and on. The world around you is full of not well understood technology, as well as non deterministic processes. We know how to engineer around that.

Again, you're side stepping my argument so you can restate things that are technically correct, but not really a point in of themselves. I see people who want to call themselves software engineers throw code they clearly don't understand against the wall because the AI said so. There's a significant delta between knowing you can heat water to turn it into a gas with increased pressure that you can use to mechanically turn a wheel, vs, put wet liquid in jar, light fire, get magic spinny thing. If jar doesn't call you a funny name first, that's bad!

grayhatter··on Games using anti-cheats and their compatibility with GNU/Linux or Wine/Proton
> I don't need gaming that much.

Counter point: gaming is fun, and indy games are worth investing in. Voting with your wallet works better if you vote for behavior that's not user hostile, rather than only abstaining.

grayhatter··on Don't push AI down our throats
> Just stop talking to your computer and use the screen interface, that still works.

This reply demonstrates you don't understand the problem. Please don't contribute to the enshittifacation of everything by being an apologist for unethical behavior.

grayhatter··on Writing a good Claude.md
> There's no need to be upset with the fact that some people want to hack explosive stuff together before it becomes a predictable glacier of Real Engineering.

You misunderstand me. I'm not upset that people are playing with explosives. I'm upset that my industry is playing with explosives that all read, "front: face towards users"

And then, more upset that we're all seemingly ok with that.

The driving force of enshittifacation of everything, may be external, but degradation clearly comes from engineers first. These broader industry trends only convince me it's not likely to get better anytime soon, and I don't like how everything is user hostile.

grayhatter··on Writing a good Claude.md
> Yes, during the Wright era aeronautics was absolutely dominated by tinkering, before the aerodynamics was figured out. It wouldn't pass the high standard of Real Engineering.

Remind me: did the Wright brothers start selling tickets to individuals telling them it was completely safe? Was step 2 of their research building a large passenger plane?

I originally wanted to avoid that specific flight analogy, because it felt a bit too reductive. But while we're being reductive, how about medicine too; the first smallpox vaccine was absolutely not well understood... would that origin story pass ethical review today? What do you think the pragmatics would be if the medical profession encouraged that specific kind of behavior?

> It wouldn't pass the high standard of Real Engineering.

I disagree, I think it 100% is really engineering. Engineering at it's most basic is tricking physics into doing what you want. There's no more perfect example of that than heavier than air flight. But there's a critical difference between engineering research, and experimenting on unwitting people. I don't think users need to know how the sausage is made. That counts equally to planes, bridges, medicine, and code. But the professionals absolutely must. It's disappointing watching the industry I'm a part of willingly eschew understanding to avoid a bit of effort. Such a thing is considered malpractice in "real professions".

Ideally neither of you to wring your hands about the flavor or form of the argument, or poke fun at the gamified comment thread. But if you're gonna complain about adding positively to the discussion, try to add something to it along with the complaints?

grayhatter··on Writing a good Claude.md
> You don’t think people are trying very hard to understand LLMs? We recognize the value of interpretability. It is just not an easy task.

I think you're arguing against a tangential position to both me, and the person this directly replies to. It can be hard to use and understand something, but if you have a magic box that you can't tell if it's working. It doesn't belong anywhere near the systems that other humans use. The people that use the code you're about to commit to whatever repo you're generating code for, all deserve better than to be part of your unethical science experiment.

> It’s not the first time in human history that our ability to create things has exceeded our capacity to understand.

I don't agree this is a correct interpretation of the current state of generative transformer based AI. But even if you wanted to try to convince me; my point would still be, this belongs in a research lab, not anywhere near prod. And that wouldn't be a controversial idea in the industry.

grayhatter··on Writing a good Claude.md
I am blaming the developers who use AI because they're willing to sacrifice intellectual control in trade for something that I find has minimal value.

I agree it's likely to be a complex or intractable problem. But I don't enjoy watching my industry revert down the professionalism scale. Professionals don't choose tools that they can't explain how it works. If your solution to understanding if your tool is still functional is inventing an amusing name and trying to use that as the heuristic, because you have no better way to determine if it's still working correctly. That feels like it might be a problem, no?

grayhatter··on Writing a good Claude.md
> I suspect you’re misjudging the friend here. This sounds more like the famous “no brown m&ms” clause in the Van Halen performance contract. As ridiculous as the request is, it being followed provides strong evidence that the rest (and more meaningful) of the requests are.

I'd argue, it's more like you've bought so much into the idea this is reasonable, that you're also willing to go through extreme lengths to recon and pretend like this is sane.

Imagine two different worlds, one where the tools that engineers use, have a clear, and reasonable way to detect and determine if the generative subsystem is still on the rails provided by the controller.

And another world where the interface is completely devoid of any sort of basic introspection interface, and because it's a problematic mess, all the way down, everyone invents some asinine way that they believe provides some sort of signal as to whether or not the random noise generator has gone off the rails.

> Sounds like the friend understands quite well how LLMs actually work and has found a clever way to be signaled when it’s starting to go off the rails.

My point is that while it's a cute hack, if you step back and compare it objectively, to what good engineering would look like. It's wild so many people are all just willing to accept this interface as "functional" because it means they don't have to do the thinking that required to emit the output the AI is able to, via the specific randomness function used.

Imagine these two worlds actually do exist; and instead of using the real interface that provides a clear bool answer to "the generative system has gone off the rails" they *want* to be called Mr Tinkerberry

Which world do you think this example lives in? You could convince me, Mr Tinkleberry is a cute example of the latter, obviously... but it'd take effort to convince me that this reality is half reasonable or that's it's reasonable that people who would want to call themselves engineers should feel proud to be a part of this one.

Before you try to strawman my argument, this isn't a gatekeeping argument. It's only a critical take on the interface options we have to understand something that might as well be magic, because that serves the snakeoil sales much better.

> > Is the magic token machine working?

> Fuck I have no idea dude, ask it to call you a funny name, if it forgets the funny name it's probably broken, and you need to reset it

Yes, I enjoy working with these people and living in this world.

grayhatter··on Writing a good Claude.md
> A friend of mine tells Claude to always address him as “Mr Tinkleberry”, he says he can tell Claude is not paying attention to the instructions on Claude.md, when Claude stops calling him “Mr Tinkleberry” consistently

this is a totally normal thing that everyone does, that no one should view as a signal of a psychotic break from reality...

is your friend in the room with us right now?

I doubt I'll ever understand the lengths AI enjoyers will go though just to avoid any amount of independent thought...

grayhatter··on I don't care how well your "AI" works
The take was supposed to be read as slightly hyperbolic. Because while the fastest user of an IDE, has never come close to the fastest I've seen in vim, as you pointed out, thats not really a reasonable comparison either. Here I'm intentionally only considering raw text editing speed, jumping across lines, switching files. If you're including IDE features, when you expect someone in vim to leave vim, you're comparing something that doesn't equate to my strawman.

My larger point was it's absurd to say someone who's faster using [interface] is a luddite because they don't use [other interface] with nearly identical features.

> Oh wait. No true vim user would come to this conclusion, right?

I guess that's fitting insult, given I started with a strawman example too.

edit: I can offer another equally absurd example, (and why I say it's only slightly hyperbolic because the following is true), I can write code much faster using vim, than I can with [IDE], I don't even use tab complete, or anything similar either. I, personally, am able to write better code, faster, when there's nothing but colored text to distract me. Does that make me a luddite? I've tried both, and this fits better for me. Or is it just how comfortable you are with a given interface? Because I know most people can find tab complete useful.

grayhatter··on I don't care how well your "AI" works
> Many people actually are becoming more productive. I know you're using quotes around productive to insulate yourself from the indignity of admitting that AI actually is useful in specific domains.

Equally, my read is you're fixating on the syntax used in their comment to insulate yourself from actually engaging with their idea and point. You refuse to try to understand the parts of the system that negate the surface level popularity, eer productivity gains.

People who enjoy the productivity boost of AI are right, you can absolutely, without question build a house faster with AI.

The people who claim there's not really any reasonable productivity gains from AI are also right, because using AI to build a multistory anything, requires you to waste all that time starting with a house, to then raze it to the ground and rebuild a usable foundation.

yes, "but its useful in specific domains" is technically correct statement, but whataboutism is rarely a useful conversational response.

grayhatter··on I don't care how well your "AI" works
> It wasn't caught in review because the file that was changed didn't require my teams review (because we ship more slowly, they removed us as code owners for many files recently).

I've lost your fight, but won mine before, you can sell this as risk reduction to your boss. I've never seen eng win this argument on quality grounds. Quality is rarely something that can be understood by company leadership. But having a risk reduction team that moves a bit slower and protects the company from extreme exposures like this, is much harder to cut from the process. "Imagine the law suits missing something like this would cause." and "we don't move slower, we do more than the other teams, the code is more visible, but the elimination of mistakes that will be very expensive legally and reputationally is what we're the best at"

grayhatter··on I don't care how well your "AI" works
lol, no. They're people who think faster. Someone who uses vscode will never produce code faster than someone proficient in vim. Someone who clicks through GUI windows will never be able to control their computer as fast as someone with a command prompt.

I'm sure that there are some examples who enjoy it for the interface. I think CRT term/emulator is peak aesthetic. And a few who aren't willing to invest the time to use a gui an a terminal, and they learned the terminal first.

Calling either group a luddite is stupid, but if I was forced to defend one side. Given most people start with a gui because it's so much easier. I'd rather make the argument that those who never progress onto the faster more powerful options deserve the insult of luddite.

grayhatter··on APT Rust requirement raises questions
> but let's not pretend it's a situation that people are in due to economic hardship.

I'd encourage you to not strawman my response. Because I already said myself that it appears to me it's only hobbyists who are losing support.

My objection isn't to the argument that it's dropping support, my objection is that it's dropping support without cause. Other than, the assumed this would be more comfortable for me.

Maintainers are absolutely not required to support everything for ever, but I recall a story where someone from Linux paid for a user to upgrade, not because that was required, because more because that would make dropping support for that floppy driver feel ethical.

This is the level of compassion everyone should expect from software engineers in critical positions of power.

I have no sympathy for people who lack the compassion to expend the effort to help others. I do have sympathy for people who have to watch the world that they, even if it's them alone. Have to watch their world get worse, so that others can avoid a trivial amount of perceived discomfort.

Should this solo maintainer (who understands C) be required do things exactly the way that I want? Of course not, but I'll be damned if everyone expects me to remain silent while I watch them disrespect other people who were previously depending on their support.

grayhatter··on APT Rust requirement raises questions
> You can grab a $150 NUC

I grew up without money, it makes me laugh when I read comments like this. You can just, yeah when you're fortunate enough to have a strong support system; you can.

My understanding is that the systems are not meaningfully common, and are hobbyist archs. But the idea that dropping support is fine because you can just throw money at it is so incredibly divorced from reality that I actually feel bad for anyone that believes this.

I deeply believe that if you don't like what a maintainer of FOSS code has done, you should fork the project. Admittedly that's a very onerous suggestion. But more important than that, you should help people when you can. If you're deciding to drop support for a bunch of people because it makes your job easier or simpler, when you don't need to. You're the bad guy in the story. That's the way this announcement has been written, and most reasonable people object to that kind of behavior. Selfishness should feel a bit offensive to everyone.

grayhatter··on APT Rust requirement raises questions
That was needlessly rude.
grayhatter··on Personal blogs are back, should niche blogs be next?
Wow, I really wanted to like this site. The domain name is perfect. But it's a sad example of what's wrong with the web.

The top 25% is the nav bar, ok, maybe that's reasonable? the next 25% is a "new here", please join upsell. The next 20% is another upsell, please give me your email so I can spam you. The bottom 35% is tags, and the bottom 25% is finally links... but they're mostly covered by another upsell. Holy shit!

And to top it all off, even with uBo enable there's a full sceen popup/modal add dimming the whole page, begging me to let them help me with my overwhelming debt.

WTF is wrong with people who do this stuff?!

grayhatter··on Personal blogs are back, should niche blogs be next?
I don't read AI generated output.

I read and enjoy content from people who want to share something.

There is no chance I'll read anything within 0.01s of it being posted, but there is a chance I'll find it eventually. So if the first thing to read the content is all that matters, or if having complete exclusive control so that only the people you like can read it, and importantly only in the way that you decided they should read it.

Then don't bother?

But if you want to create something of value for someone else, and the exact time it takes them to find it, or the browser they use isn't actually important. Seems like it's still a good time to write stuff. Maybe even better, to resist AI content drowning out new stuff.

grayhatter··on Personal blogs are back, should niche blogs be next?
The most depressing?

More depressing than the realization how few people are able to evaluate the quality of something?

More depressing than the number of people who proudly advocate for having pride in plagiarism?

More depressing than the number of people who unwittingly use genai as an excuse to DoS sites, individuals, organic content ranking? (Each being their own completely diff method)

grayhatter··on Personal blogs are back, should niche blogs be next?
> I just know that what I write will be endlessly picked on, probably mocked in some cliquey discord channel

So if you started creating content to teach other people, it would then live rent free in the heads of people who cant do it? If anything, that sounds like a reason to start blogging.

I don't mock people who write about stuff I already understand. https://xkcd.com/1053/ was kind enough to grant me immunity to that. I do mock people who write incorrectly from authority, without curiosity. Equally I look down on those who try to use their knowledge as a cudgel against people still learning. I would mock those trying to weaponize knowledge, but I try not to punch down.

If I wrote something, and found out it was being mocked in some discord channel, that would make me laugh, much more than worry or feel ashamed. In order for me to value some criticism, I would first have to value or desire their opinion. I can't imagine a single person who's feedback I would be interested in, if their version of humor is trying to dunk on someone who is learning, and sharing that learning. They're too stupid for me to waste my limited attention to care about what they ̶t̶h̶i̶n̶k̶ say.

Chances are, there is someone out there who would be grateful you explained something in a way they could understand, when everyone else failed. That's how the intersection of knowledge and shared context works.

You don't need to value the opinions of idiots. Let them be idiots. It's ok, and even desired to focus your attention on making something good with value to people who aren't stupid.

grayhatter··on Kagi Assistants
> That's a fair comment.

Equally, And despite my disagreement, I do genuinely appreciate the reply, especially given my dissent.

> Just to give my point of view: I'm head of ML here, but I'm choosing to work here for the impact I believe I can have. I could work somewhere else.

> As for the net positive effect, the point of my essay is that the trust relation you raise [...] to me is a product design issue.

Product design, or material appropriability issue? Why is the product you're trying to deliver, based ontop of a conversational model? I know why my rock climbing rope is a synthetic 10mm dynamic kernmantle rope, but why is a conversational AI the right product here?

> LLMs are fundamentally capable of bullshit.

Why though? I don't mean from a technical level, I do understand how next token prediction works. But from a product reasonability standpoint? Why are you attempting to build this product using a system that you makes predictions based on completely incorrect or inappropriate inputs?

I admittedly, am not up-to-date on state of the art, so please do correct me if my understanding is incomplete or wrong. But if I'm not mistaken, generally, attention based transformers themselves dont hallucinate when producing low heat language to language translations, right? Why are conversational models, the ones very much prone to hallucinations and emitting believable bullshit the interface everything uses?

How much of that reason is because that ability to emit believable bullshit, is actually the product you are trying to sell? (The rhetorical you, I'm specifically considering LLM as a service providers egear to over sell the capabilities of their model. I still have a positive opinion about Kagi, so I could be convinced you're the ones who are different) The artificial confidence is the product. Bullshitting something believeable but wrong has better results, in bulk, for the metrics you're tracking. When soliciting feedback, the vast majority of the answers are based on vibes, right?

If you had two models, one that was rote and very reliable, very predictable, rarely produced inaccurate output. But wasn't impressive when trying to generate conversational feeling text, and critically, was unable to phrase things in a trivial to understand way exuding an abundance of confidence. Contrasted with another that very very rarely would produce total bullshit, but all the feedback shows everyone loves using that model. But it makes them feel good about the answer, yet there's stil that nagging hallucination issue bubbling under the surface.

Which would you ship?

Again, I'm asking which would you ship with the rhetorical you... perhaps there is someone in charge of AI that would only ship the safe version, even if few users ranked it higher than normal organic search. Unfortunately I'm way too much of a cynic to believe that's possible. The AI is good crowd doesn't have a strong reputation for always making the ethical selection.

> So products that leverage them have to keep that in mind to build workflows that don't end up breaking user trust in it.

> The way we're currently thinking of doing that is to keep the user in the loop and incentivize the user to check sources by making it as easy as possible to quickly fact check LLM claims.

Do you feel that's a reasonable expectation from users when you've already given them the perfect answer they're looking for with plenty of subjective confidence?

> I'm on the same page as you that a model you can only trust 95% of the time is not useful because it's untrustable. So the product has to build an interaction flow that assumes that lack of trust but still makes something that is useful, saves time, respects user preferences, etc.

> You're welcome to still think they're not useful for you, but that's the way we currently think about it and our goal is to make useful tools, not lofty promises of replacing humans at tasks.

I don't think I'm the ideal person to be offering advice. Because I would never phrase the problem statement as "we have to give users to tools to verify if the confident sounding thing lied this time" I know far too much about both human nature, and alarm fatigue. So I can only reject your hypothetical, and ask what if you didn't have to do something I worry will make the world worse.

I attribute a large portion of the vitriol, anger, and divisiveness that has become pervasive, and is actively harming people and communities; as stemming directly from modern algorithmic recommendation systems. These systems prioritize speed, and being first, above the truth. Or they rank personalized results, that selectively offer only the content that feels good, and confirms preexisting ideas, to the detriment of reality.

They all tell you want you want to hear, over what is true. It will take a mountain of evidence to assure me, conversational LLMs wont do exactly the same thing, just better or faster. Especially when I could uncharitably summarize your solution to these defects, as merely "encouraging people to do their own research"

← PreviousPage 11 of 34Next →