Hacker News Reader

Comment by kozikow | Hacker News Reader

At least for me, personal agents have been quite useful. If you integrate them well - shopping, filling random government forms, keeping track of my personal life, optimizing training plan and nutrition, planning vacations, etc. I can do more home repairs or renovations myself with help of chat

So basically you get personal digital assistant - very useful - but not changing my life.

What would I need for real transformation though? Food, house cleaning, commute - "every day chores". Even if we had ASI tomorrow, what could it do to change my life, besides taking away my job? Design good house robot to help me with food preparation I guess... Maybe finally self driving cars launched in my area? Or it will take control, "optimize humanity survival chance" and realize we don't need as many people on this planet.

View on HN

> besides taking away my job

> self driving cars launched in my area

It sounds like the transformation would be a massive collapse of the economy with scores of people unemployed.

> Or it will take control, "optimize humanity survival chance" and realize we don't need as many people on this planet.

Maybe, but more likely it will be the humans who control the AIs and hold the power who will make this decision.

Yes, and additionally, on the other side, there are a bunch of people using it to build exploitative products, spam, weapons, etc.

Personal agents are wonderful! You can ask for hiking trails and it will customize one for you, frequently resulting in emergency calls for rescues! Your purchasing/selling AI will arrange sales for you but forget to tell you, leaving you in breach of contracts you didn't know about! Your personal nutrition agent will absorb all the broscience and customize a diet that will result in hair loss, but at least your in ketosis, amiright! And instead of hiring a plumber/electrician/contractor to do work subject to code, you can just ask an AI to tell you how to do it and then wonder why your buyer is suing you for $$$!

(Those are all examples from just the past 4 days, and just the ones that made it into mainstream media.)

Almost every time I use an AI chatbot it eventually turns out it's lying to me. I would not trust them with any of the things you mention.

> Almost every time I use an AI chatbot it eventually turns out it's lying to me. I would not trust them with any of the things you mention.

If you talk to free chatgpt, sure.

But if you launch Opus 5.5 in claude code - for some reason Opus is the smartest for me, even for "general" type of questions, when launched from Claude Code. It has harness/prompts optimized for correctness, while more "customer-grade" products, even when used with top model tend to optimize for latency. And I tend to give "context" - e.g. I have "personal wiki" + skills for lots of those things.

I don't recall i last few weeks where Opus 5.5 in CC made an obvious lie, made an unreasonable assumption, or made a mistake that smart human couldn't make. E.g. if it "failed" it failed in a way that non-genius human could fail in similar way.

opus 5.5 uses agents and skills in chat sessions as well and maintains internal context that it tries to distill from what you provide

I tried to have it write a synopsis website for the different types of dance patterns in Dance Dance Revolution, with guides on their execution, explanations, and provided song sources. I provided all of the source articles and search websites needed for it to be able to do that, and gave it a sample set I wanted it to utilize and build off of.

It has bits and pieces of information wrong in 90% of the sections it created. Poor song choices, missing song choices, incorrect explanations, incorrect step-patterns, the list goes on.

If you're not an expert in a given field for which you're using it, it's going to look like a genius. If you are, it's untrustworthy and borderline useless.

Maybe it wouldn't be if there were ample training data on my topic, but there isn't.

It utilized the research from millions of papers many very close to the destination answer to make those proofs. What would it do on something that isn't supported like that? Be confidently wrong while sounding right. That's what we've seen repeatedly.

> If you're not an expert in a given field for which you're using it, it's going to look like a genius.

For someone highly literate, llms aren't going to help you write better. If you're borderline functionally illiterate (this is > 50% of the population according to some studies), I'm sure the llms are awesome.

I've long thought the same about all the hype around coding (plenty of examples from years ago now), but will continue to test the new models. They are improving.

I think my anxieties would be different if I were confident a DDR guide full of lies would be spurned and unprofitable.

In a way, I'm not worried about the LLMs per se as much as a kind of moral and ethical collapse. A debasement of the currency of communication, a market of intellectual-lemons.

I saw electroboom's latest video lamenting facebook AI slop about electricity, such as a way to wire a three-way switch that blows a fuse when you turn the light on. For some reason, I am shadowbanned on HN.

I see your comment, perhaps it was a delay between representations or nodes?

It’s sort of weird how

1) open weights and cheap models are only about 6 months behind the frontier in evals, but…

2) it’s always the premium model that only the biggest fans pay for, that’s just now finally achieved enlightenment.

Oh well, I guess we’ll see in another 6 months.

I have ChatGPT Pro and it sources every single thing. For my Asia trip right now it's been helping me every single day to figure out optimized routes on what to do and where to go (if I feel like it, not to say I don't wander around too).

That sounds terrible. I like to live like a human being

What that does that even mean? Even if I grant your premise (I don't), if one is traveling and engaging these places in person....what's the problem?

Where did I say it was a problem? Using chatgpt to plan an itinerary sounds terrible TO ME

Why? In the past I would've used Google so it's just a faster search engine for me that works with natural language, what do you use for planning then?

Indeed, planning an itinerary, making calculations, building excitement for a place, is part of the journey

I suspect the HN techno-brained crowd will not appreciate it, many are quick to throw away their human experience

> Indeed, planning an itinerary, making calculations, building excitement for a place, is part of the journey

Not sure how they're perceiving using ChatGPT to plan an itinerary but I'm also doing all this while using ChatGPT. As I said in the sibling comment, it's a faster Google search and route optimizer, not something where I type in "make me an itinerary for X city" and follow whatever it shits out.

How often do you check the citations? I’ve found with the Google search AI summary it will provide answers and link to their citations, but a good chunk of the time the answer isn’t on the linked page.

It just makes stuff up (or gets it from somewhere, who knows) and then links to a tangentially related page as if it was the source. Either that or it identifies the page as being a source it wants to use, and then summarizes it incorrectly. Not sure the order of operations it’s doing internally, but regardless the citations are bullshit.

Google's models are worse that OpenAI's, doubly so on Google's free AI overview versus ChatGPT paid plans. 6.1 Sol is very good at synthesizing actual information it finds without hallucinating and I've not seen any instances it failed while I checked the sources.

The citations provided by the free version of Gemini are comically bad. They almost never contain information which directly supports the relevant claim, and more often than not they're completely irrelevant.

me too, but I recently blew the transformer on my boiler by following its directions to wire up a C wire to power a smart thermostat. oops. $500 mistake.

When is the last time you used an AI chatbot, and do you only use the free versions?

Which model? Super weird

Every time I use Google search AI for anything more complicated than a search query I feel that way. A paid model from pretty much anyone is much better.

Yeah, I’d hope that on HN, of all places, people know that free models optimize for price and speed, not for accuracy. But even with free models the accuracy was massively improved over the last year.

Agreed. Google search is still pretty rough, but it's hilariously better than it was when it was first released.

Apparently, I'm not supposed to extend that trend line though.

This is the kind of gap in shared understanding that is widening.

When people use words like “chatbot” to talk about their AI use and can’t really name basic things like the model they were using, it reveals a wide chasm between the people who are deep into understanding how to use these things and those who poke at easy options every few months and then look for ways to confirm their priors instead of trying to understand how other people are getting value.

using skills with opus 5.5 and agents for each individual task that have shared knowledge of all of your records does not lead to a significant increase in trustworthiness

The incentives seem to be lined up to push users to the frontier of "how much can the harness around the model do and claim without most people pushing back on the claims."

The bigger the pile of stuff you have the tool create unsupervised, the more incomplete/inaccurate the overall picture of it is gonna be. That doesn't matter for most hobbyist/helper-tool/game-port/etc type of tasks.

I get a ton of value out of the tools and yet part of that value has come from being hyper-aware of the shape of the limitations, which changes much less and more slowly than the specific of the limitations.

"How'd you notice that one!?" - well, it helps to be the only person on the team who took the time to interrogate the e2e interaction of all the code...

The median piece of OSS software was already pretty rough once you had to depend on it in production at scale to make money. The median piece of the new AI-generated wave of software, also pretty rough.

The bet that OpenAI/Anthropic is making is that they can automate the interactions generally/globally enough to work around this and eliminate the need for humans in the loop. But it's a tricky problem because it's precisely where the permutations and special cases get nasty and much harder to deal with than more broadly-applicable cases of codegen/test-gen/mash-it-until-the-tests-pass-and-the-code-review-angent-is-happy.

This has all started to feel very Emperor's New Clothes-ish. "Well, if it's not working for you, then it's your fault. You're just not using it properly. You're not worthy of its wonders".

Not to say there's no value to be had. However, it's also been oversold and its error-proneness is a real problem.

But, it seems there's a steady stream of commenters and creators who are quick to remind us that it's never a problem with the tech. It's always a user problem.

Meanwhile, the errors are renamed "hallucinations", but let's not focus on those. In fact, if you just look right over here at this .md file...

It's like driving a PT Cruiser and concluding all cars are bad and swearing off them entirely.

Are you deliberately performing "the thing" in your reply as irony, or have you found a PT Cruiser that doesn't hallucinate?

I am not sure what your question is, what is "the thing?" And what does a car have to do with hallucinations? Don't mistake a simile for what it's comparing.

> When people use words like “chatbot” to talk about their AI use and can’t really name basic things like the model they were using, it reveals a wide chasm between the people who are deep into understanding how to use these things and those who poke at easy options every few months and then look for ways to confirm their priors instead of trying to understand how other people are getting value.

While the latest and newest should be needed for greater capability, the average should be enough to give a glimpse pf what is possible. Like a basic netbook is enough to enjoy computing, no need to purchase the latest supercomputer.

If you need to switch your setup every quarter or so, it’s not a tool, it’s a rat race.

Reply on news.ycombinator.com