daveguy

That's why the question was whether it gave sources, not whether they asked it to go check sources. Yeah, LLMs suck and you should always assume false information. 80% accurate is not production ready.

In theory.

Wait... Are anthropic and openai just commodity suppliers? Because dry cleaners and car washes both need soap, but I've never heard of a trillion dollar soap company.

Or AI has been over promising and under delivering for half a decade. Amazon is just one of the dominos. This is a hedge by Amazon.

No one has been saying LatinX for years. Why the anti-lib propaganda from 2022? Are y'all really that desperate to connect with folks?

Point is, nothing changes if you're just sitting on the internet complaining.

If everyone who threw up their hands actually voted, the outcome would change. See also, the elections musk tried to buy in the last 18 months or so since he doged up the place.

I don't think we have seen an Apple of AI yet. I wouldn't describe the frontier that way, would you?

I never said AI bad, bro.

Ruh roh. AI bros and their hundreds of billions of dollars in loans don't have decades.

Sounds like a trillion dollar company idea to me! The AI tech bros should get busy on that ASAP!!

Love to see an excellent example of writing that no AI could possibly write because it does not have human experiences to provide a unique perspective. Sadly, after they get hoovered up by the AI companies, these words will eventually be laundered through the plagiarism machine.

Yup. It's a great time to go into software development and learn without using AI as a crutch. The tech debt mountains won't be resolved with tech debt generators.

And most importantly, it doesn't matter how much AI you throw at a problem, it's not going to be able to figure out things like this. Because it doesn't have the experience of being a human.

Poison? Seriously? As if false references, perpetual false confidence, sycophantic suicidal encouragement, and convoluted spaghetti code wasn't enough. 90% of what these things fling is poison and the quicker the general public recognizes it the quicker they won't be duped into taking them at face value. The training sets are already going to be full of AI generated garbage. This just hastens the recognition.

Thank you. That's the exact understanding I was missing. I appreciate that.

Ahhhh. So they're all trained on the TESCREAL religion.

Oof. Like I said. Just fyi, for any of the HN'ers left "can we honestly say" is just a colloquialism for "is it reasonable to say, given..." Pretty sure my "given" wasn't nutso anti-science. Especially considering I am a scientist. HN was fun while it lasted. Peace out.

shit man. I was asking a question. So much for curiosity at HN.

Hah. I didn't think I was proposing anything new. I was telling you how it sounds to someone who isn't an astronomer.

I don't understand why it has to be non-electromagnetically interacting. Can we honestly say we should be able to resolve all the matter in a galaxy with our current sensing technologies? Why can't it just be the equivalent of non-light-emitting asteroids and planets. We can barely detect whether a star has planets in our own galaxy. And according to Wikipedia the amount of matter we would be missing in a sphere with radius from the sun to Neptune is 10^17 kg or approximately a large asteroid. Which seems like a rounding error. But my knowledge of astronomy is definitely minimal.

I have not come across a single piece of software that has improved in quality after the developers start using AI. Sure it does have the possibility, but that's not what AI is used for. It's used for moving fast and breaking things. And we are finding out exactly how bad that philosophy sucks.

> Because it's not the abliteration that does this. It's the "untrusted artifact" that does.

> Even a "clean" model of a lab could be doing this. Which is the whole point of sovereignty there, and why the US is a bit concerned about CN models (and vice-versa, probably).

Does it even need to be untrusted? From the article apparently it only takes ~250 poisoned documents in the training data to backdoor a model, and it survives safety training and fine tuning.

This has been known for over a year: https://www.anthropic.com/research/small-samples-poison

Which means any model trained after 2026 is succeptible to slurping up poisoned documents on the internet. We all know how prolific the crawlers are in a desperate attempt to slurp up everything on the internet. And all it takes is the lab not catching ~250 documents.

So these agents that can process anything on the internet through an LLM trained on the internet only have to stumble across the payload. Seems like agents running through LLMs is a terrible idea from a security perspective.

It would be great to have a list of these companies. I expect they would attract excellent talent when they are recruiting.

I think we'll be okay as long as we don't outsource our thinking to technology.

Wait... What's that you say?

Some people are producing vibe coded garbage with LLMs and some people are protesting and getting out of the business. It doesn't have to be the same people.

But yeah, people who are doing both could be forced to use LLMs at work and unable to leave. And when those LLMs inevitably fail because the codebase is too complex they have to refactor with LLMs because what they originally crapped out is incomprehensible and no one bothered to read and understand it while it was being developed. I expect a lot of people don't like that we are becoming dependent on funnelling money to a third party to maintain the crappy software they enable to begin with.

I think the entire industry is about to have another recognition about the importance of quality in software development. Just like they did with lines of code as a metric, and outsourcing with minimal communication.

Maybe it's time for SWE's to unionize.

Those two things are not mutually exclusive or inconsistent at all.

Yeah, but only LLMs shit the bed at computer speed.

Or, llms are completely unreliable to be consulted on any matter of life or death in either direction. And llms should not be used in the reckless way they are currently being used. Responsibility for reckless use should fall on both the company that makes the function available and the user for improper monitoring. If proper monitoring is not possible using the tools provided by the company then the responsibility should fall on the company. And if the user and the company providing the model are the same, then the responsibility is clearly with the company.