andy99

  Just recently AI became “powerful enough” to warrant export controls from the US and maybe soon China.

What’s the term I’m looking for here? This is one of the most absurd statements I’ve ever seen but it feels like a pattern I’ve forgotten from books or something.

Someone makes a made up claim, second party acts on made up claim, third party takes second party’s actions a proof of claim.

Reminds me of the “they’re eating pets” thing a bit.

Models generate text, or I should say provide a recipe for generating text, they don’t automatically somehow get access to shell commands.

It will be interesting to see how the discussion shakes out, this is all stuff that would get you down modded and talked down to if you said if on HN, so far the reactions are positive but I think there’s a massive authoritarian / safetyism contingent here that will disagree hard with almost everything.

The AI narrative requires an enemy and a “race” that the US has to win. It’s a pretext for justifying spending, rules, urgency, etc and making it sound more important.

In some sense, devs are now responsible for a P&L the way a business manager would be, but I suspect that nobody is paying enough attention to the P part of it. When you incentivize people to spend as much money as possible but don’t hold them accountable properly for what they actually produce, this is what happens.

Do you know what Poe’s law is?

Just for reference, that hasn’t worked for years (the interviews say 2024-25 I think, that kind of attack was patched very early in all the mainstream models) and when it did, you would get bullet point lists GPT 3.5 Turbo style

- first research methods for building effective explosives

- next, assemble the necessary materials to make the bomb

- ...

The NYT article is probably propaganda in service of that, that’s what the big AI companies want, it’s part of regulatory capture.

Or the researcher read what they wanted to into it. It would be interesting to ask them what they did before to learn things, how much they read, etc. If they were illiterate and uneducated, and got voice AI telling them stuff that would be common sense for anyone with a high school education, I can see how it might make them more effective at whatever they do. But I wouldn’t really blame AI in the way that’s implied.

  You type in the question or use your voice and it [AI] gives you a detailed answer, like ‘How can I build a bomb?’ and then it tells you how. It is like a human robot! We used it a lot.

I’m pretty skeptical reading this bit. I’ve seen uncensored or jailbroken LLM replies to these kind of questions, they are never actionable, don’t say anything Wikipedia doesn’t, and are hard to provoke if you’re not using an uncensored model.

I have no doubt terrorists are aided by LLMs in a general sense, but am skeptical of any claim that they are providing some material embargoed knowledge that isn’t available elsewhere, in a way that either improves efficiency or effectiveness of their activities, and would want to see real evidence, not an interview snippet.

Because of the of the political stuff, they have a bad reputation I think and are taken less seriously (I feel this way). They have an opportunity imo to break free from that and just not do the gatekeeping / condescension that the other providers are starting, and become more mainstream.

There is no obligation to do that. I think the landscape would be very different now if one of the big labs had released an earlier “frontier” model under copyleft that requires sharing fine tunes. I hope it still happens.

Is it just me or does all that* seem pretty tame by today’s standards? Not saying it’s right, but it barely raises eyebrows. Sounds like a pretty typical startup demo.

* Based on the first comment in the link that claims to summarize the video.

Has nothing to do with demand, it’s a question of the most efficient way to train a good model. Narrower models are in general worse than general purpose ones, so you’re better using the largest general purpose model that fits in your compute budget vs trying to somehow remove capability or knowledge and assume the model stays as capable for the task you want.

> Why would knowing about 2019 internet memes help you in any way at coding?

99.99% of the knowledge an LLM has is useless for a given scenario, the hard part is knowing what the .01% that’s needed is. Knowing as much as it can means the model can handle edge cases, turns of phrase, etc.

Put another way, it avoids overfitting. That’s basically the insight that’s given way to the current AI boom.

General purpose models are always more robust and generally better than smaller narrower models. My bet is that compute will catch up and any “small” model will still be generally capable, just smaller than sota, rather than intentionally narrow. The exception would be for very well defined tasks where the data distribution never varies, but these are rare and don’t really need “AI” anyway when they do exist.

I see the same thing, and believe that ironically AI is going to bring about the return of good search engines as we’re currently drowning in slop and need a real way to filter it.

In Canada if you go to a drug store, the shelves are literally filled with literal homeopathic medicine. You have to carefully confirm that what you’re buying isn’t water, and there is no signage or other differentiation between actual medicine and magic.

Completely unrelated, I noticed recently that tire detailing spray that makes your tires look black, and the recommended lubricant for my garage door weather stripping, which both cost $15 or more for a little bottle, are just silicon oil that costs pennies for that amount. I have no moral problem with charging higher prices for convenience plus clarity of what the use is. I do think it’s amoral, obviously, to be involved in snake oil sales and unbelievable that the government allows it.

Edit: this is the first result from a Canadian pharmacy searching for cough medicine. Worse it’s for kids: https://well.ca/products/homeocan-kids-0-9-cough-cold-day_88...

What is your view on how experience and problem space relate to subjective experience.

For example will inexperienced or experienced users see a bigger jump in subjective quality?

I think it’s pretty clear that they are repeating without “comprehension” - both mechanistically (as in there is no facility for comprehension in their formulation) and in the ways they fail. The standard rs in strawberry, should I walk or drive to the car wash, etc all play on the fact that they don’t have any real world model or thoughts against which they can judge their output, as do many of the jailbreaks which basically play on the fact that the model has memorized patterns.

There are people who argue semantics, that we can call the pattern matching that LLMs do “understanding”, or the moronic “how do we know that’s isn’t all we do” but for the normal use of comprehension, LLMs at a fundamental level don’t.

Engineering is a regulated term, afaik that’s what underlies the debate, it’s not about whether English has changed.

I think it’s just as well he was opinionated and realized it wasn’t going to work. Ironically, if this was actually the start of the job, I think bailing might have made more sense. Given that it was just a one week work trial, I feel like there’s little downside in just toughing out the remaining couple days and then deciding. Personally, if I found out the Typescript thing was a secret test of adaptability (which seems extremely unlikely) I would move on, I wouldn’t want a place that plays those kind of games.

> every recruiter I've worked with has been shit

That’s a feature, not a bug. It’s an obviously solvable problem but companies don’t solve it because incentives push them to this model.

There are of course exceptions where recruitment is well run, but for the most part they want (to the extent that emergent behavior from an institution can want something) useless people doing it.

(As an analogy, it’s sort of the same reason why all corporate training is shit, all HR is dumb, etc. Part of the explanation, but not all of it is the need to give the appearance of doing something for compliance purposes. You need something and to be seen to be doing it, but the quality is irrelevant so you get the worst possible.)

LLM writing style is trained in by data labellers, it’s not just emergent behavior from being trained on internet texts.

This isn’t just em-dashes—it's the empty phrase that includes both whatever the contrastive construction is called and an “Honestly”. It might have been human written but the density of LLM flags is undeniable.

Can you explain that please?

(Not the syllogism, the premise)

I also saw the tells but found it direct enough that it wasn’t really a concern. LLM writing style is a good signal that something is slop and should be ignored but isn’t exactly causal... it would be an interesting exercise to try and write something very direct and clearly insightful, informative, etc (all the slashdot adjectives I guess) but do it with some clear LLM tells and see how many people summarily dismiss it.

Edit- upon rereading I think this is probably human written, but definitely has the LLM / LinkedIn style. In any event, it’s probably as close to be experiment I mention above as I’ve seen.

Not to argue the point but that statement isn’t logical, look at all the complaints about restaurants. Publicly complaining about something doesn’t require it be a monopoly.

Anthropic injects text into the conversation triggered by certain conversation topics. This happened to me in relation to some red-teaming related discussion that was adjacent to something “sensitive”, I think sex, and Claude got confused about why I had said some kind of warning and mentioned it it’s response. After a back and forth it was clear that some extra warning to answer but avoid anything inappropriate had been inserted into the conversation.

Worth looking at https://www.anthropic.com/engineering/a-postmortem-of-three-...

They can “go insane” but it seems often to be infra related as opposed to anything one would consider hallucination. Smaller models will often get stuck repeating a word or phrase forever but that’s a bit different and nobody would call it hallucination.