johnfn

Pretty impressive to accuse the author of not knowing formal verification when even minutes of research would immediately prove the opposite (https://x.com/victortaelin/status/2100942399132312059?s=46, https://x.com/victortaelin/status/2100374221671051472?s=46).

Some guy at OpenAI asked AI to solve a task and it ended up hacking RubyGems. Interesting that that never happened when I ran Bubblesort.

Wow totally missed this, thanks for sharing

I'd be happy to read a source as I am fairly confident that audio transcription -> facebook (or other) ads has never been true.

Models do not get nerfed. There has never been evidence of this. This would be trivial to prove if it were true, and such a proof would be a huge story and scandal to a news market hungry for a shred of a signal on AI's downfall.

This is the "your iPhone is listening to you and serving ads based on what you say" of the 2020s.

As a biker, I've had human drivers intentionally try to drive me off the road or hit me. Neither driver education nor higher test standards will solve that problem.

Are you claiming that an independent investigation is actually a marketing stunt?

Zitron is wrong, not "early", and the post has an extremely long list of examples.

The article claims that the finances fall apart because OpenAI won't hit the growth it wants. It's an interesting thing to say when using their product to write 100% of the article.

I really want to get in the mindset of people who are like "AI is totally going to fail! Haha! Now let me just use AI to write a piece about it..."

How is this a "gotcha" question? "gotcha" implies there's some sort of trick. This is just... a question.

How is it different from any other installation method?

It’s really fascinating to me how when the community dislikes certain content people always jump to conspiratorial justifications rather than the much more mundane “this content is not very good”. The first article is poor and all the comments on it say exactly that. The second one actually did fairly well for what is a fairly middle-of-the-road Ask HN. There’s no shadowy cabal removing anti-AI content from the front page.

I'm not sure how that changes the question -- just add `has_watermark(text) < 0.5`.

Sure, but I imagined it'd be something like Anthropic handing over this API only to trusted third-parties, not everyone in the world.

> We will soon be offering a watermark detection API. We’re in the process of working out the details of its implementation.

Dumb question - doesn't this defeat the purpose of a watermark? i.e., anyone who wants to avoid detection can simply run `while (has_watermark(text)) text = slightly_rewrite_with_non_anthropic_llm(text)` until it's gone? I feel I am missing the intent of the watermark if it is so easily defeated.

I remember when I raised this point like a year or two ago -- in response to someone saying that coding AI made all their work trivial I said something like "if you have multiple agents the work changes and becomes more managerial - don't you think that managers contribute value" and I got a bunch of downvotes and all the responses were like "no manager has ever contributed value." Ahh, good times.

This does look pretty incredible, but don't forget that incredible token thoroughput can only necessarily solve certain bottlenecks. If your e2e tests take an hour, they'll still take an hour after Ultracode. If the agent runs a 10 minute typecheck after a change, that will still take 10 minutes. grep over a massive codebase is still just as slow, etc. I say this not to take away from this accomplishment but just to ensure everyone here keeps a clear head about what it means - 14x faster tokens does not mean it completes every task 14x faster.

I suspect Humanity's Last Exam is without tool-calls, making it kind of the perfect benchmark to highlight how fast Ultrafast is, but not really the same as the everyday work you or I do.

I think “soulless” crosses a line into unkind. Just say you don’t like AI generated text.

> the one-word lines

The funny thing is that there are no "one-word lines". This more or less cements the fact that it must be AI in my mind.

Ouch, this seems unnecessarily harsh for a side project made for fun. This might be acceptable if this was a low effort single prompt to an AI but it’s pretty clear from the write up that that is far from the case.

FWIW I thought this was really cool.

384 is a tad small. How does it perform on 1k/2k/4k..?

AWS never "humblebrags" about an outage. No blog post looks as good as an extra 9.

Companies post deeply embarrassing eng blogs all the time. See: every post about downtime or a security incident ever.

> If their AI finished it, where's the release?

`bun upgrade --canary`. You can use it today.

Keeping a large release, like a total rewrite, on canary for a while to iron out bugs is a very reasonable software engineering practice.

This is really good - miles ahead of almost all other math writing I have read. One particular thing I really liked was the explanation of the determinant. When I was in college I spent a very long time trying to get some intuitive explanation for a determinant of a matrix and the best I could get is that you just multiplied and subtracted some numbers and poof there you go. I KNEW there had to be some more comprehensible explanation and it drove me crazy that people just parroted textbooks without any real understanding.

This reminds me of that silly video everyone was shown in middle school about inverting a sphere :-)

I am very curious how it was written (will there be a write up to the write up?!). It feels likely AI assisted, but this seems to me to be the best possible use of AI -- crunching out a bunch of visualizations of complex ideas so that they are easy to understand just by looking. The writing also feels a little AI written but definitely feels very human-assisted.

But surely at some point you need to stop, right? Most web developers are perfectly capable without understanding the intricacies of the file system or OS, even though the browser sits on top them. Most engineers don’t know assembly, or transistors or electrical engineering. Steve Jobs didn’t know every line of Objective-C that went into OSX. I don’t think Michelangelo collected all his paint materials by hand when painting the Sistine Chapel. In some sense I don’t understand a lot of the panic around AI when the entire career progression of what it means to be a software engineer has simply been to deal with ever-increasing levels of abstraction.

The response says "I also believe we can do better". How is that in any way "you're holding it wrong"?

To be blunt the degree to which I take these projects seriously is basically the same as how extensive their eval suite is. A single guided refactor is better than nothing, but one success could just be a fluke. I want to see hundreds, if not thousands, of evals to convince me you’re a project worth looking into.

The AI generated text is also not doing you any favors.

Pangram said a bunch of reasonable things and responded very coherently to the point and you skipped over it and misinterpreted this one section.