_cs2017_

Sorry but I'm sure that's not what he meant.

Does anyone have AI agent use cases that that you think might happen within this year and that feels very exciting to you?

I personally struggle to find a new one (AI agent coding assistants already exist, and of course I'm excited about them, especially as they get better). I will not, any time soon, trust unsupervised AI to send emails on my behalf, make travel reservations, or perform other actions that are very costly to fix. AI as a shopping agent just isn't too exciting for me, since I do not believe I actually know what features in a speaker / laptop / car I want until I do my own research by reading what experts and users say.

"But for better or worse, the actual outcome does matter" -- curious what you mean by that?

This is solvable in roughly half an hour on pen and paper by a random person I picked with no special math skills (beyond a university). This is far from a difficult problem. The "95%+" in math reasoning is a meaningless standard, it's like saying a model is better than 99.9% of world population in Albanian language, since less than 0.1% bother to learn Albanian.

Even ignoring the fact that this or similar problem may have appeared in the training data, it's something a careful brute-force math logic should solve. It's neither difficult, nor interesting, nor useful. Yes, it may suggest a slight improvement on the basic logic, but no more so than a million other benchmarks people quote.

This goes to show that evaluating models is not a trivial problem. In fact, it's a hard problem (in particular, it's a far far harder than this math puzzle).

"The bitter lesson" didn't merely claim that a sufficiently large amount of compute would obsolete an engineered solution. Its claim was far stronger: the time it takes for the compute growth to catch up with the hand-engineered solution is so short that the investment in the latter won't pay off in the sense of a researcher's personal career investment, or in the sense of a big-tech R&D effort ROI.

You may well dispute such a claim, of course. Would be interesting to read your thoughts if you are willing to share them.

For comparison's sake, this is what the UK does:

https://www.youtube.com/watch?v=991kRp8KUmo

I see. So the timeline was:

1 Ship broken and delay announced until 2025

2 Musk offers his ship for return before 2025

3 Offer not accepted,

4 Pro-Musk claims the refusal was for political reasons, anti-Musk crowd claims it's not for political reasons but simply to comply with the original plan

Is that correct? If so, both groups agree with your comment, right?

The fault in the Starliner was known in June 2024 (https://www.space.com/nasa-boeing-starliner-crew-flight-test...). By August 2024, NASA said that the crew would stay until Feb 2025. That was not the original plan like you claim. See also: https://www.reuters.com/technology/space/nasa-astronauts-hea....

They wouldn't have any way of knowing this before choosing the female.

I suppose the can develop a more advanced social structure with prenups, courts, and police, but then we're talking about millions of years.

I wonder if people could just write their blogs posts in a short form: claim, argument in favor, counter-argument; consequences (this is optional).

Like this whole blog post could be:

Claim: Current AI is unlikely to usher in an era of dramatically accelerated scientific discovery.

Argument in favor: A genius does not come to life when you linearly extrapolate a top-10% student. Newton or Einstein is not just scaled-up good students. To create an Einstein, we need a system can ask questions nobody else has thought of or dared to ask. One that writes 'What if everyone is wrong about this?' when all textbooks, experts, and common knowledge suggest otherwise.

Existing benchmarks don't test such skills. And existing systems are likely hopelessly far from this capability (based on the author's personal feelings).

Counter-argument: none.

Consequences: obvious.

Skeptical as I am about the generative AI, the quality of this particular article (in terms of evidence provided, logic, insights, etc) is substantially lower than ChatGPT / Gemini DeepResearch can generate. If I was grading, I'd rate an average (unedited) AI DeepResearch report at 3/10, and the headline article at 1/10.

I also assign a 50% probability to the authors of the paper not realizing their mistake even after reading your post. I think competence is quite rare even in the academia.

I didn't mean that it's mathematically impossible. I meant that it's practically not gonna happen. Much as they'd love to, no VC gets a chance to invest in an early stage startup knowing that if the startup fails, there's a buyer ready to make the VC whole.

How can you get back the money that was spent? Fancy terms like M&A won't return the money that is gone.

Yes readers here.

I am pessimistic, I think only 2-3% would understand, but I'd be happier to be proven wrong than proven right.

Thank you for writing up your analysis!

Out of curiosity, what fraction of readers do you think would understand the points you're making? And what fraction of readers do you think would blame you for taking the wrong side of some debate?

Nope there's no attack here.

The training data is the Internet. It has mistakes. There's no available technology to remove all such mistakes.

Whether LLMs hallucinate only because of mistakes in the training data or whether they would hallucinate even if we removed all mistakes is an extremely interesting and important question.

I don't see Claude 3.7 on the official leaderboard. The top performer on the leaderboard right now is o1 with a scaffold (W&B Programmer O1 crosscheck5) at 64.6%: https://www.swebench.com/#verified.

If Claude 3.7 achieves 70.3%, it's quite impressive, it's not far from 71.7% claimed by o3, at (presumably) much, much lower costs.

Your footnote 3 seems to imply that the low number for o1 and Grok3 is without parallelism, but I don't think it's publicly known whether they use internal parallelism? So perhaps the low number already uses parallelism, while the high number uses even more parallelism?

Also, curious if you have any intuition as to why the no-parallelism number for AIME with Claude (61.3%) is quite low (e.g., relative to R1 87.3% -- assuming it is an apples to apples comparison)?

I can only confirm two mistakes in the apper: 1) As you say, the reversed(self.dict) is actually correct; 2) as another poster below said, hints are not part of the input. These two mistakes are so egregious given the objective of the paper that I'm convinced the authors are not qualified to write it.

IMHO, it is probably better to discard this paper, and wait for someone else to cover this important topic.

Wow thanks man for sharing! This is so unexpected, I thought you're trolling! But google search doesn't lie: https://www.history.com/news/national-state-of-emergency-us-....

Quoting from History.com: "When Donald Trump started his second term on January 20, 2025, the United States had around 40 active emergency declarations (no really, we are serious), including the national emergency George W. Bush declared in response to the 9/11 terrorist attacks".

Do people experience the frequent loss of part of the trading history when reviewing at round end on the browser version?

WTF dude, check your source (@deedydas). He seems to be posting garbage. The Gemini 2.0 Flash Thinking price isn't known yet. And on top of that, he gave the wrong number for R1 test results on AIME 2024 (it's 79.8%, far ahead of Gemini rather than far behind.

Wtf is wrong with you dude? It's just another tech, some jobs will get worse some jobs will get better. Happens every couple of decades. Stop freaking out.

As a reader, I prefer not to be misled by articles linked from the HN front page. So I do want to know whether someone is or is not a whistleblower. This has nothing to do with respect for the dead.

No internet access makes it very hard to benefit from o1 pro. Most of the complex questions I would ask require google search for research papers, language or library docs, etc. Not sure why o1 pro is banned from the internet, was it caught downloading too much porn or something?

Because it's not enough, you still need to do post-exposure shots (or else you still have a not exactly known but reasonably high risk of death). The main benefit of pre-exposure vaccination is that if somehow you have no access to the vaccine the day of exposure, you might still have a (small?) chance of survival. This benefit is nearly zero if you live within a driving distance from a hospital.

So why go through the trouble and pay $1-2k (plus more for boosters every 2-3 years) if you still need to go to the hospital after you get bitten?

Well... I kinda expect some people to be overly emotional. But I just didn't expect this particular group of people to be that.

Curious why there's so much emotion and unpleasantness in this dispute? How did it evolve from the boring academic argument about benchmarks, significance, etc to a battle of personal attacks?

Unsubstantiated claims accusing companies of criminal offense and implying that individual people are also criminally liable. This should be flagged.