WhitneyLand

To answer that lets get more specific:

To buy presents for a family Christmas list Mom drives to Store A and Dad drives to Store B.

As more items get added to the list, they must decide who should drive to a new store location to buy the present. How can they minimize total driving distance while kids are randomly adding new items to their list?

The proof above guarantees its possible to never drive more than twice the mileage you would knowing all the items in advance.

The big news is this guarantee works for any number of drivers with any arrangement of gifts.

The algorithm to do this was already known, what we’ve learned is it’s not possible to do any better.

This is an important result, sometimes called the holy grail of competitive analysis.

One way to think about competitive analysis is bulk discounts. In life we’re constantly having to choose between quantity and discount. We could buy 1 item for a higher price, or say quantity 5 or 10 to get better discounts. The problem comes when we don’t know in advance exactly how many we’re going to need.

What should be our strategy for choosing how many to buy, and whatever the strategy is how well does it compare with having perfect knowledge upfront?

How is he very underrated?

I mean, I otherwise agree with the sentiment of your post but I feel like most people are rating him pretty well.

By that logic we should also consider the case of cutting the number of layers in half because that would also reduce hidden state between token generation.

In your generalized example I think the concern is when the additional evaluation effectively becomes a replacement for CoT, where something like the coconut research could replace it completely.

However, I don’t think we’re anywhere close to that with Astra.

No. It’s not at all by definition hidden reasoning.

Looping transformers uses additional calculations (repeating layers) to generate a token.

Reasoning (in this context) is test time generation of multiple tokens that allow a model to have a scratch pad to refine its thoughts, chain of thought reasoning in other words.

Doing the former in no way means that you have to hide the latter.

Raschka is right in this post, The Information article was wrong. The Astra system card does concede reasoning traces are sometimes smaller, but this could be for a lot of reasons, including simple efficiency. And it absolutely doesn’t mean they are going away or completely obscured.

The Last Week in AI podcast from Sept 8 seems to have gotten this wrong as well. Jeremie Harris rages that OpenAI implemented latent reasoning, ala the coconut paper, which could potentially actually obscure reasoning traces. But for the life of me, I do not know how he arrived at this conclusion and see no evidence that this has happened in Astra.

I don’t know that it’s that shocking, remember Go it’s not solved game, so the the limits of what’s really possible is not known in all cases.

For example, we don’t even know whether perfect White play can possibly overcome two correctly placed Black stones against perfect play.

If you’re wondering how they wrote to the wiki having only GET ability…

Basically it was a bug in the wiki code. They transferred the POST form parameters to GET URL parameters, and wiki internally doesn’t distinguish between the two.

There are important gaps in that hot take.

For example, it's not even close to Opus 5 on Terminal-bench 4.0, 19.1% vs. 51.8%.

Why do people say document database when they really just mean json database?

“It's no where near those numbers”

“get inflated in anecdotes”

Just because one study has the number 34% in it, does not prove the numbers I gave were wrong and does not mean they are anecdotal.

This is something that’s been studied from many angles and there’s more than one number that’s defensible based on the context and dataset.

For example, even the study you cited says: “In a study of 344 patients by Montejo-Gonzalez et al, 58% of patients reported sexual dysfunction when physicians directly inquired”, and “14% with spontaneous reporting”.

In another example, a 1,000 patient study found 57.7–72.7% for common SSRIs. https://pubmed.ncbi.nlm.nih.gov/11229449/

The “scientific numbers” you allude to are not one value. It’s a body of research not one definitive number.

Taken a whole I don’t think anyone is being that mislead when it’s suggested more than half of people may experience these side effects.

So, this is not supported by the data when people are asked.

Sexual effects alone hit about 50–70% people. Then you could face nausea, insomnia, profuse sweating when you’re still, emotional blunting, and weight gain.

I don’t want to discourage anyone, they can save lives. It is a net benefit for some people, but even then most people will experience side effects and wish they didn’t have to take them.

That’s only true regarding the one sentence about the singularity not being a point.

People like to reduce papers to a simple hot take, but the paper is more than that, offers viewpoints that are non-standard and speculation about new possibilities.

Why would you bring up fraud in the midst of science research in the US being burned to the ground?

Fraud is not the reason it is happening. Even the people who are making the cuts in this case have clarified the purported reason and it has nothing to do with fraud.

The DeepSeek team is so strong, very impressive.

Imagine if they had GPU resources of western labs.

They chose to compare against Open AI’s mid tier model Terra instead of Sol and still lost some benchmark against it.

They left Opus in and got beat in all but one benchmark.

Nothing wrong with trying to improve, but why the marketing games?

Instead of trying to say in the post you’re “closer” to frontier, first set a clear goal to beat the Chinese labs on price or performance and demonstrate it convincingly.

Then when your ready, come back and talk frontier without playing hide the model.

Nowhere in the paper do they mention the reasoning level or budget used for the experiments?

You’ve got to be kidding me. That one variable could make a huge difference in the results. I can’t understand why they would leave that out.

Another headline of “model runs on x”, which usually means “let’s list how much you give up to run on x”.

Dumbed down quantization?

No. Full intended inference weights preserved, so far so good.

Slow performance?

No again. Looks like you could get over 150 tokens/second.

Give up context window size?

Yes. Original model is trained for and served at 1M, this is 256k. A very practical tradeoff though. Codex is in this range, and quality does start to drop off toward the full size.

How do you figure that?

When they just loaded the weights alone, it was taking 156GB in vLLM. After warm-up and adding a KV cache pool, it took over 200GB.

And this implementation is already cutting down the 1M token context window you would normally get.

If speed is a metric for you, tokens required to solve a problem affects that metric.

All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.

Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.

It’s not outdated at all to use tokens to estimate performance, it’s directly related.

It’s exciting that a model scoring this high is dirt cheap.

It’s also so inefficient, when they release the full performance numbers it’s not going to be good.

One example, it takes about 3.6x more tokens to finish the same work as Gemini Flash 3.6.

False dichotomy right?

Are Chinese labs impressively innovating? Clearly.

However this doesn’t rule out possible gains due to distillation.

I don’t know the degree of the latter but both things could certainly be true.

The problem is what you’re skeptical about, the true cost, is probably the least important part.

Did it really cost $1 million instead of the 150k that’s been floating around?

If you don’t like the price now just give it some time.

The point is that these kind of heroics can actually work now and they’re only going to get cheaper.

I don’t mean to sound negative about what you wrote, it was an interesting read.

Cool worlds says no?

https://youtube.com/shorts/qreL7htXp98?is=G9zJVbpMT0yZsLQ-

The silence is deafening.

Google watches over the last few months a flat out assault on the Pareto curve from American and Chinese companies. Release after release pushing the boundaries of frontier intelligence and price/performance.

And the response from arguably the biggest AI research labs in the world by headcount is Flash 3.6.

What do you do when you are given essentially unlimited resources and still find yourself falling behind?

The world doesn’t need advisors like Moh that are bitter enough to gossip about a family funeral.

Not convinced just using top n sigma is going to beat a SOTA detector. The fingerprints Pangram uses should in principle be able to detect style above the token selection level.

Other papers have tried to beat it with temperature and it didn’t work, although I haven’t seen anyone try insane levels.

Give it a shot and let me know if you have any success.

I’m personally interested in it as part of the research to improve LLM writing. Detecting “AI voice” is part of understanding what’s wrong with it in the first place and how to improve it.

But yeah, in general I think you’re right, the actual utility is pretty niche.

I’m not sure what you’re saying I’m wrong about.

The comment I was responding to, about changing LLM output, referred to prompting, not temp/sampling tricks. I’m not aware of Pangram being beat by clever prompting. There’s some interesting work on creative writing using contrastive prompt techniques, but I haven’t seen it tried as evasion.

Even if you control temp and sampling, they’re not magic. If you raise the temperature too much writing can go to hell, so you may beat the detector but end up with junk. There are some ways to mitigate such a quality drop like raising temperature in conjunction with min-p, but still, I haven’t read any research that shows it getting good results at anything close to 10.

Now you want to get more clever and manipulate logprobs…well ok, you could come up with elaborate strategies designed to evade specific detection methods. But I don’t see that getting done as a weekend project while maintaining writing quality. And if it does happen there’s no guarantee the detector can’t train on its characteristics and start an arms race.

That’s a different point.

I’d want detectors to be as accurate as possible, false positives of 1 in 10000 seems like a good starting point. I believe their results have been independently tested.

And as a separate matter, any tool for evaluating students should be applied fairly, safely, and with adequate human review and due process.

You need good tools and good oversight.