_bobm

People are focused on the drama but the problem showing is the data. This is the elephant in the room and I am surprised that openai can be that stupid with it.

How can openai do this, what is being claimed, at the scale of their entire userbase? If they do this only for particular sessions then how do they sieve through sessions for the good stuff?

How are sessions stored, how are they processed, how much storage and how much compute is used in these pipelines, how economical is it, how fast are the requirements on the storage on the compute growing as userbase grows and generated data grows.

All these questions are far more pertinent than the navier-stokes, but i can only imagine all at openai doubling down on this "very important" mathematical milestone.

what is behind "process" it and "further selection" and "refinement" and how big are these "datasets"? These companies ship the encrypted session to you not because they want to.

I agree that they have pipelines for what you are describing but how effective they are at scale and at focusing is the question.

Hah, what is the infrastructure which takes user sessions (chats with API keys, directions, navier-stokes math/progress) and regurgitates this into pre-training, RL, fine-tuning data? Or better, in-context data?

People talk about the "compute" but what about the "storage"? Is storage exponentially greater, or soon to be, than the compute? Is the storage going to slow down growing to some constant rate, i.e. all people on earth using chatgpt, or no, on the contrary, it will keep growing?

If there were any shady business, I do not condone it, but technologically we are not there yet for said shady business to happen.

All this AGI/ alien nonesense trumpeting from these guys (OpenAI, Anthropic, nVidia) makes me think we are drifting even further from stated goal and money is running dry.

The interesting question is who is left standing after the party is over.

I think it is simpler, but bannin open weight is a collateral worth scoring nonetheless.

AI fails to deliver on the multi trilion usd promises:

Gov/venture says: oh, wow, what will we do with all these datacenters, we need money back!

OpenAI/ Anthropic: let's monitor the citizenry. They may be plotting nefarious schemes using AI models.

Gov: great idea. Whew.

Investors: whew!

Tax payers: paying to be in prison.

So not so much as defense against foreign actors, but failure of the self-tooted AGI goal + gov being gov.

I think this is the wrong approach because everything is external to the model. You end up creating an ad-hoc externalized model scaffolded out of coarser systems, RAGs, files, and so on.

This leads to, if useful at all, to this process of ad-hoc recall inference which has to happen in time. By itself this is not a problem.

The problem is that the model has to be constantly injected in context with the newest version of the "memory state" at each turn or relevant turn.

The newest coherent memory state is also a problem. More or less 50 years of not failures, but not success either. This may be even deeper problem than the externalization problem.

I think we are just in the very beginning and we are slapping database stuff to the transformer hoping it will work, but these deep neural-net architectures categorically show that they are not databases.

There will be a synergistic middle ground, but its shape is still not clear.

In your closing thoughts, you don't mention what is this staying firmly in control? how do you stay firmly in control while reaping the benefit of the model while being more productive?

because if you start a curriculum over what the model wrote then, one can argue, you are better off writing it yourself in the first place.

What is the purpose of the "zellij" in the top left corner and why isn't there an option to remove it?

are people getting the `<!-- -->` sentinel'd reasoning summaries?

I find it funny that some comments are arguing why "the innocent users have to fall victims, becoming/ being collateral damage" to this american governmental whim and thus being deprived of access to these models.

But hold on, collateral to what? Is this "our own personal jesus" access that we cannot live without or what?

People don't and cannot learn how to code or what? We don't know how to think?

I am calling their bluff. Fill your own gddam datacenters with "meaning".

Panta rei.

Yes, but it isn't.

Yes, local models have already all that is needed, they have all the prerequisites.

But what they do not have is the correct shape, the correct approach. This is missing and it shows on multiple scales: it shows in the COT, it shows in the output itself, it shows in the infra to serve the models, it shows in the model orchestration.

This is what anthropic said one year ago:

> Finally, we've introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. Users requiring raw chains of thought for advanced prompt engineering can contact sales about our new Developer Mode to retain full access.

idk what is "minifying outputs" in the context of what we are talking about. Opencode is opensource, you can find out what it is doing.

Last time I checked, OpenAI even send (in the response) the summary of the thinking part alreafy in markdown, so opencode has to remove the formatting to format it to their liking.

> Many models now no longer return the entire chain-of-thought (to avoid distillation attacks).

This is what they say: to avoid distillation attacks. And to some large extent this is true. I am saying there is a side- effect and this side- effect (depending on how tin-foilly you want to go) may be either a nice thing to have or it may be the "main reason" for all of this.

The side effect is splicing the inference, brokering requests, and what not, which brings huge benefits at scale.

This was my original point: openweights model to a sota model may be apples to oranges. So when will a local model catchup with its single cot run which is not even shaped properly: well never.

It is apples to oranges.

> Can you give an example?

Sure, connect opencode to an openai/chatgpt endpoint and use it. You will notice multiple "thinking" parts per "turn".

I put all of these in quotation because... they are part of the orchestration game. For example, it is not known if the thinking parts of a particular turn are chain of thought thinking summaries or just plain response which is masquaraded and thus orchestrated into appearing as thinking.

Further notice the cadence, word choice and sentence formation. Notice sentence construction. Notice "thinking part" construction and sequencing.

There is pretty heavy orchestration.

> I don't understand, why does it make you think this is the case?

Because not all tokens are equal. And if you waste expensive tokens on mundane tasks you will go out of business. This is the reason.

As I said, if you observe the output from these api endpoints you will notice it.

But, guys, when you say Claude/ GPT models, do you stop to think what are these "models"?

One day I thought about how can GPT send thinking parts one after another with a markdown header summary of the thinking block itself. Just think about it.

As a matter of fact, think about these operations, api endpoints, observe their output.

These so called SOTA models are not what meets the eye, and are not at all comparable in the infra department to local models. There is crazy orchestration going on due to the scale of these operations. But also these hard constraints lead to innovation. Innovation nobody speaks about.

I wouldn't say we cannot catchup, but serving our local models through llama, vllm is just the A, B, C of it all. In reality I think what is needed is a replication of said orchestration which I hinted at above.

The SOTA models are a deep orchestration of multiple models operating together it isn't a single model. As such no single model ever will catchup to them until it replicates through training first and then maybe through model architecture this orchestration.

Finally, I would wager that the SOTA "models", as one of these models in this orchestration setup, as served for general consumption, are not so much more capable than qwen 3.6.

I am sure that if you change your perspective you will start noticing the scale of the "magic".

Very confident. But will it stick? And if it doesn't -- what then? Back to scheming?

What are the news recently?

amen

hah, good on them.

nice catch.

How do you see Zulip comparing to anytype, https://anytype.io/ ?

This is how I view it as well.

And... and...

This results in a _very_ deep implication, which big companies may not be eager to let you see:

they are context processors

Take it for what it is.

But you are not having a free meal lunch are you? You _are paying_ for your meal.

Worse: you are the meal as well.

Do you see this?

I have been a bit out of the loop. what is relevant these days for writing ebpf code? what about ebpf code in python?

Despite trying and reading it carefully I didn't understand it. I wonder if it is me or the article, or both.