16,549 karma · joined April 2, 2010
This is not unique to ACs. Anything on the outside of a building requires your co-owners to agree.
Author here: maybe I wasn't too clear about this, but I have seen people direct their anger about about what AI does to their jobs at people who themselves are not in a situation to do anything about it (eg: line managers or leaderships in businesses who are trying to roll out AI based coding tools).
The startup time is a known issue and scales badly with extensions. We’re aware of it but fixing it is tricky.
I’m not a fan of config directories being in different locations on different platforms because it’s now one extra thing everyone needs to handle.
(Disclaimer: I work on Pi but I dislike XDG in all settings)
Which part are you referring to here? web_search coming from the provider hiding the tokens is just a convenience, there is no reason it can't be done with revealing information. Their compaction on the server might be amazing, but at least in principle it can be replicated on the client side as well and it could have been implemented in a way that reveals the new initial context.
Now unless you fully want to commit yourself to one provider, you will on the application layer need to deal with this anyways in one form or another. We want to at least try to address most of this on the harness layer, regardless of how tricky that is.
But not on the level that providers need to agree to a common format. Within Pi we are able to model portable sessions between providers even if they have provider specific APIs. For instance most models do not agree on how to do deferred tool loading. But they now have some support for that even if it diverges between models and APIs. So we can model deferred tool loading specifically for different providers without having to align on a common set of APIs or similar.
What cannot be made portable, is when information is sealed in the transcript or the information is entirely locked away on the provider's servers (store = true).
I start out with what I want to talk about, and then make an initial draft. In this case I also had Sol research all current APIs so I don't have blind spots. I tend to also talk with an AI to see if the structure structure of the post makes sense. I then wrote all sections by hand but used Sol to fix up typos, grammar and punctuation. I also had Sol apply notes and patches that were provided via Discord.
If I find the earlier drafts I can check if it did some more outrageous edits, but I kinda doubt it did. I do know that my workflow of having an LLM to fixes up later is increasingly breaking on SOTA models and I have complained about this before. Normally I now carefully apply fixes that it proposes, but in this case I didn't due to time constraints.
I am curious though why it claims that this is AI generated.
That's sort of the thing: very little (beyond switching models and providers). Obviously what you lose is a convenient way to run analytics over your own sessions if more and more information is not revealed to you, but before encrypted prompts that wasn't that much of an issue anyways.
This post only talks about the former.
Your idea though is the core of the responses API with store where everything is stored on their side and then you append to it.
For the LLM there is no difference btw. In either way you need to locate the cache and use it.
The real thing that will take this alive though is people pushing back a bit against some of these newfangled APIs. Now that there is real competition from the new generation of Chinese models which have much fewer of those restrictions, I think there might be a moment.
Encrypted reasoning content is one of the things we keep as blobs in the transcript, even though we can only send it back to the original provider that created them, as otherwise we cannot keep caches live.
Is OpenCode pushing the LLM into doing these refactors via LSP? I found that making these refactors with ast-grep is quite efficient.
> It can also use LSP to build context without having to grep around a ton. You don't have to grep and hten pull lines and make guesses for a function. You just ask the LSP for it.
I guess I was not particularly successful with having the agent actually do that. I think what fff is attempting to do (with fuzzy expansion of slightly inaccurate greps) results in better results to me, but even that is debatable.
To be fair, we do not have evals for this today, but we're generally looking into token efficiency and how likely models are at using tools the right way, and we did not find a ton of evidence of LSP helping, even for finding data.
But I would love to see some sessions from people where they have success with LSP data for either refactoring or looking up, because this would be super useful to better understand out blind spots.
I would love to learn how LSP helps you. I did various basic evals with LSP and I did not find it to improve the agent performance at all. Maybe on newer models this gets better because they are now RLing with LSP in the context, but at least in the past having a commit hook that runs lints and typecheck at the end, is more token efficient than having every edit be injected extra LSP results. You are almost guaranteed to be wasting tokens since partial edits are unlikely going to result in type checks passing.
(Disclaimer: I work on Pi)
Pi is doing that.
That is not the case. There are some subtle differences between subscription and regular inference API, but not to the degree that behaviors change entirely. In Pi we're doing tests against both API and subscription API regularly to see how they behave.