sdesol

This is a hard problem, but one worth solving, I think, since it means less tokens and better AI reasoning. I believe LLMs are good enough that, if given the right context, it can very much solve almost all tasks.

If this works, it means we can probably get by with smaller models (since it doesn't need to know everything). LLMs are pattern matchers, and if you can provide them with the right shape (context), they should produce the expected output.

For my solution to work, you need business buy-in, which I don't think will be a problem. Enterprise wants to know how tokens are being spent, so I can see them wanting structured analysis during code reviews.

What may also not be obvious is that the information is ultimately designed to live with your code. Lessons and notes are designed to be mapped to files, so if you want to know why a piece of code is implemented in a certain way, you can have the LLM filter by files to help find the needle in the haystack.

It is a hard problem, but the only missing piece is discipline, which I believe business leaders will not have an issue with enforcing since we are ultimately talking about eliminating/significantly reducing the bus factor in our code.

If you look at https://github.com/gitsense/smart-ripgrep, you can get a better sense of how context can be injected when it is needed.

> So I want to say there's still a lot of value in context engineering though it seems to diminish with each model release

I can't see how it would diminish unless you are literally working on public domain stuff. Unless stuffing context becomes cost effective and will not affect AI reasoning (this will be much harder), I don't see why context engineering is here to stay until we have close to AGI.

The user flow I am trying to get adopted for sessions is to turn them into notes and lessons when you have finished and it should be part of the code review process.

By propery categorizing lessons and notes, it should make it easy to scrub and keep up to date.

I also suggest mapping lessons and notes to files when possible to make discovery and cleanup easier.

> You don't need another layer

I do think we need another layer, but it should be a routing layer. I am finalizing my pi-brains extension for Pi (https://github.com/earendil-works/pi) which does this:

https://github.com/gitsense/pi-brains

Right now "humans" need to define the routing rules for how to access information, but I will support what I call "knowledge agents" that can monitor conversations to inject context when needed.

> For AI agents, it means well-designed CLI tools that help the agent orient itself in a task and pull exactly the 'context-for-the-job' it needs right then.

This is exactly what I am trying to solve and I have what I call smart repositories that demonstrates this at

https://github.com/gitsense/smart-ripgrep

https://github.com/gitsense/smart-codex

The issue I am finding is, getting the agent to pull what it needs, even when the data is there is still challenging since LLMs are trained on blind discovery where the pattern is:

grep -> read -> grep -> read ...

What is working for me now is thanks to Pi (pi.dev). I am working on a pi-brains extension that makes it dead simple to control the lifecycle for an agent so if I detect that it uses `rg` without `gsc rg`, I can block the agent and inject a steering message that says always search with context.

I can also see if they try to "read" without first looking at the files metadata and so forth.

I'm finalizing things right now, but I think pi with my brains extension should allow domain experts to better guide agents so they can find what they need, when they need it.

I talked about this before but China would be in much better position if LLMs turn into a commodidty. Where they can dominate is in hardware, as fast and cheap inference is probably going to be the moat.

> It’s demoralized a lot of people

Isn't this a good thing? Doesn't it mean your company doesn't want employees to treat LLMs like a slot machines?

Was there at least performance gains to be measured?

The big thing is, the western world has moved so much of the manufacturing to China and think a lot of people will not forgive Samsung and others, so I can see China owning a good portion of the supply chain.

> Microsoft adding Deepseek support

I believe this hasn't been confirmed yet but I think it speaks to a bigger problem for the AI companies which is, if you give capable developers a good reasoning LLM, they can make it work like it was a really expensive model.

I believe we are 100% at the stage of good enough for the vast majority of tech companines. Fable and others will be more valuable for non-traditional tech companies.

I read somewhere that the chinese AI companies are sharing knowledge and it would not surprise me if the government is applying pressure by saying work together or else. If they work together, they can truly commoditize LLMs and with China ramping up hardware support for AI, I see the future being inference speed and hardware being the moat.

I think the issues is, it is going against a very well established pattern. I have a tool that wraps ripgrep so that search results always includes context and from time to time, the agent will use ripgrep by itself and when I ask why, it would go "yeah I should have done that"

There are work arounds though and I am creating what I call knowledge triggers for Pi that are similar to claude's "PreToolUse" so having the agent use oak all the time is not an issue in my opinion.

The challenge for oak is why? Considering how I actually want to slow agents down so I can ensure it is doing the right thing and because the massive bottle kneck is the LLM themselves, speed when measured in milliseconds or even seconds will not concern many.

I thought oak was more of, we know how to prompt inject context based on code that is stored in oak for example, but faster operations can help, but the use case is limited. The missing piece for better/correct code is context at the right time.

> Great! So next time the human will prompt the agent to watch out for and avoid this bug.

I actually created a system for something like this. The basic idea is, once you have identified what the issue was and fixed it, you can create lessons that lives inside the repository. Lessons are designed to be mapped to one or more files so if the LLM changes the files again, they can see what the issue was.

The main challenge is being able to summarize and create proper tags so the AI after any code change can easily find the lesson.

What behavioural things are you noticing that Claude does not pickup from the CLAUDE.md. I am working on a `pi-brains` extension for the Pi agent. It is designed to inject rules into write and edits tool calls for matching files.

I am curious if the behaviour you want is outside of writing and editing files.

> Filling context with what you think the model needs adds nothing and possibly just inflates context which is harmful.

The solution that I've developed is, let the agent figure things out efficiently, without inflating the context. I have what I call a smart repo that better explains this at

https://github.com/gitsense/smart-ripgrep

The basic idea is, when the agent does a ripgrep it gets back files + matching lines + context.

The AI agreed

> There isn’t a magic prompt I could have written before knowing the solution. A retrospective prompt can reproduce the 200 lines only because it embeds the invariants you discovered. The useful AI task would have been systematically finding counterexamples and iterating toward those invariants, not generating the final file in one shot.

For me the value with AI isn't the one shot, it is the fact that it can help you iterate faster. My biggest take awway so far with AI is it can help you fail fast. The faster you can fail, the faster you can iterate.

The code in http://github.com/gitsense/gsc-cli contains over 300 go files and I would say 90% were one shots. I can even prove that they are AI generated since I attribute the AI in each file, as shown in the example below:

https://github.com/gitsense/gsc-cli/blob/main/internal/cli/r...

Since I don't know go, I can't say which is the best. I am also sure they would not live up to a go experts expectations. I just know that since I don't know go, I would not have been able to create the tool. I can also prove that you can easily maintain AI generated code since I have been expanding and evolving it for over 6 months with pretty much no issues.

> playing a slot machine hoping

I still believe the slot machine analogy holds to some extent, but I can honestly say my winning percentage is at least 90% for one shot generated code now.

I think if you know it's limitations (inlcluding your own), I don't think about hoping anymore.

I should note that when I say AI, I mean the collective models from all the major providers. The most important lesson is, you need to ask around.

> There’s definitely a skill to using AI but it just doesn’t generalize very well.

This I agree with. The only way working with AI can really be benefical outside of dealing with AI is, we are visited by extremely intelligent beings that will fuck up in the weirdest ways.

> and no ways to measure whether AI is working better.

What I do with my product is I explicity tell you to ask your agent. I have real world examples and real world repositories that you can try with:

https://gitsense.com

https://github.com/gitsense/smart-ripgrep

https://github.com/gitsense/smart-codex

Token saving on average is not what I am mostly interested in though. I am more interested in knowing that the AI doesn't load unnecessary files in context, which can affect reasoning.

You can just ask the agent after a task how many files do you think was not read by knowing the files purpose first?

I am working on solving the AI Code Provenance problem and I believe my repos may be the first that provides AI code provenance. See the following example:

https://github.com/gitsense/gsc-cli/blob/main/internal/cli/r...

Notice how the code block header attributes the model. The UUID can be traced to the conversation so everybody can tell exactly how the code came about. For this to work though, you need to use my chat app as it ensures you can't tamper with things if you are truly serious about AI code provenance.

I also have a lot more human-focused method which is part of my CLI tool.

https://github.com/gitsense/gsc-cli

I am currently looking at making pi (https://github.com/earendil-works/pi) support AI code provenance, but for now if you want a more structured way to capture what you have done in an agent session that can be used in code reviews and be carried forward as knowledge that lives inside your repository, I have

gsc lessons

The basic idea is, after you have finished chatting/working with the agent, you would work with it to identify lessons worth carrying forward. You can store your session if you want, but really, the lessons should be something that can help you review code better and to prevent future mistakes.

I have a real working example at

https://github.com/gitsense/smart-ripgrep

This is a fork of the BurntSushi/ripgrep repository. It shows how you can use lessons to learn from past design decisions.

> GLM writing

This is honestly what I care bout the most now, which is how well they can write. I think we have reached a point now, if you know how to program, you can provide enough information for the models to pretty much do what you need.

What they still struggle immensely with is the writing which has too many nuances but they are truly getting better.

> So it's a scary time to work in tech even if I think the trend will ultimately reverse.

I honestly can't see things going back to what it was 5 years ago. We will probably not have the future that Anthropic hopes for, but I think every developer will be required to chat with AI as part of the planning process to reduce a companies "bus factor" risk.

> A well thought out pros and cons story wins over binary yes/no answers at pro and anti ai companies alike.

The issue with this is, you need to know how to really program to be able to articulate the pros and cons, which a new grad would mostly likely not have.

For example, if you want to include how AI can onboard quickly, you really need to understand the pain points like, I tried asking people but really, everybody is busy. Or I've found coding agents help me speed up making code changes, but it some situations, they can help accelerate making mistakes.

I think the issue that a lot new grad are faced with is, you don't know, what you don't know.

> playing trendy tech lottery.

I don't know about that, and I am 100% biased so take what I say with a grain of salt. My position is very much this: you may not trust coding agents to make code changes, but if you're not willing to treat them as a research aid or have them work for you, you're pretty much saying they can't help you work more efficiently.

I'm working on a Show HN post that includes:

https://github.com/gitsense/smart-ripgrep

It's a fork of BurntSushi/ripgrep. What I hope to show with it is that you don't have to use coding agents to code. They can be used to surface knowledge that's buried in documents, issue comments, PR discussions, and other places.

Believing coding agents are trendy would be like saying search was trendy in 1998. They're not going to change the world the way Anthropic wants us to believe, but they will shape how humans develop software. And I think for the better, since AI is capable of processing information at scale to help you move forward.

> replacing deterministic systems in their support flows

The issue is, they don't want to provide "better" support but "cheaper" support. Imagine a trained agent that understands the big picture. Now imagine a company investing in humans to use AI to retrieve knowledge that the human can easily identify as being relevant or not, and using that knowledge to better aid the customer.

Right now AI is being sold as a "we don't need support personells" instead of "how can we provide better service." For a lot of products, better service will probably not matter as "cheaper" products will win most of the time.

Most people don't want to pay for better. They want to pay the same for something better, which is what companies are not investing their time in figuring out how to use AI properly for I think.

> first for the OSS project and again for a commercial product.

Is there a way to reach out to you as I would like to hear what you have to say about what I am working on. You can update your HN profile to include contact information or you can reach out to me using my HN profile.

I'm basically working on a portable intelligence layer for AI that I will be open sourcing and the commerical product will be to make the intelligence layer even smarter. I can share the Show HN post that I am working on that better explains the value proposition and would love to learn any lessons you have gained while trying to sell AI tools commerically.

Edit: In case somebody calls me out it. I didn't want to use the `tensorzero` email domain incase the domain was going to become defunct soon.

> What is that worth? :-)

This is one of those double edge sword situations. It is on the front page and it stays because it will trigger a lot of people and he has to spend a lot of effort explaining himself. What is that worth?

His explanations would most likely be buried deep so the impression that others get might be worsened. What is that worth?

In my opinion, this is one of those find a harder problem and you would still have the same content...but it might not draw as much feedback and stay on the front page longer.

I created this and I would say glm-4.7 accounts for 80% of the code in https://github.com/gitsense/gsc-cli

If you look at a file like:

https://github.com/gitsense/gsc-cli/blob/main/internal/cli/r...

you can see that I attribute the models used. What I found was 4.7 was not very good at `go` code which was why you started to see `Gemini 3 Flash` in the attributions.

4.7 is what Cerebras provide and for me, speed in iterations is a lot more important. Having played around with MiMo v2.5.0-Pro, I am 100% sure it could have done what Gemini 3 Flash did.

There were a few points where I was stuck and needed Sonnet to explain things to me, but I think the dirty secret that Anthropic and OpenAI won't tell you is, if you know how to code, the models are honestly good enough.

Based on my experience with MiMo and what others are saying about GLM 5.1, we are now in a hardware race. The Chinese Models are 100% drop in replacement for Claude if you know how to program but want to AI to help amplify what you know. What I will consider now is what provider can provide the fastest inference.

MiMo-v2.5.0-Pro-Ultraspeed is really good at generating good results quickly and burning your money as fast.

Fixed, thanks!

This sounds like it is more aligned with what I have created which is "We need to capture your conversations with AI". If you look at

https://github.com/gitsense/gsc-cli/blob/main/internal/cli/r...

you can see that every file has a code block header with a UUID and the AI that was attributed to it. With the UUID, I can tell exactly how the code came about.

What they are working on will be more useful for AI code provenance. It is only a matter of time before you are expected to show your chats with AI as part of the code review and for performance reviews.

So I don't see human collaboration being the main use case. I see tracking, studying and improving the Human-AI relationship...and seeing if somebody should be promoted or not.

An interesting take I've heard is, we will have a token/impact stat where if you spent a shitload of tokens to produce the same impact as somebody else who spent a lot less, you will be the prime candidate for layoffs and/or less pay. This is why I think AI code provenance will become a serious thing in the future.

> at a Sonnet 4.6-level model

MiMo v2.5.0-Pro is honestly the first Chinese model that I've tried where I really though why should I use Claude Sonnet when I can get the same results for a fraction of the cost. There was always something off about Chinese models that made it apparent that it couldn't fully compete with GPT, Claude, Gemini, etc. but this was the first model where I was like, this feels like Sonnet.

I can't prove it, but I think they trained heavily on Claude output. From my perspective I don't care since Anthropic trained on my data.

Using them also works well for North Americans as our peak hours is not theirs.

If I had one complaint, the v2.5.0-Pro model thinks too much.