sdesol

I don't use Cursor so I can't say, but based on what I've read, they optimize for smaller context to reduce cost and probably for performance. The issue is, I think this is severely flawed as LLMs are insanely context sensitive and forgetting to include a reference file can lead to undesirable code.

I am obviously biased, but I still think to get the best results, the context needs to be human curated to ensure everything the LLM needs will be present. LLMs are probabilistic, so the more relevant context, the greater the chances the final output is the most desired.

> I really desperately need LLMs to maintain extremely effective context

I actually built this. I'm still not ready to say "use the tool yet" but you can learn more about it at https://github.com/gitsense/chat.

The demo link is not up yet as I need to finalize an admin tool but you should be able to follow the npm instructions to play around with.

The basic idea is, you should be able to load your entire repo or repos and use the context builder to help you refine it. Or you can can create custom analyzers that you can do 'AI Assisted' searches with like execute `!ask find all frontend code that does [this]` and the because the analyzer knows how to extract the correct metadata to support that query, you'll be able to easily build the context using it.

Joking aside, I do believe we are moving into a era where we have LLMs write for each other and humans have a dedicated TL;DR. This includes code with a lot of comments or design styles that might seem obvious or stupid but can help another LLM.

> I once again ask Claude Code to fix the bug, this time including the little bit I learned in #3. Claude Code fixes the problem in one round.

Context is king, which makes sense since LLM output is based on probability. The more context you can provide it, the more aligned the output will be. It's not like it magically learned something new. Depending on the problem, you may have to explain exactly what you want. If the problem is well understood, a sentence would most likely be suffice.

I think it is more of, LLMs should be treated as a utility service. Unless Google and others can clearly show the training data involved, the price that providers can charge for LLMs should be capped. I have no issue with contributing my conversations and my open source code, and I should expect in return a fair price.

This reads like it could result in "the blind, leading the blind". Unless the Supervisor AI agents are deterministic, it can still be a crapshoot. Given the resources that SourceGraph has, I'm still surprised they missed the most obvious thing, which is "context is king" and we need tooling that can make adding context to LLMs dead simple. Basically, we should be optimizing for the humans in the loop.

Agents have their place for trivial and non-critical fixes/features, but the reality is, unless the agents can act in a deterministic manner across LLMs, you really are coding with a loaded gun. The worst is, agents can really dull your senses over time.

I do believe in a future where we can trust agents 99% of the time, but the reality is, we are not training on the thought process, for this to become a reality. That is, we are not focused on the conversation to code training data. I would say 98% of my code is AI generated, and it is certainly not vibe coding. I don't have a term for it, but I am literally dictating to the LLM what I want done and have it fill in the pieces. Sometimes it misses the mark, sometimes it aligns and sometimes it introduces whole new ideas that I have never thought of, which will lead to a better solution. The instructions that I provide is based on my domain knowledge and I think people are missing the mark when they talk about vibe coding, in a professional context.

Full Disclosure: I'm working on improving the "conversation to code" process, so my opinions are obviously biased, but I strongly believe we need to first focus on better capturing our thought process.

> Have you tried building agents?

I think the issue right now is so many people want to believe in the moonshot and are investing heavily in it, when the reality is we should be focusing on the home runs. LLMs are a game changer, but there is still A LOT of tooling that can be created to make it easier to integrate humans in the loop.

Nice write up. I don't necessary think it is "if you adopt it, you will flourish" as much as, if you have this type of "personality you will easily become 10x, if you have this type of personality you will become 2x and if you have this type, you will become .5x".

I'm obviously biased, but I believe developers with a technical entrepreneur mindset, will see the most benefit. This paradigm shift requires the ability to properly articulate your thoughts and be able to create problem statements for every action. And honestly, not everybody can do this.

Obviously, a lot depends on the problems being solved and how well trained the LLM is in that person's domain. I had Claude and a bunch of other models write my GitSense Chat Bridge code which makes it possible to bring Git's history into my chat app and it is slow as hell. It works most of the time, but it was obvious that the design pattern was based on simple CRUD apps. And this is where LLMs will literally slow you down and I know this because I already solved this problem. The LLM generated chat bridge code will be free and open sourced but I will charge for my optimized indexing engine.

> I've had it do some huge things that run for an hour or more,

Can you clarify this? If I am reading this right, you let the llm think/generate output for an hour? This seems bonkers to me.

> This is a huge opportunity, maybe the next big breakthrough in AI when someone figures out how to solve it

I am not saying I solved it, but I believe we are going to experience a paradigm shift in how we program and teach and for some, they are really going to hate it. With AI, we can now easily capture the thought process for how we solve problems, but there is a catch. For this to work, senior developers will need to come to terms that their value is not in writing code, but solving problems.

I would say 98% of my code is now AI generated and I have 0% fear that it will make me dumber. I will 100% become less proficient in writing code, but my problem solving skills will not go away and will only get sharper. In the example below, 100% of the code/documentation was AI generated, but I still needed to guide Gemini 2.5 Pro

https://app.gitsense.com/?chat=c35f87c5-5b61-4cab-873b-a3988...

After reviewing the code, it was clear what the problem was and since I didn't want to waste token and time, I literally suggested the implementation and told it to not generate any code, but asks it to explain the problem and the solution, as shown below.

> The bug is still there. Why does it not use states to to track the start @@ and end @@? If you encounter @@ , you can do an if else on the line by asking if the line ends with @@. If so, you can change the state to expect replacement start delimiter. If it does not end with @@ you can set the state to expect line to end with @@ and not start with @@. Do you understand? Do not generate any code yet.

How I see things evolving over time is, senior developers will start to code less and less and the role for junior developers will not only be to code but to review conversations. As we add new features and fix bugs, we will start to link to conversations that Junior developers can learn from. The Dooms day scenario is obviously, with enough conversations, we may reach the point where AI can solve most problems one shot.

Full Disclosure: This is my tool

> Why not just put the conversation / prompt at the end of the commit/PR message?

This might be something I would do. My only concern is, the conversations can be quite long and mean very long. My "Chat Flow" right now is to discuss what needs to be done. Produce the code, which can span multiple chats and then have the LLM summarize things.

What I think might make sense in the future is to include detailed chat summaries in commit messages and PRs. Given that we get a lot of text diarrhea from LLMs, I think putting them in a LLM as is, may do more harm than good.

When dealing with prompts, there is no such thing as reproducible builds, since we are dealing with a non-deterministic system. The purpose for connecting prompts and conversation to code is to aid in debugging and developing new features. I also believe by connecting code to conversations, we will produce better LLMs in the future.

I would say AI has generated about 98% of my code for my chat app in the last 3 months and it was definitely not vibe coding. Every function and feature was discussed in detail and some conversations took over a week.

My reasoning for building my chat app wasn't to support vibe coding, but rather to 10x senior developers. Once you know how things work, the biggest bottleneck to a senior developers productivity is typing and documentation. The speed at which LLMs can produce code and documentation cannot be matched by humans.

The only downside is, LLM don't necessary produce pretty or more readable code. The more readable code is something I would like to tackle in the future, as I believe post-processing tools can make LLM code much more readable.

> All in all, a prompt history would be really edicational if anyone thinks about doing something similar.

As we start to develop AI first programs, I believe we will need to start connecting LLM conversations to code, not only for educational purpose but for maintenance as well. I'm currently experimenting with what I call Block-UUIDs, and they are designed to make it easy to trace LLM generated code. You can see what I mean in the link below, which contains a simple hello world example.

https://app.gitsense.com/?chat=7d09a63f-d684-4e2c-97b2-71aa1...

Something worth noting is, you can't expect the LLM to properly generate a UUID. If you ask the LLM, it'll says it can, but I don't trust it to do it correctly all the time. Since I can't trust the LLM, I instruct the LLM to use a template string, which I can replace on the server side. I've also found LLMs will not always follow instructions and will generate UUID and how I handle this is, when the LLM stops streaming, I will validate and fix if needed, any invalid UUIDs.

How I see things playing out in the future is, we will alway link to LLM conversations which will give use the Block-UUIDs generated and by looking at the code, we can see what Block-UUID was used and how it came about.

Full Disclosure: This is my tool

I think it being Chinese is part of the reason as some of the examples in the early days were Chinese only, which could deter some people. It is certainly more complex (for a good reason in my opinion), I can see why it is not well known since I think the vast majority just wants to create simple charts. However, with echarts, it really can meet Enterprise needs.

Full Disclosure: I built an indexing engine for Git and GitHub that can process repos at scale and my words should be taken with scepticism.

I think using MCP is an interesting idea, but the heavy lifting that can provide insights, is not with MCP. For fetch and search to work effectively, the MCP will need quality context to know what to consider. I'm biased, but I really looked into chunking documents, but given how the LLM landscape is evolving, I don't think chunking makes a lot sense any more (for code at least).

I've committed to generating short and long overviews for directories and files. Short overviews are two to three sentences. And long overviews are two to three paragraphs. Given how effectively newer LLMs can process 100,000 tokens or less, you can feed it a short overview for all files/directories to determine what files to sub query with. That is, what long overviews to load into context for the sub query.

I also believe most projects in the future will start to produce READMEs for LLMs that are verbose and not easy to grok for humans, but is rich in detail for LLMs. You may not want the LLM to generate the code for you, but the LLM can certainly help us navigate complex/unfamiliar code in a semantic manner, which can be game changer for onboarding.

Not really. I had to do the following:

- Identify the files that should be put into context since tokens cost money and I wanted to use a model that was capable like Sonnet, which is expensive.

- There were 35 messages (minus 2 based on how my system works) so I wrote and read quite a bit. I was actually curious to know how it worked since I have domain knowledge in this area.

- Once I knew I had enough context in the messages, I switched to Gemini since it was MUCH cheaper and it could use the output from Sonnet to guide it. I was also confident the output was accurate since I know what would be required to put a Git repo into context and it isn't easy if cost, time and accuracy is important.

Once I went through all of that I figured posting the parent questions would be a good way to summarize the tool, since it was very specific.

So I guess if that is the next LMGTFY, then what I did was surely more expensive and timeconsuming.

Full Disclosure: I'm building a LLM chat app for software developers and domain experts

> I struggle to find much utility in terms of actually writing code.

I personally feel you need to give up some control and just let the LLM do its thing if you want to use it to help you build. It honestly does a lot of things in a more verbose way and I've come to the conclusion that it is an LLM writing code for another LLM. As long as I can debug it, I'm okay with the code, as I can develop at a pace that is truly unreal.

I finished my "Recent" contexts feature in a half a day, today. Without the LLM, this would have taken me a week I think. I would say 98% of my code in the past few months has been AI generated. You can see a real life work flow here:

https://app.gitsense.com/?chat=eece40e2-6064-46d2-9bf1-d868c...

I truly believe if you provide a LLM with the right context, it can meet your functional specs 90% of the time. Note the emphasis on functional and not necessary style. And if *YOU* architecture your code properly, it should be 100% maintainable.

I do want to make it clear that what I am doing right now is not novel, but I believe most problems are not. If the problem is not well understood, it can be a challenge like my my chat bridge feature. This feature allows you import Git repos for chatting but I will probably need to rewrite 50% of the LLM code since the solution it built is not scalable.

Wouldn't Occam's razor at this point be OpenAI doesn't have a moat and Microsoft knows this?

> I'm sorry, we're assholes when it comes to nationalism sometimes

Please speak for yourself, as a fellow Canadian. This is not about nationalism, but rather about not giving into a bully. If we start begging for mercy now, things will just get worst. And no, most Canadians do not fear the US invading, as many believe a civil war in the US will happen before that. We also have the luxury of having goodwill among many around the world, so selling our goods will not be an issue but we will obviously have to lower our margins to overcome geographical barriers.

I just want to make it clear for any readers here that GypskyKing716 is not the voice for all Canadians, and his voice is his own, like mine is.

It just feels natural to me. The person knows the language but they are not trying to sound smart by using words that might have more impact "based on the words dictionary definition"

GPT 4.5 does feel like it is a step forward in producing natural language, and if they use it to provide reinforcement learning, this might have significant impact in the future smaller models.

I didn't get the impression that the city was focused on creating housing for Toyota employees; rather, it seemed to be about understanding what will work and what won't. Toyota, like many Asian companies, has many different verticals, and I think whoever better understands the needs of the future will be able to stay relevant.

Having Toyota employees as residents just makes sense, since it will be much easier to set expectations.

I understand the reasoning and I hope there is legislation in the future that basically goes "If you can't produce the data, you can't charge more than this for it". Basically, LLM producers will have to treat their product as a commodity product that can only be priced based on the compute resources plus some overhead.

> LLMs will compete with outsourced developers

I guess the question is whether the person you are replying to is potentially living in a country where most of the work is currently being outsourced, as this could significantly impact their career path.

It is interesting that you bring up outsourced work, as I strongly believe that a lot of the bad code generated by AI is the result of not feeding it enough information.

When you outsource work, you are usually forced to document things more thoroughly to work around language and domain knowledge. Basically, clarity is a requirement and maybe outsource companies will experience the most impact.

> It’s AI corporate-speak. It is not meaningful

It is if you want to better understand how models are trained and in what directions they are leaning.

> post seems to go into all different directions, which seems strange.

I asked AI to give feedback on the negative points in the article, with a focus on the human impact. I asked Gemini Flash 2, GPT-4o, Llama 3.3, and DeepSeek v3 and they had diverging thoughts in some areas, so your take seems valid.

Here is the overall summary

- *Consensus*: All LLMs agree on core risks (deskilling, code quality, workflow disruption).

- *Divergence*: Job displacement and bias concerns split opinions, with some LLMs deeming them speculative.

- *Scope*: Gemini and DeepSeek v3 extend discussion to broader AI ethics, while Groq/Llama 3.3 and GPT-4o prioritize strict article alignment.

I was actually surprised by the Gemini 2 response

---

While the article focuses on the benefits of GitHub Copilot and its new features, it's important to critically evaluate the potential downsides and unintended consequences. My initial response aimed to provide a balanced perspective by highlighting these concerns, which are relevant to the broader discussion about AI in software development.

Here is the full analysis https://beta.gitsense.com/?chat=ec90dd73-0873-43ab-9da0-c613...

> The idea of using a smaller version of the same (or a similar) model as a check is interesting.

I built my chat app around this idea and to save money. When it comes to coding, I feel Sonnet 3.5 is still the best but I don't start with it. I tend to use cheaper models in the beginning since it usually takes a few iterations to get to a certain point and I don't want to waste tokens in the process. When I've reached a certain state or if it is clear that the LLM is not helping, I will bring in Sonnet to review things.

Here is an example of how the conversation between models will work.

https://beta.gitsense.com/?chat=bbd69cb2-ffc9-41a3-9bdb-095c...

The reason why this works for my application is, I have a system prompt that includes the following lines:

# Critical Context Information

Your name is {{gs-chat-llm-model}} and the current date and time is {{gs-chat-datetime}}.

When I make an API call, I will replace the template strings with the model and date. I also made sure to include instructions in the first user message to let the model know it needs to sign off on each message. So with the system prompt and message signature, you can say "what do you think of <LLM's> response".

> using LLMs is learning how to manage your context.

This is the most important thing in my opinion. This is why I switched to showing tokens in my chat app.

https://beta.gitsense.com/?chat=b8c4b221-55e5-4ed6-860e-12f0...

I treat tokens like the tachometer for a car's engine. The higher you go, the more gas you will consume, and the greater the chance you will blow up your engine. Different LLMs will have different redlines and the more tokens you have, the more costly every conversation will become and the greater the chance it will just start spitting gibberish.

So far, my redline for all models is 25,000 tokens, but I really do not want to go above 20,000. If I hit 16,000 tokens, I will start to think about summarizing the conversation and starting a new one based on the summary.

The initial token count is also important in my opinion. If you are trying to solve a complex problem that is not well known by the LLM and if you are only starting with 1000 or less tokens, you will almost certainly not get a good answer. I personally think 7,000 to 16,000 is the sweet spot. For most problems, I won't have the LLM generate any code until I reach about 7,000 since it means it has enough files in context to properly take a shot at producing code.

> 1. I find a "pair programming" mentality is key. I focus on the high-level code, and let the model focus on the lower level code. I code review all the code, and provide feedback. Blindly accepting the code is a terrible approach.

This is what I've found to be key. If I start a new feature, I will work with the LLM to do the following:

- Create problem and solution statement

- Create requirements and user stories

- Create architecture

- Create skeleton code. This is critical since it lets me understand what it wants to do.

- Generate a summary of the skeleton code

Once I have done the above, I will have the LLM generate a reusable prompt that I can use to start LLM conversations with. Below is an example of how I turn everything into a reusable prompt.

https://beta.gitsense.com/?chat=b96ce9e0-da19-45e8-bfec-a3ec...

As I make changes like add new files, I will need to generate a new prompt but it is worth the effort. And you can see it in action here.

https://beta.gitsense.com/?chat=b8c4b221-55e5-4ed6-860e-12f0...

The first message is the reusable prompt message. With the first message in place, I can describe the problem or requirements and ask the LLM what files it will need to better understand how to implement things.

What I am currently doing highlights how I think LLM is a game changer. VCs are going for moonshots instead of home runs. The ability to gather requirements and talk through a solution before even coding is how I think LLMs will revolutionize things. It is great that it can produce usable code, but what I've found it to be invaluable is it helps you organize your thoughts.

In the last link, I am having a conversation with both DeepSeek v3 and Sonnet 3.5 and the LLMs legitimately saved me hours in work, without even writing a single line of code. In the past, I would have just implemented the feature and been done with it, and then I would have to fix something if I didn't think of an edge case. With LLMs, it literally takes minutes to develop a plan that is extremely well documented that can be shared with others.

This ability to generate design documents is how I think LLMs will ultimately be used. The bonus is producing code, but the reality is that documentation (which can be tedious and frustrating) is a requirement for software development. In my opinion, this is where LLMs will forever change things.

Yeah, it almost feels like you are talking to somebody with OCD. The frustrating part is, output tokens are usually a lot more expensive than input tokens, so they are wasting energy and money :-). Also, the more they generate, the greater the chance it will create attention issues as the conversation progresses.

This is why I built my chat app to let me manipulate LLM responses. If I feel it is not worth knowing, I'll just erase parts of it to ensure the conversation doesn't get side tracked. Or I will go back to the original user message and modify it to say

### IMPORTANT

- Do not generate more code than required.

The nice thing about LLM conversations are, every time you chat, the LLM treats it as a first time conversation, so this trick will work if the model is smart enough.

I guess the question is, did DeepSeek force them to rethink pricing? It's crazy how much cheaper it (v3 and R1) is, but considering they (Deepseek) can't keep up with demand, the price is kind of moot right now. I really do hope they get the hardware to support the API again. The v3 and R1 models that are hosted by others are still cheap compared to the incumbents, but nothing can compete with DeepSeek on price and performance.