sdesol

No, it is much more involved and not all providers allow the necessary tweakings. This means you will need to use local models (with hardware caveats) which will require us to ask:

- Are local models good enough?

- What are we giving up for deterministic behaviour?

For example, will it be much more difficult to write prompts. Will the output be nonsensical and more.

> Cheap code can be had by hiring in cheap locations (outsourced for example).

If you outsource and like what you get, you would assume the place you outsourced to can help provide continued support. What assurance do you have with LLMs? A working solution doesn't mean it can be easily maintained and/or evolved.

> And i have not seen a single customer demonstrate that they care about the quality of the code base behind the product they enjoy paying for.

That is true, but they will complain if bugs cannot be fixed and features are added. It is true that customers don't care, and they shouldn't, until it does matter, of course.

The challenge with software development isn't necessarily with the first iteration, but rather it is with continued support. Where I think LLMs can really shine is in providing domain experts (those who understand the problem) with a better way to demonstrate their needs.

> unhampered by OpenAI’s schedule

Sure, but if the advancements are to catch up to OpenAI, then major improvements by other vendors are nice and all, but I don't believe that was what the commenter was implying. Right now the leaders in my opinion are OpenAI and Anthropic and unless they are making major improvements every few months, the industry as a whole is not making major improvements.

> assuming you have full control over which compiler youre using for each step ;)

With existing tools, we know if we need to do something, we can. The issue with LLMs, is they are very much black boxes.

> What's to say LLMs will not have a "compiler" interface in the future that will reign in their variance

Honestly, having a compiler interface for LLMs isn't a bad idea...for some use cases. What I don't see us being able to do is use natural language to build complex apps in a deterministic manner. Solving this problem would require turning LLMs into deterministic machines, which I don't believe will be an easy task, given how LLMs work today.

I'm a strong believer in that LLMs will change how we develop and create software development tools. In the past, you would need Google and Microsoft level of funding to integrate natural language into a tool, but with LLMs, we can easily have LLMs parse input and have it map to deterministic functions in days.

> we can expect major improvements every few months.

I'm not sure this is grounded in reality. We've already seen articles related to how OpenAI is behind schedule with GPT-5. I do believe things will improve over time, mainly due to advancements in hardware. With better hardware, we can better brute force correct answers.

> junior devs will always be junior devs

Junior developers turn into senior developers over time.

> Machine -> Asm -> C -> Python -> LLM (Human language)

Something that you neglected to mention is, with every abstraction layer up to Python, everything is predictable and repeatable. With LLMs, we can give the exact same instructions, and not be guaranteed the same code.

I would say this is true until we hit a "oh shit, they can really sue us, even if we warn them" moment. I'm imagining legislation will come into play that will make it less ideal for business critical solutions. I know Air Canada has been sued and probably others.

I still think we are in the honeymoon phase, and once that is over, LLM will become what it is meant to be, which is a power tool for domain experts.

I asked a bunch of LLMs using your prompt. The only change was to just use meters.

https://beta.gitsense.com/?chat=bb57a248-e14a-4f33-bbe9-2fa9...

1622m is most agreed upon. The interesting numbers are the ones with less than 50% agreement. Not sure if they are hallucinations or if they are outdated data.

Click the conversation link in the user message bubble to see the response from each LLM.

Thanks for the info on local LLMs. Based on my chats with multiple LLMs, the biggest issue appears to be hardware.

Non-deterministic hardware: All LLMs mentioned that modern computing hardware, such as GPUs or TPUs, can introduce non-determinism due to factors like parallel processing, caching, or numerical instability. This can make it challenging to achieve determinism, even with fixed random seeds or deterministic algorithms.

You can find the summary of my chats https://beta.gitsense.com/?chat=1c3e69f9-7b8b-48a3-8b99-bb1b.... If you scroll to the top and click on the "Conversation" link in the first message, you can read the individual responses.

I agree with this. I keep harping on this, but we are sold automation instead of a power tool. If you have domain knowledge in the problem that you are solving, then LLMs can become an extremely valuable aid.

> LLMs are, at their core, search tools.

This is the wrong take. Search tools are deterministic unless you purposely inject random weights into the ranking. With search tools, the same search query will always yield the same search result, provided they are designed too and/or the underlying data has not changed.

With LLMs, I can ask the exact same question and get a different response, even if the data has not changed.

I asked a bunch of models to review the Phind response at

https://beta.gitsense.com/?chats=a5d6523c-0ab8-41a8-b874-b31...

The left side contains the Phind response that I got and the right side contains a review of the response.

Claude 3.5 Sonnet, GPT-4o and GPT-4o mini was not too happy with the response and called out the contradiction.

Edit: Chat has been disabled as I don't want to incur an unwanted bill

> whether Kotaemon as a standalone tool will be needed for much longer.

I think a lot of AI startups will find themselves in this situation. Searching and summarizing docs is a no-brainer for OpenAI, Anthropic, etc. The only issue they have right now is that their models might not be reliable enough due to the non-deterministic nature of LLMs. In the long term, I believe Google, Amazon and Microsoft will probably be the big winners in this area since they can offer multiple models from major providers to de-risk things.

Unless AI complements an existing solution that is unique and/or is done well by existing businesses, it will be very difficult to compete.

> Perhaps stemming from the over-broad term “AI”.

No, I think if we follow the money, we will find the problem.

Like I've said before, this isn't a physical good that is being sold. LLM is knowledge and many governments and people are going to be concerned with how and who is packaging it. Using LLMs created by China will mean certain events in history will be omitted (which will not be exclusive to China). How the LLM responds will be dictated by the LLM training and so forth.

LLMs will become the ultimate propaganda tool (if they aren't already), and I don't see why governments wouldn't want to have full control over them.

> if today you told me you had a website that was 100% technologically identical to Amazon, but you were willing to make slightly less money, you'd still have no products

It is widely understood that you really can't compete with "as good as". People won't leave Google, Facebook, etc. if you can only provide a service as good as, because the effort required to move would not be worth it.

> if you show me a product that works just as well as ChatGPT but costs less--or which costs the same but works a bit better--I would use it

This is why I believe LLMs will become a commodity product. The friction to leave one service is nowhere near as as great as leaving a social network. LLMs in their current state are very much interchangeable. OpenAI will need a technological breakthrough in reliability and/or cost to get people to realize that leaving OpenAI makes no sense.

> Then again I'd suggest that the majority of people participating on those sites are already employed, so they're not doing it for that benefit.

But the company benefits from less confusion and a better user experience. Companies are literally paying employees to provide content as it benefits the company.

I do believe there are people who freely choose to contribute with no strings attached, and I guess we'll learn in the coming years if people will contribute their time and effort for benevolent reasons.

I have that in the chat app that I am working on.

https://beta.gitsense.com/?chat=51219672-9a37-442d-80a3-14d8...

It provides a summary of all the responses and if you click on "Conversation" in the user message bubble, you can view all the LLM responses to the question of "How many r's in strawberry".

You can fork the message as well and say create a single response based on all responses.

Edit: The chatting capability has been disabled as I don't want to incur an unwanted bill.

I will not be surprised if Open AI, Claude, Meta and others use the feedback system to drive corrections. Basically, if we use the API, we may never get the best answer, but it could also be true that all feedback will be applied to future models.

We are definitely heading into untreaded territory. It is one thing to use Chinese EVs, but using a knowledge system that will be censored (not that other countries won't be censored) and trained in a way that may not align with a nation's beliefs is a whole different matter.

Most people may not care but that doesn't matter if the government cares. I will not be surprised if countries restrict LLMs to trusted countries in the future. Unless there is a regime change in China, seeing it adopted in other countries may be an issue.

Canada would be reluctant. So would Europe, South Asia and so forth. The biggest hurdle for China is the CCP. It is one thing to use physical products from China but relying on the CCP for knowledge may be a step too far for many nations.

Not prohibit training, but make it illegal for US companies to embed, use or distribute LLMs created by China. Basically the general consumer will need to go through the effort of use the LLM that they want, which we know means only a small fraction of people.

Here's more LLMs

https://beta.gitsense.com/?chats=ba5f73ac-ad76-45c0-8237-57a...

The left window contains all the models that were asked and the right window contains a summary of the LLM responses. GPT-4o mini got it right but the super majority got it wrong, which is scary.

It wasn't until the LLM was asked to count out the R's that it acknowledges that GPT-4o mini was the only one that got it right.

Edit: I've disabled chatting in the app, since I don't want to rack up a bill. Should have mentioned that.

> That's a very small price to pay for what we all stand to gain.

Sure, if AI was made free for everybody (or only be charged for cost to run).

With Stack Overflow, GitHub and others, there is a mutual understanding that contributing can benefit the contributor. What is the incentive to continue contributing if the social agreement is, you get to help define a statistical weight for the next token and nobody will know?

I think the future business model may require AI companies to pay people to contribute, or it might not be a technology roadblock, but rather a data roadblock that prevents further advancement.

> China would still pose a problem

Not if regulation prohibits LLMs from China, which isn't that far fetched to be honest.

I think LLM will turn into a commodity product and if you want to dominate with a commodity product, you need to provide twice the value at half the cost. Open AI will need a breakthrough in reliability and/or inference cost to really create a moat.

With StackOverflow, GitHub, etc. you would likely have people reach out to you for opportunities. With AI, if you contribute to StackOverflow and if it gets picked up by AI, people may or may not know.

> I didn't start contributing to make money

Your intentions might not have been to make money, but you were creating social credit that could be redeemed for a higher and better paying job. With GitHub, Stack Overflow, etc., you are adding to your resume, but with AI, you literally get nothing in return for contributing.

I agree LLMs are good at text processing and I believe they will obsolete jobs that really should be obsoleted. Unless OpenAI, Anthropic and other AI companies come up with a breakthrough on reliability, I think it will be fair to say they will only be players and not leaders. If they can't figure something out, it will be Microsoft, Amazon and Google (distributors of diverse models) that will benefit the most.

I've personally found it is extremely unlikely for multiple good LLMs to fail at the same time, so if you want to process text and be confident in the results, I would just run the same task across 5 good models and if you have a super majority, you can be confident that it was done right.

I think your point stands, but your example shows that anyone using those calculators daily should not be concerned. Those that need precision to the 6+ decimal places for complex equations should know not to fully trust consumer-grade calculators.

The issue with LLMs is that they can be so unpredictable in their behaviour. Take the following prompt that asks GPT-4 to validate the response to "calculate 2+3+5 and only display the result":

https://beta.gitsense.com/?chat=6d8af370-1ae6-4a36-961d-2902...

GPT-4o mini contradicts itself, which is not something one would expect for something we believe to be extremely simple. However, if you ask it to validate the response to "calculate 2+3+5," it will get it right.

https://beta.gitsense.com/?chat=43221de5-bff6-487a-8c0f-48ca...

By adding "and only display the result," GPT-4o mini was thrown for a loop; examples like this should give us pause.