lnenad

Yeah 122B is the sweet spot for me as well. Even deepseek flash overthinks on stuff way too much. I think they fully rely on large reasoning turns to achieve better quality. The result of course means we wait a long time to get results even with high throughput as a lot of tokens are wasted.

Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.

I think as with most human undertakings, building isn't too much of a problem. Maintaining is. Even with what is still a relatively tame number of chargers you get a large number of them that are broken.

I LOVE the concept. I will play around with the execution, if it works as described this is a great product.

> If your needs are met otherwise stick to that and move on.

Weird to post such a thought in a forum where OP has posted their project for people to look at. I never mentioned any needs, I am saying the readme holds very little value for a project in a space that has 50 different projects/coding harnesses available.

How would I know if this fits my needs based on it saying LLMs can build tools for themselves? That's something they can do with any and every open source harness.

> But I am a hardware engineer biased by experience with minimalism; I got a single data structure (electromagnetism) to manipulate and BOMs that come with hard constraints.

Alrighty.

> On the contrary, web SaaS devs want to run a business/rocket to the moon moreso than be an engineer. As such git pulling, pip installing the world allows them to focus on their get rich quick with as little labor involved as possible goal front and center.

Posting your open source project to a forum such as hn means people will have opinions they want to voice. No need to shit on opinions you disagree with.

I think as many things that are posted here lately there is no *why* attached to the readme. Why would one use this, what is the benefit of this approach? Am I really gonna need my model to build exotic tools around it; or is exec/web_search/web_fetch enough for 90% of the use cases? Is my agent not capable of writing new plugins/tools for pi/opencode?

You've built a great piece of software, thank you!

I'm running gitea successfully with very little resources.

Deepseek is the enemy, the implication is being on hn you should know that and not work for them. /s

They've probably got companies lined up to give them jobs that will pay multiples of those. The runner up is even more interesting, with a slightly larger reason to feel like they should have won it. They did ~3nm (out of 4 iirc) of 4.04:1 ratio.

It's a great mindset to have, *if* you're able to control your emotions.

It's also a warzone visualization tool unintentionally.

Yeah not sure why this is on the front-page

Again the analogy isn't complete. If I went in knowing the teacher isn't an expert in the field and they might get some things wrong I'd definitely double check stuff. I'm sure you've had similar teachers in your life and you did this either way (I know I have).

> Part of the entire premise of the article is that many people aren’t being given a choice

Yeah, "part", but that's not the part I wrote about?

We're not talking about "the public". The person writing the article is a very experienced software engineer. There shouldn't be a gap in understanding.

You are right, it's not 500x8 it's 90x8.

I don't really see your point. You're still using the screwdriver and it's up to you to use it when it makes sense, see if it's doing it's job properly, fix issues it creates etc... If it's not doing a good job for your particular screw don't use it. But don't say it's a horrible tool since you choose to not use it properly.

There is a middle ground between CEOisms "LLMs are literally Jesus" and people similar to this blog's author "LLMs are mostly trash."

> It's more like a hammer that never drives nails straight unless you hold it in a specific uncomfortable manner.

I mean sure, pick any analogy, still doesn't change the fact that there are right uses and wrong uses; right ways to use it and wrong ways to use it.

> information retrieval tool

How is an LLM this? It's a non-deterministic word generator.

But English is literally the interface. It's the *same* as `--make-no-mistakes`, it's just your preference compared to directly telling it to the AI which is ok. That doesn't change the fact the using the tool correctly != using the tool.

> An Honest Review

> I do admit that this approach immediately triggered my contrarian side and made me very defiant of any AI tool.

Makes sense.

> While this could be partially remedied by always asking for a primary source or citation, I dislike the idea that one has to add magical incantations to their queries to get the right results. It’s a good laugh to make fun of “make no mistake” memes, until you start having to consider similar things seriously.

Man I am genuinely stumped at the obvious lack of desire to use something in a way it's supposed to be used. LLMs are tools and like with any tool it's on us to use it properly, not hitting a screw with a hammer and saying that hammers are a very stupid tool.

830t/s is burst aggregate. ~500 is sustained and it's for 8 concurrent users. Meaning for $1.99/hour if you serve 8 users it's 8*$0.54, not just $0.54.

You shouldn't rent one out if you're just serving it for yourself, but from a financial standpoint if you sell to users you can take a 100% margin.

Not just that, even if we had the power to do so, we don't know anywhere near 100% of our bodies.

I actually made something that has that exact purpose, making reviewing easier. https://github.com/lnenad/difiko

I actually made the same thing in Rust https://github.com/lnenad/difiko

Just FYI a large portion is vibecoded for those that don't like that.

Cool demo, but even the smallest model does 2.5 tps on my 7900xtx which is not great.

Another happy user here. Thank you for the awesome project.

I don't hate AI companies. The key value proposition is gather data > feed it to AI for semantic analysis (does the actual work, is a prompt) > display it in a UI

It's still a prompt, it's just not a static one. Either way props for building a company from it.

I thought startups wrapping prompts would require something a more complex than semantic analysis, which is literally what this is. And for 500 bucks. Wow. Props for being able to sell this.

I don't get the appeal of the UI, why is it so complex/convoluted.