HNHacker News
TopNewBestAskShowJobs

noodletheworld

444 karma · joined December 14, 2024

submissionscomments
noodletheworld··on Ollama's new engine for multimodal models
I'm responding to this assertion:

> Ollama is written in golang so of course they can not meaningfully contribute that back to llama.cpp.

llama.cpp consumes GGML.

ollama consumes GGML.

If they contribute upstream changes, they are contributing to llama.cpp.

The assertions that they:

a) only write golang

b) cannot upstream changes

Are both, categorically, false.

You can argue what 'meaningfully' means if you like. You can also believe whatever you like.

However, both (a) and (b), are false. It is not a matter of dispute.

> Whatever gotcha you think there is, exists only in your head.

There is no 'gotcha'. You're projecting. My only point is that any claim that they are somehow not able to contribute upstream changes only indicates a lack of desire or competence, not a lack of the technical capacity to do so.

noodletheworld··on Ollama's new engine for multimodal models
What nonsense is this?

Where do you imagine ggml is from?

> The llama.cpp project is the main playground for developing new features for the ggml library

-> https://github.com/ollama/ollama/tree/27da2cddc514208f4e2353...

(Hint: If you think they only write go in ollama, look at the commit history of that folder)

noodletheworld··on Refactoring Clojure
Seems fair, but refactoring is generally tricky and:

1) replacing a func with a simpler recursive func may or may not be ideal.

2) this fills me with no joy when I read it:

> It seems easier to start from scratch as opposed to refactor it. The characterization tests above will help ensuring that the returned hash map stays the same.

Its enormously hmm… what word to use. Ambitous? Foolish? Brave?

Anyway, taking code, writing some tests for it and then rewriting the entire thing so it passes the tests you wrote only barely falls into “refactoring”.

If anyone refactored my code base by simply rewriting from scratch the parts they dont like, I would be upset.

Rewriting things is hard, and its … brave… to think you can just write a few tests and then if they all pass its all good!

That only works in very trivial examples, and really, doesnt show case the strengths of clojure at all.

noodletheworld··on Flattening Rust’s learning curve
If you’re not prepared to be humble when you learn, you’re not actually trying to learn. (Or not trying very hard anyway),

There’s a difference between “do it like this” and “keep an open mind when you do this”.

You can learn rust any way you want; this is just a guide for how to learn it effectively.

A fairer comparison would be learning Japanese by going to Japan and insisting on speaking English except in Japanese language classes.

Yes, you can do that.

…but it is not the most effective way to learn.

The best way to learn is full immersion. It’s just harder.

If you don’t want that, don’t do it. It’s not a cult. That’s just lazy flippant anti-rust sentiment.

noodletheworld··on Claude's system prompt is over 24k tokens with tools
It makes no difference.

Theres no difference between feeding an LLM a prompt and feeding it half the prompt, saving the state, restoring the state and feeding it other half of the prompt.

Ie. The data processed by the LLM is prompt P.

P can be composed of any number of segments.

Any number of segments can be cached, as long as all preceeding segments are cached.

The final input is P, regardless.

So; tldr; yes? Anything you can do with a prompt you can do, becasue its just a prompt.

noodletheworld··on Semantic unit testing: test code without executing it
I don't think this is particularly terrible.

Broadly speaking, linters are good, and if you have a way of linting implementation errors it's probably helpful.

I would say it's probably more helpful while you're coding than at test/CI time because it will be, indubitably, flakey.

However, for a local developer workflow I can see a reasonable value in being able to go:

Take every function in my code and scan it to figure out if you think it's implemented correctly, and let me know if you spot anything that looks weird / wrong / broken. Ideally only functions that I've touched in my branch.

So... you know. Cool idea. I think it's overselling how useful it is, but hey, smash your AI into every possible thing and eventually you'll find a few modestly interesting uses for it.

This is probably a modestly interesting use case.

> suite allows you to run the tests asynchronously, and since the main bottleneck is IO (all the computations happen in a GPU in the cloud) it means that you can run your tests very fast. This is a huge advantage in comparison to standard tests, which need to be run sequentially.

uh... that said, saying that it's fast to run your functions through an LLM compared to, you know, just running tests, is a little bit strange.

I'm certain your laptop will melt if you run 500 functions in parallel through ollama gemma-3.

Running it over a network is, obviously, similarly insane.

This would also be enormously and time consuming and expensive to use with a hosted LLM api.

The 'happy path' is probably having a plugin in your IDE that scans the files you touch and then runs this in the background when you make a commit somehow using a local LLM of sufficient complexity it can be useful (gemma3 would probably work).

Kind of like having your tests in 'watch mode'; you don't expect instant feedback, but some-time-after you've done something you get a popup saying 'oh hey, are you sure you meant to return a string here..?'

Maybe it would just be annoying. You'd have to build it out properly and see. /shrug

I think it's not implausible though, that you could see something vaguely like this that was generally useful.

Probably what you see in this specific implementation is only the precursory contemplations of something actually useful though. Not really useful on its own, in its current form, imo.

noodletheworld··on RustAssistant: Using LLMs to Fix Compilation Errors in Rust Code
> so types are not as big a help as one might think.

Yes, they are.

An agent can combine the compiler type system and iterate.

That is impossible using clojure.

The reason you have problems woth ocaml is that the tooling youre using is too shit to support iterating until the compiler passes before returning the results to you.

…not because tooling doesnt exist. Not because the tooling doesn't work.

—> because you are not using it.

Sure, rust ownership makes it hard for LLMs. Faaair point; but ultimately, why would a coding agent ever suggest code to you that doesnt compile?

Either: a) the agent tooling is poor or b) it is impossible to verify if the code compiles.

One of those is a solvable problem.

One is not.

(Yes, what many current agents do is run test suites; but dynamically generating valid tests is tricky; checking if code compiles is not tricky.)

noodletheworld··on RustAssistant: Using LLMs to Fix Compilation Errors in Rust Code
Hot take: this is the future.

Strongly typed languages have a fundamentally superior iteration strategy for coding agents.

The rust compiler, particularly, will often give extremely specific “how to fix” advice… but in general I see this as a future trend with rust and, increasingly, other languages.

Fundamentally, being able to assert “this code compiles” (and iterate until it does) before returning “completed task” is superior for agents to dynamic languages where the only possible verification is runtime.

(And at best the agent can assert “i guess it looks ok”)

noodletheworld··on MCP on AWS Lambda with MCPEngine
The thing about MCP is people can “do AI” without any AI.

That makes it enormously attractive for people who want to be part of the AI hype cycle but not devote much actual effort to it.

Especially since you dont have to actually do anything useful, just write a wrapper around something that already exists.

Wow! Now youre part of the AI hype cycle!

Maybe you too (like windsurf) can be bought for billions of dollars.

So… lots of people.

noodletheworld··on Atuin Desktop: Runbooks That Run
I mean, is it worse than having it:

- in excel

- in a confluence document

- in a text file on your desktop

The use case this addresses is 'adhoc activites must be performed without being totally chaotic'.

Obviously a nice one-click/trigger based CI/CD deployment pipeline is lovely, but uh, this is the real world. There are plenty of cases where that's simply either not possible, or not worth the effort to setup.

I think this is great; if I have one suggestion it would just be integrated logging so there's an immutable shared record of what was actually done as well. I would love to be able to see that Bob started the 'recover user profile because db sync error' runbook but didn't finish running it, and exactly when that happened.

If you think it's a terrible idea, then uh, what's your suggestion?

I'm pretty tired of copy-pasting commands from confluence. I think that's, I dunno, unambiguously terrible, and depressingly common.

One time scripts that are executed in a privileged remote container also works, but at the end of that day, those script tend to be specific and have to be invoked with custom arguments, which, guess what, usually turn up as a sequence of operations in a runbook; query db for user id (copy-paste SQL) -> run script with id (copy paste to terminal) -> query db to check it worked (copy paste SQL) -> trigger notification workflow with user id if it did (login to X and click on button Y), etc.

noodletheworld··on CaMeL: Defeating Prompt Injections by Design
I have to say I’m a bit skeptical.

The problem with a sandbox that executes arbitrary code (which is what this is; convert a request into code and execute it in a restricted runtime), is that if you expose APIs in that sandbox that can “do things”, then you have to be extraordinarily careful with your security policies to allow “good actions” and deny “bad actions”.

The side channel attacks are a good example; what if fetching an external url is the task you want an agent to perform?

How do “know” in your security policy what a good url is and what a bad one is?

What if the action is to apply a change to a database element? How does your security policy know how to only allow “good” updates?

Certainly you can hand craft guard rails (security policies), but at the end of the day you’re no closer or further than any other environment where you’re executing arbitrary code; it just takes different efforts to find the holes in those security policies and apis.

Ie. it’s easy to say “if you write a good sandbox and covert your LLM request into code and run it in the sandbox you’re fine”.

…but you’re as fine as your sandbox is; if your goal is a sandbox with holes in it for privileged actions; guess what, the arbitrary code you run in it can call those privileged actions.

Certainly the data provenance is a cool idea, but I foresee see a lot of “but but but…” when people try to enforce the boundaries in practice.

noodletheworld··on Gemma 3 QAT Models: Bringing AI to Consumer GPUs
https://huggingface.co/google/gemma-3-27b-it-qat-q4_0-gguf/t...

> 17 days ago

Anywaaay...

I'm literally asking, quite honestly, if this is just an 'after the fact' update literally weeks later, that they uploaded a bunch of models, or if there is something more significant about this I'm missing.

noodletheworld··on Gemma 3 QAT Models: Bringing AI to Consumer GPUs
I understand that, but the qat models (1) are not new uploads.

How is this more significant now than when they were uploaded 2 weeks ago?

Are we expecting new models? I don’t understand the timing. This post feels like it’s two weeks late.

[1] - https://huggingface.co/collections/google/gemma-3-qat-67ee61...

noodletheworld··on Gemma 3 QAT Models: Bringing AI to Consumer GPUs
?

Am I missing something?

These have been out for a while; if you follow the HF link you can see, for example, the 27b quant has been downloaded from HF 64,000 times over the last 10 days.

Is there something more to this, or is just a follow up blog post?

(is it just that ollama finally has partial (no images right?) support? Or something else?)

noodletheworld··on The AI skeptic's guide to AI collaboration
> The only thing that I have seen convince people (and it always does)

...when anyone starts talking in universals like this, they're usually deep in some hype cycle.

This is a problematic approach that many people take; they posit that:

1) AI is fundamentally transformative.

2) People who don't acknowledge that simply haven't tried it.

However, I posit that:

3) People who think that haven't actually used it a serious capacity or are deliberately misrepresenting things.

The problem is that:

> In reality, I go back and forth with AI constantly—sometimes dozens of times on a single piece of work. I refine, iterate, and improve each part through ongoing dialogue. It's like having a thoughtful and impossibly fast colleague who's always available to help me develop and sharpen my ideas.

...is only true for trivial problems.

The author calls this out, saying:

> It won't excel at consistently citing specific papers, building codes, or case law correctly. (Advanced techniques exist for these tasks, but they're not worth learning when you're just starting out. For now, consider them out of scope.)

...but, this is really the heart of everything.

What are those advanced techniques? Seriously, after 30 days of using AI if all you're doing is:

> Prepare for challenging conversations by using ChatGPT to simulate potential scenarios, helping you approach interpersonal dynamics with empathy and grace.

Then what the absolute heck are you doing.

Stop gaslighting everyone.

Those 'advanced techniques' are all anyone cares about, because they are the things that are hard, and don't work.

In reality, it doesn't matter how much time you spend learning; the technology is fundamentally limited. It can't do some things.

Spending time learning how to do trivial things will never enable you to do hard things.

It's not missing the 'human touch'.

It's the crazy hallucinations, invalid logic, failure to do as told, flat out incorrect information or citations, inability to perform a task (eg. as an agent) without messing some other thing up.

There are a few techniques that can help you have an effective workflow; but seriously, if you're a skeptic about AI, spending a month doing trivial stuff like asking for '10 ideas about X' is an insult to your intelligence and doesn't address any of the concerns that, I would argue, skeptics and real people actually have about AI.

noodletheworld··on Defold: cross-platform game engine
… space age is a mod.

Obviously any engine will have a lua scripting api written in c++ but if youre somehow casting shade about factorio being a lua success story, youre barking up the wrong tree.

noodletheworld··on Defold: cross-platform game engine
That seems kind of unfair.

There are things that aren't great about lua, and I'm not going to pretend they're not. buuuut...

- It's really perfectly ok if you use competently.

- There are large complex systems and games built using it (eg. factorio) that show that scaling it is both pragmatic and possible.

- It has a big active community.

When you compare it some other scripting solutions (boo, unityscript) that were just incompetently done, or mystifyingly deciding that writing your own language is the solution (jai, gdscript), I feel you have to admit it's a reasonably pragmatic choice.

There aren't a lot of other languages that embed as a scripting layer the same way that lua does, with the same performance.

Many candidates are either very hard to work with embedded (python) or simply lack any feasible way to possible embed them as a runtime scripting layer (go, zig, c, etc).

Maybe there's a future for WASM scripting layers in games, but for now, I certainly haven't seen it done well.

I can confidently say: Lua, the language, is not a minefield. Not to me. Not to you. Not to anyone.

Some projects expose a lua scripting API that is a minefield; ...but those two things are not the same thing, and by all accounts (I haven't used it personally, but just idly reading the forums) the defold scripting layer seems fine to most people.

noodletheworld··on Everything wrong with MCP
> that information applies to all interactions made via that session

Humm.. maybe you should run an llama.cpp server in debug mode and review the content that goes to the actual LLM; you can do that with the verbose flag or `OLLAMA_DEBUG=1` (if you use ollama).

What you are describing is not how it works.

There is no such thing as an LLM 'session'.

That is a higher level abstraction that sits on top of an API that just means some server is caching part of your prompt and taking some fragment you typed in the UI and combining them on the server side before feeding them to the LLM.

It makes no difference how it is implemented technically.

Fundamentally; any request you make which can invoke tools will be transformed, as some point, into a definition that includes the tool definitions before it is passed to the LLM.

That has a specific, measurable cost on LLM performance as the number of tool definitions go up.

The only solution to that is to limit the number of tools you have enabled; which is entirely possible and reasonable to do, by the way.

My point is that adding more and more and more tools doesn't scale and doesn't work.

It only works when you have a few tools.

If you have 50 MCP servers enabled, your requests are probably degraded.

noodletheworld··on Everything wrong with MCP
> they only incur a cost to the client, and only impact the user context, if/when they're invoked.

Look it up. Look up the cross server injection examples.

I guarantee you this is not true.

An MCP server is at it's heart some 'thing' that provides a set of 'tools' that an LLM can invoke.

This is done by adding a 'tool definition'.

A 'tool definition' is content that goes into the LLM prompt.

That's how it works. How do you imagine an LLM can decide to use a tool? It's only possible if the tool definition is in the prompt.

The API may hide this, but I guarantee you this is how it works.

Putting an arbitrary amount of 3rd party content into your prompts has a direct tangible impact on LLM performance (and cost). The more MCP servers you enable the more you pollute your prompt with tool definitions, and, I assure you, the worse the results are as a result.

Just like pouring any large amount of unrelated crap into your system prompt does.

At a small scale, it's ok; but as you scale up, the LLM performance goes down.

Here's some background reading for you:

https://github.com/invariantlabs-ai/mcp-injection-experiment...

https://docs.anthropic.com/en/docs/build-with-claude/tool-us...

noodletheworld··on Everything wrong with MCP
Some of the other issues are less important than others, but even if you accept “you have to take responsibility for yourself”, let me quote the article:

> As mentioned in my multi-agent systems post, LLM-reliability often negatively correlates with the amount of instructional context it’s provided. This is in stark contrast to most users, who (maybe deceived by AI hype marketing) believe that the answer to most of their problems will be solved by providing more data and integrations. I expect that as the servers get bigger (i.e. more tools) and users integrate more of them, an assistants performance will degrade all while increasing the cost of every single request. Applications may force the user to pick some subset of the total set of integrated tools to get around this.

I will rephrase it in stronger terms.

MCP does not scale.

It cannot scale beyond a certain threshold.

It is Impossible to add an unlimited number of tools to your agents context without negatively impacting the capability of your agent.

This is a fundamental limitation with the entire concept of MCP and needs addressing far more than auth problems, imo.

You will see posts like “MCP used to be good but now…” as people experience the effects of having many MCP servers enabled.

They interfere with each other.

This is fundamentally and utterly different from installing a package in any normal package system, where not interfering is a fundamental property of package management in general.

Thats the problem with MCP.

As an idea it is different to what people trivially expect from it.

noodletheworld··on Wasting Inferences with Aider
It may not be as stupid as it sounds.

Randomising LLM outputs (temperature) results is outputs that will always have some degree of hallucination.

That’s just math. You can’t mix a random factor in and magically expect it to not exist. There will always be p(generates random crap) > 0.

However, in any probabilistic system, you can run a function k times and you’ll get an output distribution that is meaningful if k is high enough.

3 is not high enough.

At 3, this is stupid; all you’re observing is random variance.

…but, in general, running the same prompt multiple times and taking some kind of general solution from the distribution isn’t totally meaningless, I guess.

The thing with LLMs is they scale in a way that actually allows this to be possible, in a way that scaling with humans can’t.

… like the monkeys and Shakespeare, there probably a limit to the value it can offer; but it’s not totally meaningless to try it.

noodletheworld··on The Agent2Agent Protocol (A2A)
yes.

This has been discussed before, but the short version is: there is no solution currently, other than only use trusted sources.

Unless there is a way beyond a flat text file to distinguish different parts of the “prompt data” so they cannot interfere with each other (and currently there is not), this idea of arbitrary content going into your prompt (which is literally what MCP does) can’t be safe.

It’s flat out impossible.

The goal of “arbitrary 3rd party content in prompt” is fundamentally incompatible with “agents able to perform privileged operations” (securely and safely, that is).

noodletheworld··on Senior Developer Skills in the AI Age
> Once again, the AI agent implemented this entire feature without requiring me to write any code manually.

> For controllers, I might include a small amount of essential details like the route name: [code]

Commit history: https://github.com/dx-tooling/platform-problem-monitoring-co...

Look, I honestly think this is a fair article and some good examples, but what is with this inane “I didn’t write any of it myself” claim that is clearly false that every one of these articles keeps bringing up?

What’s wrong with the fact you did write some code as part of it? You clearly did.

So weird.

noodletheworld··on Show HN: Nue – Apps lighter than a React button
Is there any documentation about them?

For example what is this 200kb binary for?

https://github.com/nuejs/nue/blob/master/packages/examples/s...

noodletheworld··on AI agents: Less capability, more reliability, please
Isn't the point he's making:

>> Yet too many AI projects consistently underestimate this, chasing flashy agent demos promising groundbreaking capabilities—until inevitable failures undermine their credibility.

This is the problem with the 'MCP for Foo' posts that recently.

Adding a capability to your agent that the agent can't use just gives us exactly that:

> inevitable failures undermine their credibility

It should be relatively easy for everyone to agree that giving agents an unlimited set of arbitrary capabilities will just make them terrible at everything; and that promising that giving them these capabilities will make them better is:

A) false

B) undermining the credibility of agentic systems

C) undermining the credibility of the people making these promises

...I get it, it is hard to write good agent systems, but surely, a bunch of half-baked, function-calling wrappers that don't really work... like, it's not a good look right?

It's just vibe coding for agents.

I think it's quite reasonable to be say, if you're building a system, now, then:

> The key to navigating this tension is focus—choosing a small number of tasks to execute exceptionally well and relentlessly iterating upon them.

^ This seems like exceptionally good advice. If you can't make something that's actually good by iterating on it until it is good and it does work, then you're going to end up being a devin (ie. over promised, over hyped failure).

noodletheworld··on Utah becomes first US state to ban fluoride in its water
Dentists have to spend 8 years at school right? …and do various annual training to stay licensed?

I’d say that’s a reasonable sign of someone qualified to have an opinion.

I think you’re getting confused with a dental technician.

noodletheworld··on Show HN: We made an MCP Server so Cursor can build things from REST API docs
> I understand MCP, the problem it solves, and why it seems a good idea

I don't.

Can you ELI5 why it's a good idea for me?

I see what people are doing (1), and I see what the intent is (2), but no one seems to have an explanation of why it helps.

The original MCP documentation has some vague hand waving about (3) about having prompts that would help ... with, something. ...but I don't see it in practice.

?

Is this why people are asking "does this actually work?"

...because, its obviously not any better than the 'tool use' API that everyone was already using. It can't be. It doesn't do anything differently.

I would bet that it actual makes agents less capable when you overload their context with unrelated tools that aren't relevant. Right?

How can it possibly help?

It sounds a lot like it rests on the assumption that adding tools is a zero-cost zero-impact +capability action only, but that can't be true.

(1) - https://github.com/punkpeye/awesome-mcp-servers

(2) - https://modelcontextprotocol.io/quickstart/server#what%E2%80...

(3) - https://modelcontextprotocol.io/docs/concepts/prompts

noodletheworld··on Apple restricts Pebble from being awesome with iPhones
I don't think this is a reasonable take (and also doesn't make sense). If you read the other comments here its extremely clear to everyone that no one here is seriously concerned about your watch sending spam.

If you 'opt-out' then it does not fix the problem of spammers having easier access to imessage.

I'll say that again: If you personally, either a) opt out, or b) dont have a watch at all, it makes no difference to spammers. They are not sending spam using your watch.

People are concerned about the result of allowing anyone to send messages in general.

...

The real argument to be made here is, as other people have pointed out, this isn't technically impossible; I mean, apple watch can do it right?

So there is a solution; but Apple isn't allowing other people to use it.

THAT is the issue here.

Pretending there is 'no issue' is flat out wrong. If that what you think, you're wrong and you haven't understood the problem.

The issue is why only Apple is, according to Apple, technically capable of implementing the controls required to do it in a secure manner.

That's a fair question to ask, and there really isn't a strong answer for it.

Certainly, making it easy for anyone to send messages would not be a solution. That would be stupid. That's why they haven't done that.

...but, you have to ask, is there really no certification process that would do the job? Really? None? Only Apple engineers are smart enough and diligent enough to do it without screwing it up? Hmmmmmmmmmm...

noodletheworld··on RubyLLM: A delightful Ruby way to work with AI
The threads have presumably been merged; look at the submitters in the history.

> 166 points|ksec|4 days ago|21 comments

> 168 points by ksec 4 hours ago

But yes; it’s a bit dodgy to resurface old news like this imo and pretend it’s new news.

I’d go as far as to say that being at #1, under the circumstances, means it’s been artificially boosted somehow.

I haven’t the foggiest why anyone would bother though.

noodletheworld··on A 10x Faster TypeScript
This seems like an oddly specific take on this topic.

You can get a conflicting resource in a shell by typing 'npm start' twice in two different shells, and it'll fail with 'port in use'.

My point is that you can do not conflicting IPC, but by default IPC is conflicting because it is intended to be.

You cannot bind the same port, semaphore, whatever if someone else is using it. That's the definition of having addressable IPC.

I don't think arguing otherwise is defensible or reasonable.

Having a concern that a network service might bind the same port as an other copy of the same network service deployed on the same target by another host is an entirely reasonable concern.

I think we're getting off into the woods here with an arbitrary 'die on this hill' point about semantics which I really don't care about.

TLDR: If you ship an IPC binary, you have to pay attention to these concerns. Pretending otherwise means you're not doing it properly.

It's not an idle concern; it's a real concern that real actual application developers have to worry about, in real world situations.

I've had to worry about it.

I think it's not unfair to think it's going to be more problematic than the current, very easy, embedded story, and it is a concern that simply does not exist when you embed a library instead of communicating using IPC.

← PreviousPage 7 of 8Next →