HNHacker News
TopNewBestAskShowJobs

hobofan

8,360 karma · joined March 21, 2014

Jack of all trades (- master of some?)

goisser94@gmail.com

https://hobofan.com

Mastodon: @hobofan@toot.berlin (Twitter: @hobofan)

---

Currently building https://github.com/EratoLab/erato

submissionscomments
hobofan··on The session you cannot take with you
> these are just bs to have people accept them, but is that actually the case?

I tried to outline that in my sibling comment[0], but I think the article also gives a good example in the "hidden searches" paragraph: There is really no good reason not to expose details about many of the builtin tools, or provide them as standalone services, other than protect their moat.

I don't think that "role confusion" attacks could be meaningful prevented by not showing reasoning traces. Prevention of reasoning traces is about as futile as prevention of system prompt leakage. And here you would just have to manage to exfiltrate a small number of reasoning traces per model family in order to distill the writing style of thinking traces.

[0]: https://news.ycombinator.com/item?id=49120015

hobofan··on The session you cannot take with you
I think the article gives a very good overview of a problem that most users of AI rarely evaluate / have to grapple with.

There really is a surprising amount of coupling that happens with many of the "frontier inference providers", where a lot of the powerful non-LLM extensions (web search, code execution) are packaged as simple "tools" on the surface, that build up a lot of moat. Those are parts that are in theory nicely separable from the inference API, and could be externalized via MCP servers, but are usually not offered as such by the inference providers themselves, and are often only available in a slightly less powerful variant from other providers.

We've faced that issue repeatedly while building a on-premise provider-agnostic Chat UI & platform[0], where even adding something as simple as an in-chat image generation tool for the end-users (which is just a build-in tool in the OpenAI Responses API), becomes a bit of an ordeal (though part of that is due to the MCP spec missing a native file transfer protocol as of today[1]).

I am quite hopeful though, as with recent shifts of interest towards open weight models, there will be more opportunities for companies offering alternative implementations in a easier plug-and-play manner.

[0]: https://github.com/EratoLab/erato

[1]: https://github.com/modelcontextprotocol/modelcontextprotocol...

hobofan··on UEFA and its national associations will not participate in FIFA competitions
The Corruption Perception Index only ranks the public sector.
hobofan··on Stacked PRs are now live on GitHub
We just today ran into a bug where if the branch a stacked PR points to is deleted, it can just stay stuck in "merging" without any additional feedback. I directly looked at the GH status page, because who knows, maybe the PR subsystem has a partial outage again, but no, it was just a bug in the stacked PR feature.
hobofan··on Superlogical
There are plenty industry-specific niche IDEs that you probably never heard of that make up a significant part of many companies product offering.

Jetbrains is valued around $7bil and their business is almost exclusively in IDEs.

Wolfram Research is valued probably around $100mil and their business also mostly consists of an IDE product.

---

With AI, I think the future of IDEs will be very split. A lot less work will happen in IDEs (in terms of LOC), but at the same time, I think AI will allow for a more sophisticated build-out of IDEs, so e.g. debug experiences will become a lot richer.

hobofan··on Superlogical
I'm not doubting that Mitchell Hashimoto is capable of building great products with great product market fit. Quite the opposite, as the few strands of idolization that exist within me rank him as one of the best to ever do it in the dev tooling space.

I'm also not doubting that there would be tons of investors throwing themselves at him for a new venture like this.

But why, apart from forced market access would it be in his interest to take that money, given that he can nicely fund it for a few years with ~1% of his net worth if not for forced market access?

hobofan··on Superlogical
I don't doubt that, but that phenomenon seems to be reaching egregious levels.
hobofan··on Superlogical
I'm sorry to take it out on this specific announcement, as it just seems to be one of many "offenders", but why are the 15 investors listed on an announcement blog post of a semi-stealth startup?

With most of them being VC or high-count angel investors, they can combined push whatever product is being built here into 200+ companies independent of whether it is good or not. What happened to the good old way of figuring out product market fit first? Is that the only way a product can succeed nowadays, or why else would a ~billionair that could easily fund it himself pull in that many investors that early?

---

I'm excited to see what Superlogical will build, but the startup scene seems to grow more clique-y by the day.

hobofan··on We have proof automation now
> software is 1/10, reducing that to 1/100 makes a bog difference

I fully agree! I'm all for moving the needle towards moving the needle towards more formal verification, but what GP is suggesting is full verification of programs, which is more akin to trying to achieve a 1/10^6 bug rate.

hobofan··on We have proof automation now
> I, personally, think that it is ridiculous that ~none of the software we use is known to work correctly. It just happens to work correctly, most of the time.

I don't know. It seems to match the general futility of pedantry that exists in all professions. Yes, you can try to optimize for absolute 0 flaws in your code, but in a world where a CPU itself may have bugs in the way it executes assembly, that seems a bit silly beyond a certain point.

hobofan··on Lidl Introduces the European Payment Method Wero in Germany
This appears to be only for their online shop, not for their retail locations.
hobofan··on Future euro banknote design proposals
With the amount of security mechanisms in a banknote that have to be tailed to each design, I would assume that making that many variants would be financially irresponsible.
hobofan··on Launch HN: Screenpipe (YC S26) – Record how you work and turn that into agents
Anything that tracks employees actions beyond high-level "are they doing their job according to the job descriptions" is forbidden by privacy laws and otherwise will be striken down by a workers council.

That's not just specific to digital stuff. E.g. warehouse cameras that film anything else beyond securing entries/exits, like they are commonplace in e.g. the US don't exist for the same reason.

hobofan··on Jelly UI: Soft-body physics for native HTML form controls
I think there is still a point to be made that alternative more granular settings may help in less people turning it on.

e.g. I have "reduced motion" turned on because the workspace switching animation (and some of the other animations) on macOS is unbearably slow, and that flag is the only way to get rid of it.

hobofan··on China’s open-weights AI strategy is winning
> Also, enterprises don't give a rip if models are open.

They care about control. I see many of my enterprise (or just-below-enterprise) clients very annoyed at OpenAI and Google after 2-3 years of model toil, where they had to constantly re-calibrate onto new models, on tight externally mandated deadlines, with little certainity. Now they are reaching for open weight models instead, that they currently host with the same inference providers, but have the option to in-house if push comes to shove.

hobofan··on Topcoat: The full full-stack framework for Rust
Hmm... I'm a big fan of a lot of the things that the tokio project has built and a happy user of axum.

I'm not sure that projects like Topcoat and something like their ORM is a great direction for the project, and worry that they will possibly gain outsized adoption in the community based on name recognition rather than merit.

hobofan··on Show HN: A zoomable timeline of 4M Wikipedia events
Almost everything further than a few years in the future and a few predicted pangea events appear to be data errors.
hobofan··on The state of open source AI
> open source requires humans to volunteer their time

Your idealistic of open source may require that, but in practice a huge part of open source is commercial and a large chunk of that is low on collaboration (across vendor boundaries).

hobofan··on What's the story behind the names of Cloudflare's name servers? (2013)
Yeah, I'm sure they've spent a significant time of their whole engineering department in the 13 years since the blog post on this!
hobofan··on Sleep regularity is a stronger predictor of mortality risk than sleep duration (2023)
I think the point is that while it does act as negative acceleration there isn't a causal relationship with the actual speed of the car, which is mainly related to how far the gas pedal is pressed.
hobofan··on YouTrackDB is a general-use object-oriented graph database
Postgres 19, which is about to be released is adding support for SQL/PGQ.
hobofan··on YouTrackDB is a general-use object-oriented graph database
I can tell you from quotes that I've seen, that compared to the 2018 prices listed on the page, end-of-2022 pricing was slightly up per core.

We were already well on our way of dumping Neo4J due to performance/operational/architectural reasons at the point, but seeing the quote solidified it.

hobofan··on YouTrackDB is a general-use object-oriented graph database
Most graph databases only offer an advantage in query language, as it allows for more ergonomic graph traversal compared to e.g. recursive CTEs in SQL. However with SQL/PGQ, the same query ergonomics are coming to traditional databases.

If you look under the hood there is usually nothing special about graph databases that will make them more performant. If you lay out a query plan side-by side between e.g. Postgres and Neo4J, they will look identical, just that the leaf-nodes in Postgres will be a table-scan, while in Neo4J they will be either a vertex-scan or a edge-scan (which can both be seen as special cases of a table-scan).

As someone that has worked a lot with graph databases in the past, I'd largely recommend not using them. The price you pay in terms of worse ecosystem and less battle-tested maintenance tooling is not worth it to just have better syntax.

hobofan··on How we can reduce traffic congestion
In a situation like this, induced demand could "easily" (on a technical, not a political sustainable level) be combated by shifting subsidies from car -> public transport. Induced demand won't be taken up as readily if there is a significant monetary penalty for it.
hobofan··on I love LLMs, I hate hype
> that make the software useful, usable, and updated

There is a lot of OSS software out there (e.g. in scientific communities) that I would say would barely qualify for each of those three attributes. The main reason it's valuable for the respective communities, is because it's the only thing that's available.

hobofan··on Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
> I use pi btw

Not sure if intentionally meant as a reference, but it gives "I use Arch btw" vibes.

hobofan··on ChatGPT Work
It's interesting how OpenAI at the same time pioneered chat as primary interface for LLMs (+ have one of their strongest brands and defensibility for the common user there) and also seemingly wanting to steer away from it in so many ways.
hobofan··on Mistral's Robostral Navigate: a state of the art robotics navigation model
This is honestly such a great direction (or at least hedge) for Mistral. They are already a great fit for EU companies, and are establishing a good relationship with them.

If they can stand up a robotics software platform without US or China cloud ties, pair it with robotics hardware that is already in the process of commoditization, they'll be running in open doors in the EU manufacturing/logistics sector.

hobofan··on GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
It very much depends on what kind of system you are building, as each of them have different input/output modalities, each combination of them allowing for different attacks. If you are building a generic agent that can theoretically connect to anything and should build things end-to-end without interventions, then yes, it's very intractable to defend against prompt injection.

In more narrow cases, like Chat UIs it becomes a lot easier, though if it should appeal to a generic audience, still easy for individual users to misconfigure.

And if you want to use it in the most high-security environments where nothing can leak in/out, you will have to air-gap the system anyways (like any traditional software).

> You're perhaps making it harder to craft a prompt injection, but not impossible. This is a false sense of security.

It's not a false sense security, it's part of a layered security strategy. Yes, it will never be impossible, but so are many individual steps in cybersecurity attacks. There are other systems (like email) that are essentially impossible to fully lock down with purely mechanical security measures if you want to allow for meaningful work (e.g. having email attachments). A second-judge LLM when paired with keyword/pattern blocklists, and active alterting/lockout after repeated attack attempts can form a very robust line of defense that in practice can be near-impossible to break.

For many attacks, to have actual exploitability, you also need to have compromised a peripheral system (or user account) to have repeated attempts at circumventing prompt injection measures.

> And, are you absolutely sure you'll never forget to lock each and every thing down, including things you weren't aware of?

That's part of every normal (non-LLM) security audit. If you don't know what data can potentially go where, then you are open in attacks in any system. The AI space does add a bit of complexity here, if using MCPs hosted with third parties, though.

hobofan··on GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
Prompt injections are a whole class of vulnerabilities, and I would say there is generally a pretty good idea of how to mitigate them to be impactful. However in many cases those mitigations are not implemented (in the strictness that they require), as they are usually either too costly (second LLM as judge) or lead to worse UX (tool call confirmation with appropriate review of all input parameters on every tool call; disconnecting web access).
← PreviousPage 7 of 34Next →