pscanf

I have so many use cases for this. I just hope that we'll get a standard like https://www.openresponses.org/, because the idea of coupling an app to a particular vendor doesn't seem appealing at all, especially since it seems that the functionalities they offer are more or less the same. (I'm thinking about the OpenAI's Agent API and Anthropic's Agent SDK as "alternatives.)

It's open source, so yes, though there is no "easy setup" like a Docker image or a Cloudflare template yet.

The artifacts are only kept in your local indexeddb though, so they don't get uploaded. Or well, unless you start a collaboration session, but even then they are encrypted before uploading, and the key stays in your browser (in the url hash), so I can't see any data on my end.

But I definitely understand the concern. Would you prefer it in a Docker image? Or maybe wrapped in an Electron app?

Thanks! I did indeed write it with AI¹. It actually evolved from the approach I was using beforehand: asking every time my coding agent to break down the PR into steps and build an interface for me to go through them (because GitHub PR UI is effectively unusable for me). I got tired of burning tokens building the ~same UI over and over, so I thought "why not define a format for the review and build a decent UI just once?".

¹ Though actually the "harness of guidelines" I've been using recently make Claude not produce _complete_ garbage.

I can see how for a local this system might be better, but as a tourist I strongly prefer the tap in/tap out system, especially if I can just use my credit card.

I used the SBB app a few times, and it's actually very good, but it's still an "app experience". First, I need to set it up (download, login, add payment method), for which I need an internet connection (which I might not have as soon as I enter the country). Then, every time I need to purchase a ticket I need to open the app and figure out how to do what I need to.

Tapping in and out is superior in my view, even though I might end up overpaying a bit (though I'm probably still paying much less than what I'd pay in Switzerland :D ).

Author here. I definitely agree that communicating well is a prerequisite to getting decent results. On the other hand:

1. Even if you communicate perfectly, there's no guarantee that the LLM will "behave as instructed" and as you imagined it to. Indeed, the frustration often comes from the fact that you've said something as clear as day, yet the agent takes another path.

2. Part of the value of coding agents is exactly that you don't need to lay it all out perfectly for them. I mean, if I need to give the LLM every little implementation detail, I might as well write the code. Of course, I don't expect it to work off of "I want nice app make money", but I do expect some "intelligence" in figuring out the missing pieces.

> Every sentence says something. Every sentence is important and holds its weight. [...] Also, it makes the post harder (and more boring) to read.

I actually prefer that style of writing! (When it's not AI-generated ofc.) And I also try to use it in my technical blog posts. I usually re-read my drafts asking myself: "Does the reader actually care about this? Is this sentence adding something or is it just fluff?"

And actually I feel like AI text usually produces more fluff, or anyway I notice it more, but I see how it can make the result "robotic and boring".

We sorely need a way to reliably detect AI slop, but unfortunately it doesn't seem possible and it's just getting harder and harder.

Last month I tried my hand at finding a way to tell whether an OSS project is slop or not, based on the amount of "human attention" it received vs the amount of code it contains. The idea is that a 100k LOC project which received 3 days' worth of attention from a human is most certainly slop.

The approach doesn't work very well, though¹, mostly because it's hard to gauge the amount of attention that was given. If I see one commit with +3000 LOC, I can assume it's AI-generated, but maybe you're just the type of dev that commits infrequently.

Maybe we need some sort of "proof of human attention" for digital artifacts, that guarantees that a human spent X time working on it.

¹ I wrote about it here https://pscanf.com/s/352/

I recently ran an experiment where I tried to use _quantitative signals_ (and not _qualitative_ ones) to tell whether something is vibe-coded or not.

My idea was that, if I see that your project is growing 10k LOC per week and you're the only developer working on it, it's most likely vibe-coded.

I analyzed some open-source projects, but unfortunately it turns out not to be so clear cut. It's relatively easy to estimate the growth rate of a project, but figuring out how much time developers worked on it is very error prone, which results in both false positives and false negatives.

I wrote a post about it (https://pscanf.com/s/352/) if you're interested in the details.

I just finished¹ building an experimental tool that tries to figure out if a repo is slopware or not just by looking at it's git history (plus some GitHub activity data).

The takeaway from my experiment is that you can really tell a lot by how / when / what people commit, but conclusions are very hard to generalize.

For example, I've also stumbled upon the "merge vs squash" issue, where squashes compress and mostly hide big chunks of history, so drawing conclusions from a squashed commit is basically just wild guessing.

(The author of course has also flagged this. But I just wanted to add my voice: yeah, careful to generalize.)

¹ Nothing is ever finished.

Yeah, maybe it's that, though I still wouldn't expect someone to categorize the post as blogspam, even if they just glance at it. (At least according to my definition of blogspam, but I guess each has their own.) But yes, pragmatically I should probably remove the disclaimer.

I gave up on r/programming after an article I wrote (thoughtfully, without AI, even though the content might not have been super interesting) got mod-slapped with a stickied comment "This content is low quality, stolen, blogspam, or clearly AI generated".

Ironically, that comment was added three months after I posted the article, when it was nowhere near the front page anymore, in a clearly automated and AI-driven review.

Still salty about it.

Oh, interesting, I didn't know about this project, thanks for sharing!

I tried to implement something like this for the search functionality, but ended up going with "old school" lexical search instead.

Mostly because, in my experimentation, vector search didn't perform significantly better, and in some case it performed worse. All the while being much more expensive on several fronts: indexing time (which also requires either an API or a ~big local model), storage, search time, and implementation complexity.

And Superego's agent actually does quite well with the lexical search tool. The model usually tries a few different queries in parallel, which approximates a bit semantic search.

> The intention of this is to reduce hallucination on information extraction, right?

Correct.

> Also, how do you convert your docs / information into JSON documents?

Right now you have to add it yourself to the database. The idea is that you use Superego as the software in which you record your expenses / income / whatever, so the data is naturally there already.

But I'm also working on an "import from csv/json/etc" feature, where you drop in a file and the AI maps the info it contains to the collections you have in the database.

The problem with using markdown for this is that it's unstructured, so when the LLM does calculations on it, extracting data is error prone, you're never sure what exactly gets extracted, and it's then a hassle to verify (like the author had to do).

In my app Superego (https://github.com/superegodev/superego, shameless plug) I use structured JSON documents precisely for this reason, because with a well-defined schema the LLM can write TypeScript functions that must compile. This doesn't guarantee correctness, of course, but it actually goes a long way.

But doing my taxes was a use case I hadn't considered, and it's actually pretty neat! I'll be trying it myself next month (though I'm not looking forward to it).

That would simplify things, but in my opinion that is still too high a hurdle. I'm all about privacy and FOSS, but I don't self-host anything (except for my personal website).

I wish that more apps would instead move the logic to the client and use files on file syncing services as databases. Taking tasks as an example, if a task board were just a file, I could share it with you on Dropbox / Drive / whatever we both use, and we wouldn't need a dedicated backend at all.

The approach has limitations (conflict resolution, authorization, and latency are the big ones), but it is feasible and actually completely fine for lots of apps.

I only use GitHub (and actions) for personal open-source projects, so I can't really complain because I'm getting everything for free¹. But even for those projects I recently had to (partially) switch actions to a paid solution² because GitHub's runners were randomly getting stuck for no discernible reason.

¹ Glossing over the "what they're getting in return" part. ² https://www.warpbuild.com/

Ah yeah, I understand now. And I also agree with the quote then! (Though it does change the nature of the job, and it's not terribly enjoyable...)

> your methodology would seem to be a reasonable approach/response to get the benefits of this and to shield against the detriments

If you're referring to the sandboxing / isolation of each app, I agree. Plus, the user can change the app quite easily, so if when they spot a bug, they can tell the agent to fix it (and cross their fingers!).

> ensure that a human developer understands the code before committing

Just to clarify: for Superego's app there's no human developer oversight, though. At least for the ones the user self-creates. Obviously the user will check that the app they just made works, but they might not spot subtle bugs. I employ some strategies to _decrease the likelihood of bugs_ (I wrote a bit about it here https://pscanf.com/s/351/, if you're interested), but of course only formal verification would ensure there aren't any.

Yeah, I agree, though it wants to be slightly provocative as well: it's all about you, your data, your software, your rights.

Correct. Admittedly, graphic design is not even my passion, so there's probably lots of room for improvement. But at this point I've grown accustomed to the friendly face. :D

Yes! That's more or less the angle I'm going for. I mean, I don't aim just yet for Emacs-levels of malleability, but at least for something where you can create some useful day to day personal tools.

I'm building an app that is, in a way, a modern take on Lotus Notes (https://github.com/superegodev/superego), and I couldn't feel this more:

> It is hard, today, to explain exactly what Lotus Notes was.

Whenever I try to explain what it does to a non-tech person, I'm met with confused looks that make me quickly give up and mumble something like "It's for techies and data nerds". I think to myself "they're not my target audience".

But I actually would like them to be, at some point. In the 90s "the generality and depth of its capabilities meant that it was also just plain hard to use", but now LLMs lower a lot the barrier to entry, so I think there can be a renaissance of such malleable¹ platforms.

Of course, the user still needs to "know what they need" and see software as something that can be configured and shaped to their needs which, with "digital literacy" decreasing, might be a bigger obstacle than I think.

¹ https://www.inkandswitch.com/malleable-software

Very cool project!

Question regarding the pluggable js engine: I have an electron app where I'm currently using QuickJS to run LLM-generated code. Would edge.js be able (theoretically) to use electron's v8 to get a "sanboxed within electron" execution environment?

Oh, interesting perspective. I'm Italian, but from an Alpine valley not far from Austria, so I don't know what I should prefer. :D

But joking aside, putting it like that I'd think I'd prefer the German/Codex way of doing things, yet I'm in camp Claude. But I've always worked better with teammates that balance my fastidiousness, so maybe that's my answer.

> They act like terribly arrogant agents

Oh I feel that. I sometimes ask ChatGPT for "a review, pull no punches" of something I'm writing, and my god, the answers really get on my nerves! (They do make some useful points sometimes, though.)

> On a final note: when building agentic applications, I used to prefer GPT (a year ago), but I can't stand it now. Robotic, mechanic, constantly mis-using tools. I reach for Sonnet/Opus if I want competence and adherence to prompt, coupled with an impeccable use of tools. I reach for Gemini (mostly flash models) if I want an acceptable experience at a fraction of the price and latency.

Yeah, this has been almost exactly my experience as well.

Ah OK sorry, I misinterpreted. But yes, I double checked one case and I am indeed setting the parameter explicitly (defaulting to medium effort). But no luck. It feels like the model ignores what I'm telling it.

For example, I pass it a list of database collections and tools to search through them, ask a question that can very obviously be answered with them, and it responds with "I can’t tell yet from your current records" (just tested with GPT 5.4-mini).

But I've prodded it a bit more now, and maybe the model doesn't want to answer unless it can be very very confident of the answer it produces. So it's sort of a "soft refusal".

> I've had such the opposite experience

Yeah, I've actually heard many other people swear by the GPTs / Codex. I wonder what factors make one "click" with a model and not with another.

> Codex is an ice man.

That might be because OpenAI hides the actual reasoning traces, showing just a summary (if I understood correctly).

Yes, I think? But I was talking more specifically about using the models via API in agents I develop, not for agentic coding. Though, thinking about it, I also don't click with the GPT models when I use them for coding (using Codex). They just seem "off" compared to Claude.

I quite like the GPT models when chatting with them (in fact, they're probably my favorites), but for agentic work I only had bad experiences with them.

They're incredibly slow (via official API or openrouter), but most of all they seem not to understand the instructions that I give them. I'm sure I'm _holding them wrong_, in the sense that I'm not tailoring my prompt for them, but most other models don't have problem with the exact same prompt.

Does anybody else have a similar experience?

As a user, I'm of course very excited about v7. As a developer of an app that _integrates_ TypeScript, however, I feel a bit uneasy seeing the API feature still marked as "not ready" on the roadmap.

On the other hand, I can understand leaving it as the last thing to do, after the foundation has set. Also, the TypeScript team has really done an amazing job all these years with backward compatibility, so that is extremely reassuring.

Maybe my uneasiness is just impatience to get to use the sped-up version in my app as well! :)