rgbrgb

these were not average employees - top decile engineers i've worked with, shipping quickly and rigorously to millions of global users

very cool! i'd been thinking about this idea and found vega-lite to be an interesting take as well [0].

i'll compare and look at folding this into setoku for app generation [1]. right now apps are just html blobs your claude authors + a mechanism for populating them with live data. definitely hard to audit but very flexible for operators to claude together internal apps. anyway, the charts look decent but really depend on the model that's making them and don't really follow any sort of style guide (example: https://demo.setoku.com/apps/a7a1240ae0bc202c5eefa1cc). Your lib could bring some consistency and make global styling possible.

[0]: https://vega.github.io/vega-lite/

[1]: https://setoku.com

interesting. the less "research lab" version i've seen of this is https://polsia.com

comparatively, I really appreciate the transparency here. clearly it's a little too early for this (quickly glanced at the P&L's, correct me if I"m wrong), but someone has to run the experiment and figure out when it's ready for prime time.

> semi-skilled workers

fwiw the h1-b workers I encountered in bigtech were totally on-par with their US citizen equivalents in both coding and english communication. many teams were about 50/50 in my office (SF, non-HQ).

my current talent/placement company works mostly with AI startups who are usually too early to support visa requirements [0]. being able to support the lead-time and planning for international applicants is a huge advantage later stage companies have in terms of being able to scoop up talent at reasonable prices. big loss for the local ecosystems if these jobs move across the border imo.

[0]: https://www.hedgy.works

why shadcn and not bootstrap? shadcn is like a cool weird approach to a ui kit but bootstrap got us a date picker iirc.

kind of rhymes with the reports of LLM's watching open source PRs and instantly exploiting defects. security by obscurity is so back

agree and underscores the idea that you should keep your setups portable between model providers by using external memory systems and connector gateways

tbh I use 1 custom skill (workflow for completing a github issue and opening dev server/PR) and have it checked-in to my main project repo.

For me, it's like a dev script basically and gets that level of care. I don't need an eval... I'm the only user and I use it like 5 times a day.

There’s kind of an interesting thread here about open source, which is usually thought of as more secure because of more eyes on the code, actually being less secure because an accidental merge can be exploited instantly with LLM’s monitoring. Is there a lot more security value in obscurity than before?

at Hedgy we built an MCP to give our agents read-only access to cross-system data + context about what it means [0]. daily it helps me debug customer flows and think through features and priority.

the advantage of MCP is that the tool description names what data it has so the agents know to just use it if they want that kind of data. as new data is added, everyone’s tool description updates. with a REST api or cli I suppose we could update a checked in AGENTS.md to ship it to engineers but idk how id get that to colleagues using the it from their claude.ai or co-work.

[0]: setoku.com

if you're using claude code, i find you can instruct it to watch the pr and iterate on ci failures. I put this in an /issue skill that completes GitHub issues in the way I like then PR's or boots dev with QA instructions if it needs a human touch/eye. to drive this, I have a higher level bash script that makes a fresh worktree off of origin/main named after the issue and spawns a claude session running /issue.

i'm not at the point where I'd go totally hands-off with code review / QA, so this setup is the right balance of automation for my current read on agent capability.

or maybe more like a restaurant than a kitchen... with software development (like all product development) you need to bake in the user feedback loop

like to torture the metaphors, i think we already have the factory (coding agents you can ask to manufacture software to a spec) but it's the whole process including QA, shipping, listening to users and iterating that has to happen over time (unless you have an accurate world simulation so you know exactly how your product will be received... and I know there are people working on that too but I'm skeptical)

this is sick and the video infra sounds interesting! as a fellow san diegan i really get a kick out of seeing everyone who hit the 14 stair behind my high school https://skatanica.com/spots/5eyevxzo35tcou1n6doxkw2lx?map=-9...

why no hometown hero tony hawk?

this is cool! incidentally we do a similar motion internally for members on https://www.hedgy.works

how did you think about evaluating quality when designing your ranker / fit scoring?

I’ve been curious about trying the Supabase type of RLS model for an app but haven’t yet. I like the idea of the db defined api like postgraphile and hasura but had trouble with testing and ergonomics. This looks pretty nice though, it’s cool that you’re thinking about ergonomics for working with coding agents. I feel like a hard thing with that rn is that everyone is kind of figuring out different workflows for AI coding right now so it’s hard to optimize DX. Can you talk a little about your personal agent workflow and how this fits it?

just tested (zai-org/GLM-5.3-Flash via together.ai) against latest DeepSeek-V4-Flash for a very specific task and thought i'd report here...

- price: DS4 wins... $0.0235 vs $0.0242 for ten tasks

- latency: GLM wins... 108s total against 154s

this is for a personal use-case where i'm detecting ads in a written transcript. sticking with ds4-flash for now since latency is not a critical factor

for knowledge writes in setoku, any agent that does queries can propose knowledge or edits. edits are flagged for admin/curator review. agents can query the pending and confirmed knowledge but basically have a flag that tells them what's been "blessed".

i like your conceptual focus on not pairing it with data. for me, it's just a practical thing: i want one tool i can have a colleague (or my wife) install that adds all the context they'll need (and then one spot I can curate and govern that context).

nice :)

> love to hear how you did it

this is ours we built for Hedgy https://setoku.com

our approach was to build a data lake that sucks company data into clickhouse and staple that to a knowledge store. this way the brain has a stream of live facts and builds knowledge around it. we gave up on trying to make the knowledge store human-readable -- i totally think there could be something there, but for now we just care about enhancing the agent you're using. it makes my claude code very good at debugging and gives everyone a way to vibecode dashboards and small internal tools with real data.

i also run a personal instance for my wife and I that sucks in monarch money and gmail. mostly use it to chat through big money moves.

this looks great! what are you using it for? i like the idea of being able to use one of these (sandboxed) within a larger program kind of like how I use LLM's to do small tasks within my apps now but with a few tools (web search). my current way of doing that is like building a mini-harness with a couple tools within the app, but something more drop-in would be better obviously.

yep, i found this page after doing a misclick

i love postgresql but once we added ai-generated dashboard to our homegrown analytics tool [0] some of the crazy (amazing) dashboards that the ops team was building began accumulating horrendously slow db queries. I considered dynamically adding indexes or alerting around postgres slow queries but also quickly prototyped mirroring the postgres data in clickhouse. At first could not believe how fast clickhouse was on arbitrary analytics queries - like 60s to 0.5s for some gnarly queries. truly amazing software that just works without any tuning for this kind of exploratory analytics workload.

so yes, i'm still a postgres maximalist (worker queues still in pg [1]) but (especially in the age of quick LLM prototypes) it's always worth measuring the more purpose-built approach.

[0]: https://setoku.com

[1]: https://worker.graphile.org

CI with regular e2e tests usually gets pretty expensive and slow. How does cost compare to regular playwright tests?

Homepage is basically down, pointing to status page: https://www.namecheap.com/status-updates/namecheap-com-and-h...

Found this bc a friend texted that all his sites are down.

agree there are a lot of these but they're all pretty simple (including mine [0]) so I think building your own and playing around with architecture is useful and fun.

[0]: https://setoku.com

for one, the mcp-server architecture makes it usable from claude.ai and other surfaces where you have mcp but no filesystem. there are claude-specific workarounds (workspaces) but you lose portability across systems.

that's a good idea. how might you test this? could also include a codex memory test.

I'm guessing having a portable memory that's comparable with first party memory is the goal.

Im not OP but as someone who did one of these recently, I think about it kind of like sharing an essay. I had an idea, I clauded a working thing, used and refined until it was useful for my team then wanted to share the kernel that developed. OP is not claiming it’s like a panacea or a masterpiece and idk I appreciate the thoughts it’s making me think.

My other more general thought here is that the nature of LLMs is so generic and context specific that just learning about a workflow can unlock a ton of potential. This is evident in how the null state of every chat UI has a bunch of examples to teach you to use it. MCP’s are a portable way to share a workflow. To get concrete: I was pulling a lot of business data sources into my Claude code sessions to great effect but it was a force multiplier for my team to figure out an MCP version that would tell their Claude’s how to do the same. None of it was deeply technical engineering but it was immediately useful and most of the team using it are not engineers.

I feed live monarch money data to fable to get help with:

1. high level portfolio composition stuff

2. when to rebalance what where

3. what to sell

4. thinking through money moves (e.g. real estate purchase, remodeling, company sale, angel investments)

5. one off transaction questions (e.g. how many times have I used the ATM with card X this month?)

Don't always agree with the answers, but facts are right.

In the past few years I've been using a "dirty db" approach to testing where I run parallel integration tests against a single postgres-based backend without any cleanup between tests. Every test hits the same db with unique ID's. The constraint is that you can't assert exact counts, you need to assert that specific ID's exist. With this setup, the db just gets setup once and all the tests can run in parallel. Integration points are where I've seen the most breakage in prod, so I like to test with a real db. Parallelism makes it fast and incidentally this approach sometimes catches racy interactions that otherwise might only see with concurrent prod requests (heisenbugs in waiting). Many tests hitting the API's in parallel ends up being a better simulation of prod behavior.

Very cool. What model is used for the judge? We use setoku for Claude-built internal tooling but the design doesn’t allow writes so that there’s no inference cost on the server. Having the judge check for danger is a neat design.