HNHacker News
TopNewBestAskShowJobs

stephen

1,687 karma · joined January 19, 2008

stephen.haberman@gmail.com draconianoverlord.com (the domain name was available :-)) https://joist-orm.io/
submissionscomments
stephen··on Niri – A scrollable-tiling Wayland compositor
I want to use/try Niri, but have been staying on Hyprland from my safety blanket of Omarchy [1], and really liking its hyprscrolling plugin:

https://github.com/hyprwm/hyprland-plugins/tree/main/hyprscr...

Imo Hyprland should merge this hyprscrolling plugin into the main project, and just ship it as the default (only?) layout option -- it just scales to "more than 4 windows" so much better than either of Hyprland's master/dwindle layouts.

[1] I tried vanilla arch + archinstall + sway/niri/etc but really couldn't make it work from scratch, vs. the contrast of Omarchy which was "wow this all works" :shrug:

stephen··on We rewrote the Ghostty GTK application
I've been using Ghostty, and other GPU-based apps like Alacritty / WezTerm / Zed, because they're ofc better/faster...

Ironically they've all made my DX worse, by highlighting how terrible the nvidia drivers actually worked on both my old Regolith i3wm/compositor-less or new sway/wayland setup.

Like it's ridiculously terrible.

I've tried every magical env flag that Claude can think of, and 4 of the various 550/560/575/580 driver versions--they all suck at screensharing, or resume from sleep, or just out-right buginess/crashes/segfaults in the nvidia drivers.

It must have always been this bad, but I just didn't notice, b/c I never actually used my GPU? lol

stephen··on Show HN: I've been building an ERP for manufacturing for the last 3 years
I was wondering the same, about their backend domain model (or lack of it).

Fwiw in the TypeScript space, we built Joist (https://joist-orm.io/) to do exactly this.

Granted, we went with a Rails/ActiveRecord minimalist take on DDD instead of some of the more elaborate (overkill imo) implementations that are common i.e. in the .NET space.

stephen··on Microservices are a tax your startup probably can't afford
> Microservices are a design pattern for organisations as opposed > to technology ... breakout into multiple teams

I agree, but just saying "multiple teams" has led many eng directors to think "I have two squads now --> omg they cannot both be in the same monolith".

When both squads are 5 people each.

And the squads re-org (or "right size") every 9 months to re-prioritize on the latest features.

Five years go by, 7 team/re-org changes, all of which made sense, but thank god we didn't microservice on the 2nd/3rd/4th/5th/6th team boundaries. :grimmacing:

We should stay "stable, long-lived teams" -- like you need to have a team that exists with the same ownership and mandate for ~18 months to prove its a stable entity worth forming your architecture around.

stephen··on We fell out of love with Next.js and back in love with Ruby on Rails
Well, we're not the "go to" yet :-) but if you want an entity-based ORM that isn't just a query builder, Joist has several amazing features (no N+1s) and great ergonomics https://joist-orm.io/
stephen··on Show HN: Hatchet v1 – A task orchestration platform built on Postgres
Do queue operations (enqueue a job & mark this job as complete) happen in the same transaction as my business logic?

Imo that's the killer feature of database-based queues, because it dramatically simplifies reasoning about retries, i.e. "did my endpoint logic commit _and_ my background operation enqueue both atomically commit, or atomically fail"?

Same thing for performing jobs, if my worker's business logic commits, but the job later retries (b/c marking the job as committed is a separate transaction), then oof, that's annoying.

And I might as well be using SQS at that point.

stephen··on Ask HN: What are you working on? (March 2025)
Working on v2 of our n+1-proof/reactive TypeScript ORM, Joist (https://joist-orm.io/), that moves to using the new-ish postgres.js driver (instead of knex/node-pg), so that we can leverage postgres.js's statement pipelining within transactions.

I'm anticipating a really sweet perf increase (as shown by some proof-of-concepts), but now that everything is actually working on the v2 branch, I'm putting together benchmarks that show the benefit in practice.

Love to have anyone poke around/ask questions/hang out on discord.

stephen··on JavaScript Temporal is coming
Same! We've also been using the polyfill in production for a ~year or so.

We recently configured the node-pg driver to leave the postgres DATE / TIMESTAMP / TIMESTAMPTZ dates as "just strings" (instead of its default parsing to JS dates), and then we go straight from strings -> temporals at our ORM layer, without the jank/hop of it first being a JS Date.

...shoot, we need to get the Temporal out-of-the-box support updated in our Joist (https://github.com/joist-orm/joist-orm/) docs. :-/

But it's been great, having `DATE` in the db => `PlainDate` in the TS backend => `Date` in GraphQL API => `PlainDate` again in the TS/React FE. Very clean.

stephen··on Rails is better low code than low code
> don't want to build in a language w/o types

Yep; that's the only reason I don't use Rails.

I'm building "ActiveRecord but with type(script)" at https://joist-orm.io/ and have our own internal "kinda Rails w/GraphQL" going internally--so far we're at ~450 tables and still "feeling good" (no n+1s ever!), but definitely a long way to go before AirBNB-sized problems.

stephen··on Putting a full power search engine in Ecto
Using raw SQL directly is doable, but it means you're responsible for maintaining the business logic & validation rules of every single hand-written INSERT, UPDATE, and DELETE query in your codebase.

Personally I don't trust myself to remember to do that :-) hence preferring entity-based ORMs:

https://joist-orm.io/docs/modeling/why-entities

(That said, I definitely "know SQL" and use raw SQL queries for the ~5% of queries in a CRUD/SaaS app that are actually unique/non-boilerplate, instead of forcing them to go through an obtuse query builder DSL.)

stephen··on Modular Monoliths Are a Good Idea
I mean, of course they are a good idea, what we need is more examples of actually doing them in practice. :-)

I.e. quoting from the post:

- monolithic databases need to be broken up - Tables must be grouped by module and isolated from other modules - Tables must then be migrated to separate schemas - I am not aware of any tools that help detect such boundaries

Exactly.

For as much press as "modular monoliths" have gotten, breaking up a large codebase is cool/fine/whatever--breaking up a large domain model is imo the "killer app" of modular monoliths, and what we're missing (basically the Rails of modular monoliths).

stephen··on Finding and optimizing N+1 queries on a relational database
We built Joist, an ActiveRecord-style ORM in TypeScript, on top of Facebook's dataloader library, and haven't had a single N+1 in ~4 years:

https://joist-orm.io/docs/goals/avoiding-n-plus-1s

Any "query in a loop", whether a manually-written `for` loop, or an "emergent for loop" from 100 `after_update` lifecycle hooks firing at once, are always auto-batched, every time.

Joist is also likely why we actually like GraphQL, vs. it generally having a reputation for being a pita to implement (N+1s everywhere w/o tedious, manual batching).

stephen··on Ruby methods are colorless
Maybe its Stockholm syndrome after ~4-5 years of TypeScript, but I like knowing "this method call is going to do I/O somewhere" (that its red).

To the point where I consider "colorless functions" to be a leaky abstraction; i.e. I do a lot of ORM stuff, and "I'll just call author.getBooks().get(0) b/c that is a cheap, in-memory, synchronous collection access ... oh wait its actually a colorless SQL call that blocks (sometimes)" imo led to ~majority of ORM backlash/N+1s/etc.

Maybe my preference for "expressing IO in the type system" means in another ~4-5 years, I'll be a Haskell convert, or using Effect.ts to "fix Promise not being a true monad" but so far I feel like the JS Promise/async/await really is just fine.

stephen··on Eradicating N+1s: The Two-Phase Data Load and Render Pattern in Go
> So if you access a1.b then it will also fetch a2.b and > cache it if you loaded a2 within that same transaction.

Ah ha! I'd implemented the same trick in this older Java ORM:

https://github.com/stephenh/joist-java/blob/master/domain/sr...

It worked well!

stephen··on Eradicating N+1s: The Two-Phase Data Load and Render Pattern in Go
Definitely appreciate your "sometimes ORMs/sometime raw SQL" pragmatic stance. I agree!

> then if you join on both you now are loading A * B * C rows

I'm realizing it's not actually written up in our proper docs, but the "cross join lateral" trick is pretty neat:

https://joist-orm.io/blog/nextjs-sample-app#join-based-prelo...

This will return A number of (author) rows, where each author row as a `_b` column that is a JSON array of that author's books. And if you asked for C comment rows, then one of the "columns"/indices in each of those book arrays will itself be a nested array of the book's comments.

So all the data gets returned, but with no repetition/Cartesian product.

Hat tip to PostGraphile where I first saw the "cross join later" + "json_agg" technique used. Disclaimer we're not actually running this Joist feature in production yet--just haven't had time to run it through the test suite.

stephen··on Eradicating N+1s: The Two-Phase Data Load and Render Pattern in Go
I can assure you that brandur knows SQL. :-)

(I don't know him personally, but have been following his blog for years.)

What these "just write SQL" rants are missing is encapsulation--let's say you've got a business logic, like "if the account is disabled, render the account name as 'Foo (disabled)'".

You want to write this logic in your preferred backend language, Go/TS/C/etc.

This works fine, in the /account/X endpoint (just load the account, and apply the logic).

But now what about the /accounts/client:Y endpoint (load all accounts, all the logic for all of their accounts)

As time goes by, you end up having 10-20 endpoints that all "return some part of account" as their payload, and you want the same "Foo (disabled)" business logic.

Your options are:

1. Build a single, giant SQL statement that strings together every snippet of business logic applicable to this endpoint (bulk friendly b/c the db is doing all the cross-entity work w/joins)

2. Push the business logic down into the db layer (simple for this, just string concate with an if, but what about anything that is a loop? doable in SQL but tedious)

3. Use your language's abstractions, like functions, to organize the business logic (what Brandur is attempting to do).

Nearly everyone wants to do 3, because 1 doesn't scale as your business logic becomes more & more sophisticated (copy/pasting it around every endpoint's single-giant SQL statements, or pushing it down into the db as views/stored procedures).

stephen··on Eradicating N+1s: The Two-Phase Data Load and Render Pattern in Go
Besides pitching Joist, going through OP, I'm not following how verbose the nested examples would get, i.e. ProductLoadBundle is loading "products & widgets".

But what if I need "products & widgets & widget orders". And like sometimes I want "just products & widgets" (2 levels) and sometimes I want "products & widgets & widget orders (3 levels)"?

Would these be the same "ProductLoadBundle" with some conditionals to the Product.LoadBundle method? Would the result of those conditionals be seen in the type-safe as, i.e. sometimes the 3rd level is available, sometimes it is not?

Or would it be two separate bundles, a 2-level Product.LoadTwoLevelBundle and a 3-level Product.LoadWidgetsAndOrdersBundle, which has the pro of better type-safety, but a con of repeating the bundle boilerplate for each unique shape/tree of data your app needs to load.

My guess/hope is that it's the 2nd, if only because I assume brandur also values type-safety over verbosity.

It took me awhile to find again, but this OG scala library in theory handled these adhoc shapes (called "enrichments" in their readme) somewhat elegantly:

https://index.scala-lang.org/getclump/clump

Mea culpa another Joist pitch, but TypeScript makes this "give me an adhoc type based on the call-site specific tree of data" super-pleasant, i.e. Loaded<Product, "widgets"> vs. Loaded<Product, { widgets: "orders" }> gets you the two different 2-level vs. 3-level types.

...granted, you just have to accept non-Go/non-native levels of performance., but tradeoffs. :-)

stephen··on Eradicating N+1s: The Two-Phase Data Load and Render Pattern in Go
Ah, I look forward to every brandur post! :-)

If he can give up Go, we've got a TypeScript ORM that will de-N+1 basically everything* that is not a paginated/limit-offset query:

https://joist-orm.io/docs/goals/avoiding-n-plus-1s

This works even in adhoc loops, i.e. if you have a lifecycle hook** of "after an author changes, do x/y/z logic", and you update 100 authors, every SQL operation invoked by those ~100 individual hooks is auto-batched.

We've been running this in production for ~4 years at this point, and haven't had an N+1 since then (although we didn't initially support auto-batch find queries; that came later).

Of course kudos to dataloader.

*everything --> any queries our "find" API supports, which doesn't do aggregates, sums, havings, etc.

**lifecycle hooks --> yes, a blessing and a curse; we're always attempting to find better/higher-level abstractions for declaring the intent of business logic, than raw/imperative hooks.

stephen··on N+1 in Ruby on Rails
I worked in a Rails app in ~2018 & hunting down N+1s by sprinkling `includes` in just the right places was tedious, despite being a regular occurrence.

Since then I've been building Joist, which is written in TypeScript, and has dataloader integrated into every operation (even `find`s) to basically never N+1:

https://joist-orm.io/docs/goals/avoiding-n-plus-1s

stephen··on Hyprland Crash Course
> reading through the documentation and putting your distro together gives you way more insight into the inner workings which is invaluable for when it breaks.

I definitely feel that way about the core libraries/frameworks I'm building my applications/codebases on top of, so can relate!

But for my WM/desktop, I'm pretty happy to a) not know it's innermost workings, and b) just have it not break in the first place! :-)

I guess we can each have our own yak-shaving preferences. :-)

stephen··on Hyprland Crash Course
Love a tiling WM post! Reading the "twos days to get setup" reminds me of fighting a blank xmonad setup back in the day :-), and makes me all the more appreciative of Regolith Desktop, a super-minimal Ubuntu+i3wm (and sway) setup; everything just works, Zoom sharing, multi-window, etc. Recommended!
stephen··on Show HN: Hatchet – Open-source distributed task queue
Nice! Thanks for the reply!

> hosted infra is hard to bootstrap.

Ah yeah, that definitely makes sense...

> a lot of the core infrastructure needed for that service branches > into a lot of other products

Ah, I think I see what you mean--the goal isn't to be "just a job queue" in 2-5 years, it's to grow into a wider platform/ecosystem/etc.

Ngl I go back/forth between rooting for dev-founded VC companies like yourself, or Benjie, the guy behind graphile-worker, who is tip-toeing into being commercially-supported.

Like I want both to win (paydays all around! :-D), but the VC money just gives such a huge edge, of establishing a very high bar of polish / UX / docs / devrel / marketing, basically using loss-leading VC money for a bet that may/may not work out, that it's very hard for the independents to compete. I have honestly been hoping post-ZIRP would swing some advantage back to the Benjie's of the world, but looks like no/not yet.

...I say all of above ^ while myself working for a VC-backed prop-tech company...so, kinda the pot calling the kettle black. :-D

Good luck! The fairness/priority queues of Hatchet definitely seem novel, at least from what I'm used to, so will keep it bookmarked/in the tool chest.

stephen··on Show HN: Hatchet – Open-source distributed task queue
Wow, looks great! We currently happily use graphile-worker, and have two questions:

> full transactional enqueueing

Do you mean transactional within the same transaction as the application's own state?

My guess is no (from looking at the docs, where enqueuing in the SDK looks a lot like a wire call and not issuing a SQL command over our application's existing connection pool), and that you mean transactionality between steps within the Hatchet jobs...

I get that, but fwiw transactionality of "perform business logic against entities + job enqueue" (both for queuing the job itself, as well as work performed by workers) is the primary reason we're using a PG-based job queue, as then we avoid transactional outboxes for each queue/work step.

So, dunno, loosing that would be a big deal/kinda defeat the purpose (for us) of a PG-based queue.

2nd question, not to be a downer, but I'm just genuinely curious as a wanna-be dev infra/tooling engineer, but a) why take funding to build this (it seems bootstrappable? maybe that's naive), and b) why would YC keeping putting money into these "look really neat but ...surely?... will never be the 100x returns/billion dollar companies" dev infra startups? Or maybe I'm over-estimating the size of the return/exit necessary to make it worth their while.

stephen··on Dada, an experimental new programming language
If you're cloning parts of TypeScript, please bring along mapped & conditional types!

Feel free to experiment on the syntax, but the concept is amazing, especially if you're planning on being dynamic-ish.

stephen··on StyleX – Meta's styling library
Nice! We copied StyleX's "type-safe extensions" in Truss [1] so things like `<MyButton xss={Css.mt5.$} />` are allowed (setting margin is fine) while disallowing `<MyButton xss={Css.dg.$} />` (anything "not margin") that would mess up the components internal impl details with a compile error.

That said, we don't actually use the feature that much, vs. higher-level logical props like `<MyButton compact />`.

I know we're supposed to use build-time CSS-in-JS these days, but afaiu they don't support the rare-but-handy "just spread together ~4-5 different object literals from ~random different conditionals + props", i.e. intermixing styles some inside the component + outside the component, which emotion handles really well.

Basically this [2]. StyleX says it does "cross-file styles"...but can it support that? I kinda assume not, but I'm not sure.

[1]: https://github.com/homebound-team/truss#xstyles--xss-extensi...

[2]: https://github.com/homebound-team/beam/blob/main/src/compone...

stephen··on A Node, TypeScript, TS-Node and ESM experience that works
Fwiw I appreciate your effort! That sounds really frustrating.

I agree the recent bun/tsx/esbuild (but bun especially) has shown the node CJS/ESM fiasco was a bit of an emperor-wearing-no-clothes moment, where I think us every-day JS programmers just trusted the node devs at their word, that CJS/ESM had to be this painful...

But now, seeing that it doesn't have to be that way, it's like wait a sec...the last ~5 years of pain could have been avoided with some more pragmatic choices? Oof.

stephen··on Yarn 4.0
Kinda surprised by all the "stuck on Yarn v1" comments so far...

We went through the same "~2015 everyone uses yarn", "~2020 everyone goes back to npm" cycle, but in the last ~year or so are back on yarn v3 for...reasons that I forget (oh yeah, the ability to download multiple platform libs for when you're running in docker [1])...but :shrug: it's been great.

My only complaint is that `yarn dedupe` should be automatic (as it was in yarn v1), or at least via a flag. :-D

But otherwise yarn v3 (with the node_modules linker) has been great, and look forward to using v4.

[1] here's the "afaict npm can't do this?" snippet we use in `.yarnrc.yml`: https://gist.github.com/stephenh/43c7f15b97b73baf821f2a22c62...

stephen··on The Performance Cost of Node.js and GraphQL
One of Bun's assertions is that their AsyncLocalStorage is significantly faster than Node's, b/c it purposefully only implements the subset of async_hooks that can be implemented w/o affecting performance:

https://github.com/oven-sh/bun/pull/3089

I haven't had a chance to test it out yet, but perhaps the OP could add Bun to the mix? To compare it's promise overhead vs. Node's promise overhead.

Also there is an issue in Node about performing the same optimization:

https://github.com/nodejs/node/issues/46265

stephen··on Bun v0.8
No. :-)

That was certainly the promise/hype of Rust ~2-3 years ago, that it was going to become so ergonomic that even "boring line-of-business applications" (i.e. JS backends) could be written in Rust, by everyday programmers, without any slow down in delivery/velocity due to the language complexity.

But, from what I've seen, that's not played out, and instead the community is still on the look out for the killer "systems-language performance, but scripting-language ergonomics" language.

stephen··on Bun v0.8
> Bun’s APIs try really hard to make the obvious & default way > also the fast way.

Nice! That makes a lot of sense, and look forward to trying them.

Fwiw I sometimes worry about the slippery slope to infra that exists on the JS side, i.e. I work a lot in a GraphQL backend, and even if Bun gave us (or really our framework, so fastify/mercurius) a super-optimized way of getting the raw bytes off the wire, Mercurius is still doing GraphQL parsing, validation, routing, response building in JS land.

Granted, I want to keep my application's business logic in TS, but naively it seems tempting to push as much of the "web framework" / "graphql framework" as possible into the native side of things, where as I think historically Node/etc API have stopped at the "here's the raw HTTP request off the wire".

> I’ve thought a little about this.

Sweet! That's awesome just to hear that it's crossed your mind. Agreed it would be a moonshot. And, yeah, I'm perfectly happy leaning into JITs and good APIs.

Thanks for the reply!

← PreviousPage 2 of 14Next →