Comment by nextaccountic | Hacker News Reader

Comment by nextaccountic | Hacker News Reader

The best solution I've ever seen is this Rust library https://github.com/cornucopia-rs/cornucopia

You write plain SQL for you schema (just a schema.sql is enough) and plain SQL functions for your queries. Then it generates Rust types and Rust functions from from that. If you don't use Rust, maybe there's a library like that for your favorite language.

Optionally, pair it with https://github.com/bikeshedder/tusker or https://github.com/blainehansen/postgres_migrator (both are based off https://github.com/djrobstep/migra) to generate migrations by diffing your schema.sql files, and https://github.com/rust-db/refinery to perform those migrations.

Now, if you have simple crud needs, you should probably use https://postgrest.org/en/stable/ and not an ORM. There are packages like https://www.npmjs.com/package/@supabase/postgrest-js (for JS / typescript) and probably for other languages too.

If you insist on an ORM, the best of the bunch is prisma https://www.prisma.io/ - outside of the typescript/javascript ecosystem it has ports for some other languages (with varying degrees of completion), the one I know about is the Rust one https://prisma.brendonovich.dev/introduction

Cornucopia is neat. I wrote a similar library in Go [1] so I'm very interested in comparing design decisions.

The pros of the generated code per query approach:

- App code is coupled to query outputs and inputs (an API of sorts), not database tables. Therefore, you can refactor your DB without changing app code.

- Real SQL with the full breadth of DB features.

- Real type-checking with what the DB supports.

The cons:

- Type mapping is surprisingly hard to get right, especially with composite types and arrays and custom type converters. For example, a query might return multiple jsonb columns but the app code wants to parse them into different structs.

- Dynamic queries don't work with prepared statements. Prepared statements only support values, not identifiers or scalar SQL sub-queries, so the codegen layer needs a mechanism to template SQL. I haven't built this out yet but would like to.

[1]: https://github.com/jschaf/pggen

> Therefore, you can refactor your DB without changing app code.

Indeed, that's incredible! I tend to think that the actual layout of db tables a low level concern that is mainly driven by query performance and simplicity.

> - Type mapping is surprisingly hard to get right, especially with composite types and arrays and custom type converters. For example, a query might return multiple jsonb columns but the app code wants to parse them into different structs.

In Rust there's serde for converting json into structs (or enums if the shape of the json is complicated enough). Doesn't Go have a similar serialization/deserialization library?

> - Dynamic queries don't work with prepared statements. Prepared statements only support values, not identifiers or scalar SQL sub-queries, so the codegen layer needs a mechanism to template SQL. I haven't built this out yet but would like to.

In this case, what about stored procedures?

But, generally, instead of a template language for SQL, I would like to have a real compile-to-SQL higher level language, that could also do dynamic queries. The trouble with templating is that dynamic queries are hard to do in a type-safe manner, and it's hard to prevent generating invalid SQL when you have a bad template substitution (and then you get bad error messages from the db). There's a few languages like this do this https://github.com/ajnsit/languages-that-compile-to-sql but none fits the bill

> Doesn't Go have a similar serialization/deserialization library?

There isn't anything as flexible or widespread as serde.

> In this case, what about stored procedures?

That works but moves logic out of the query and into the DB. Keeping logic in the query is nice to keep server releases independent. The lifecycle of app (and query) code is easier to manage than DB migrations.

> The trouble with templating is that dynamic queries are hard to do in a type-safe manner.

Where-clauses and order-by-clauses are straightforward since the structure of those clauses doesn't affect the structure of the query results. Group-by and table clauses are troublesome since a query might work during codegen but fail at runtime.

> Where-clauses and order-by-clauses are straightforward since the structure of those clauses doesn't affect the structure of the query results. Group-by and table clauses are troublesome since a query might work during codegen but fail at runtime.

If you do this kind of analysis (to reject malformed dynamic queries before sending them to the db) I think one can argue you created your own programming language (a DSL but still)

Reply on news.ycombinator.com