HNHacker News
TopNewBestAskShowJobs

levkk

2,570 karma · joined February 9, 2022

Sharding Postgres. <https://pgdog.dev>

lev@pgdog.dev

submissionscomments
levkk··on PgDog is funded and coming to a database near you
The docker compose example is just a demo. I don't know anyone who runs Postgres with docker compose / swarm in prod :) But yes, happy to add volumes so it seems more real.
levkk··on PgDog is funded and coming to a database near you
We do, just buried deep in our blog: https://pgdog.dev/blog/pgdog-vs-citus

The same old processes vs. threads debate, plus having the ability to scale the coordinator past a single machine. So, if you're OLTP, definitely consider PgDog. OLAP - Citus still wins because of its advanced query engine. We'll get there.

levkk··on PgDog is funded and coming to a database near you
Depends. Only pooling, very little. Load balancing/sharding needs to parse queries, so a bit more. Could go up to a GB per pod, sometimes more if you have a lot of unique SQL queries (unique by text, not by parameters). We cache query ASTs to avoid parsing them on each request - that's the bulk of memory usage.
levkk··on PgDog is funded and coming to a database near you
Old benchmark, but still good: https://pgdog.dev/blog/pgbouncer-vs-pgdog
levkk··on PgDog is funded and coming to a database near you
We should add it to brew/apt/etc for sure. Also, we could add it to crates.io so you could do something like `cargo install pgdog`. Distribution, distribution, distribution.
levkk··on PgDog is funded and coming to a database near you
Sorry, out walking the dog (not a pun). I'll post more details in a few.
levkk··on PgDog is funded and coming to a database near you
Getting there! Cross-shard writes do because of 2pc. Reads are eventually consistent.
levkk··on PgDog is funded and coming to a database near you
Crap! Missed opportunity.
levkk··on PgDog is funded and coming to a database near you
Not quite. The performance gain is to bring those features to Postgres!

Edit:

Performance gains are from having the ability to load balance reads (horizontal scaling for read queries) and scale out writes (with sharding). Once instance bottleneck in Postgres has many faces:

1. Behind schedule vacuums because of too many dead tuples (too many writes)

2. The WALWriter is single-threaded and IO-bound - Postgres can only do about 200-300MB/sec in writes per instance (real prod numbers on EC2 with NVMes and ZFS, basically best case scenario).

3. Bulkheading: single primary is a single point of failure. With 12 primaries, if one fails, 91% of your customers don't notice.

The list goes on. Rust is just a side effect. We love it because it's fast and correct - the perfect match for a database product.

levkk··on pg_durable: Microsoft open sources in-database durable execution
2026 is the year of the Postgres queue! (DBOS[0], pgQue[1]) It's awesome that the community is contributing this and giving us the option to use it.

As an ex-app engineer though, I kind of prefer my queue logic to be in code, in Git, but maybe with the right tooling, you can change my mind. :)

[0]: https://www.dbos.dev/

[1]: https://github.com/NikolayS/pgque

levkk··on SQLite is all you need for durable workflows
I don't understand this obsession with SQLite for real, production apps. SQLite is an embedded database, completely unsuitable for managing concurrency. This is what database _servers_ are for, e.g., Postgres, MySQL, etc. Their entire job is to allow you to modify data from multiple processes, on different machines, at the same time.

This is a foundational principle of computer science. It seems to me that the "SQLite for everything" crowd is a little bit inexperienced.

levkk··on Uber president says AI spending is getting 'harder to justify'
It wouldn't, at least not directly. That's why it wasn't done pre-AI.
levkk··on We're testing new ad formats in Search and expanding our Direct Offers pilot
I'm getting to a point in my life (age) where I want to put my money into things I care about. Netflix? 25$/mo, 3 hours a day. Search 10$/mo? 8 hours a day. Easy.
levkk··on Incident Report: May 19, 2026 – GCP Account Suspension
DigitalOcean. Seriously. They have been around a long long time and built a lot of the core infrastructure you rely on every day (e.g. Ceph).
levkk··on Incident with Actions
I'm using GitHub for my business and so do millions more. Might be time to prioritize paying customers, historically popular open source repos and PRs created by known human actors. Agents can wait, humans have much less patience.
levkk··on Barman – Backup and Recovery Manager for PostgreSQL
Last time I checked, Barman didn't support backups to S3. That's why (for us) pgBackRest was such a big deal: it could offload full and incremental backups to a basically limitless and reliable medium.

I think (and I'm probably wrong now) that Barman only could push backups to another Linux machine (e.g., EC2 box), so you had to worry about your backup system _on top_ of the main DB.

So I'm really hoping someone will pickup maintaining pgBackRest.

levkk··on Does Postgres Scale?
Shameless plug[0].

[0] https://pgdog.dev

levkk··on Zed 1.0
This is just a language server problem. I'm sure you can configure whatever language server PHP is using to disable specific warnings, etc.
levkk··on Pgrx: Build Postgres Extensions with Rust
Amazing project that spawned entire companies. We used it to build postgresml[0] and most Postgres extensions are built on top of it these days.

[0] https://github.com/postgresml/postgresml

levkk··on Framework Laptop 13 Pro
On top of it, intel chips are not competitive with apple silicon. Why buy a laptop that's 30% slower and uses more energy for the same price?
levkk··on Meta and TikTok let harmful content rise to drove engagement, say whistleblowers
I'm paying for Netflix to do that as a feature. Instagram uses that to drive engagement to sell ads. Disabling personalized content on Netflix is a revenue-neutral choice. On Instagram, that would mean their ad revenue takes a huge dive. Apples aren't oranges.
levkk··on The dead Internet is not a theory anymore
Lobsters is like that, basically a ghost town compared to Reddit. If you block engagement, you will succeed.
levkk··on Don't post generated/AI-edited comments. HN is for conversation between humans.
It's not clear to me how this is verifiable without constant hardware supervision. Even that'll get cracked, just like DVD encryption back in the day.

You almost need dedicated hardware that can't run any other software except a mechanical keyboard and make it communicate over an analog medium - something terribly expensive and inconvenient for AI farms to duplicate.

levkk··on Claude Code wiped our production database with a Terraform command
As the tool gets better, people trust it more. It's like Tesla's self-driving: "almost" works, and that's good enough for people to take their hands off the wheel, for better or for worse.

The "almost" part of automation is the issue + the marketing attached to it of course, to make it a product people want to buy. This is the expected outcome and is already priced in.

levkk··on Supertoast tables
All queries run inside transactions, and a slow lane like S3 will cause delays, which will in turn block vacuum and cause more problems than it will solve. Most deployments of Postgres (e.g., RDS) won't let you install custom extensions either, although they do have their own S3 extension (which I wouldn't recommend you use).

The right place to manage this is either in the app or in a proxy, before the data touches Postgres.

levkk··on Better JIT for Postgres
See prepared statements.
levkk··on What AI coding costs you
Models don't learn. They retrain them periodically, but junior engineers learn much faster and constantly improve. If you stop learning, you will only be as good as the model.

I've been coding (software engineering, I guess) for close to 15 years. The models skill set is a comfortable L1 (intern), pushing L2 (junior). They are getting better, but at a snail pace compared to a human learning the same thing.

levkk··on Show HN: PgDog – Scale Postgres without changing the app
Yeah! Take a look at our Docker demo in GitHub: https://github.com/pgdogdev/pgdog
levkk··on Show HN: PgDog – Scale Postgres without changing the app
You can I believe. We only support BIGINT, VARCHAR and UUID for sharding, but all other data types are completely fine for passthrough, i.e. to be included and used in your queries.
levkk··on Show HN: PgDog – Scale Postgres without changing the app
General statement about adoption. Last time we made a Show HN (9 months ago), it was a POC, running on my local. Now we're used in production by some pretty big companies, which is exciting!
← PreviousPage 3 of 10Next →