levkk

From the strictest CAP theorem definition, that's correct, it is not. But, it's pretty close. I know that in the database world, that's not a good answer, but in practice, it will deliver the vast majority of messages, so maybe that's good enough? We'll see.

We show that it's possible to come close without breaking the DB or the app, but I suspect, it's not quite yet at the level you'd expect from a _durable_ work queue, e.g., Kafka. Not going to replace that one anytime soon.

Very cool. At first, I thought it was yet another LLM-generated demo, but no: original work of art. Super cool. Transpiling into C does seem easier than LLVM IR, and letting GCC optimize seems like this might actually work.

Excited to see the compiler implementation when it's out -- a lot to learn from.

I cancelled Claude. The harness is kinda broken, GPT 5.5 is good, and GLM 5.2/Deepseek is good too (with pi, especially). Just not worth the trouble. And I'm not going to pay two subscriptions.

CRUD apps don't usually delete in bulk. It's also hard to structure partitions in a way that doesn't wipe out months of important business data -- this is why teams often ETL their DB into Snowflake/ClickHouse and only then drop partitions. That makes it hard for the app to use that data again.

The better approach is either to change your storage engine (e.g. OrioleDB is working on adding the undo log to Pg), or to shard which distributes the vacuum load across multiple servers.

You can have multiple. All sharding is config-based, so no real-time synchronization is required.

This just checks if the package is installed, not if the installed version is infected. Presumably, if you (me...) haven't run `yay -Syu` in a while (months), we're fine, right? ...Right?

Goddamit, don't make me reinstall Arch, took me a week last time.

Update: archinstall rocks, back in business after like 15min.

I didn't realize there are _any_ managed providers of PgDog out there...do tell!

Awesome, glad it worked!

We also do that! But it's not well documented at the moment.

Thanks! Glad we made it relatively easy to migrate!

For sure. They should be used for "metadata"-style tables only. High throuput writes should be direct-to-shard.

Yeah good callout. We'll add rendezvous soon enough. Until then, being compatible with Postgres partitions has been advantageous -- while we build everything out, people were able to migrate to PgDog for the query routing layer while doing the resharding in Postgres.

Adding a sharding function in our architecture is relatively straightforward. We also support plugins which can control the flow (and direction) for queries, so our users can add their own (and they do!).

Not yet, but actively working on this as we speak.

fwiw, we support cross-shard transactions. They are not magic though, just good old 2pc and a bit of coordination.

Thanks!

This reminds me of college. We had to cite our own papers from prior semesters or risk getting kicked out for plagiarism. I don't miss those days :)

Thanks! Yup...to be expected. If you know, you know, and have the scars to prove it :)

Yup!

Two big ones:

1. Control plane to manage multi-node deployments; "works out of the box" experience to make PgDog easy to deploy and use

2. QoS (quality of service): automatically block bad queries from taking down the database

Last but not least, you get SLA-backed support from us (up to P0).

New features are broken down into two categories:

1. Sharding / running Postgres at scale: always open source.

2. Infra management / making it easy to run PgDog at scale: enterprise.

Good thing we support HA as well: https://docs.pgdog.dev/features/load-balancer/

Load balancer with health checks and failover, works out of the box. :) Battle-tested at this point too, so could be worth a look.

I see you met Sage, our newest founding engineer :) If you're not having fun at your job...

In all seriousness, we review every single line of code that goes in and only people who work for PgDog Inc are allowed to merge.

Yup. We support schema-based sharding: https://docs.pgdog.dev/configuration/pgdog.toml/sharded_sche...

Not the place and not the time, but we are building an enterprise edition that "just works" out of the box. Not saying that the open source experience cannot be better - it always can and we'll keep improving. What you've experienced is definitely a known issue with our specific implementation of passthrough auth. Scram made things a bit harder, since we can't validate user's passwords at login time anymore (that's what makes scram secure fwiw).

We'll get there.

Always is. Marketing is not our strong suit (only engineers here). We'll get better at it.

> 8 smaller boxes handling ~500GB each and then one medium box for the proxy?

That's exactly right. Get in touch (lev@pgdog.dev), happy to help or at the very least tell you what current works (or doesn't) so you know what your options are.

This is not an extension, it's a proxy! Very different. You can deploy it anywhere already without having to wait for upstreaming or your cloud provider adding support for it. It's one of the two reasons why we built it this way, the other being performance (it's much faster to do this in the proxy than inside Postgres).

Yes, except it doesn't have any cross-dependencies on the same volume, so the uptime here should be higher.

OLAP means different things to different people. For us, it's just making sure your admin dashboard keeps working basically:

  SELECT tenant_id, COUNT(clicks)
  FROM users
  GROUP BY tenant_id
  ORDER BY 2 DESC
  LIMIT 25;

Performance is a side effect - definitely needed and we'll do everything we can, but we are not competing with ClickHouse or Snowflake - just trying to make sharded Postgres work with your app.

Two schools of thought:

1. Let it crash. Increase the RAM, try again.

2. Page to disk (swap), make it slow but ultimately work.

Both have their trade-offs. There is no free lunch here.

The docker compose example is just a demo. I don't know anyone who runs Postgres with docker compose / swarm in prod :) But yes, happy to add volumes so it seems more real.