HNHacker News
TopNewBestAskShowJobs

levkk

2,571 karma · joined February 9, 2022

Sharding Postgres. <https://pgdog.dev>

lev@pgdog.dev

submissionscomments
levkk··on Replacing Protobuf with Rust
Thank you!
levkk··on Static Allocation with Zig
That makes sense. For example, your redis instance will have fixed RAM, so might as well pre-allocate it at boot and avoid fragmentation.

Memcached works similarly (slabs of fixed size), except they are not pre-allocated.

If you're sharing hardware with multiple services, e.g. web, database, cache, the kind of performance this is targeting isn't a priority.

levkk··on The RAM shortage comes for us all
Not a bad time for the secondary market to be created. We keep buying everything new, when the old stuff works just as well. There is a ton of e-waste. The enthusiast market can benefit, while the enterprise market can just eat the cost.

Also, a great incentive to start writing efficient software. Does Chrome really need 5GB to run a few tabs?

levkk··on It’s time to free JavaScript (2024)
That's why courts don't take hypothetical cases. Someone has to be injured to demonstrate actual harm.

Are there any examples of Oracle using their JavaScript trademark to sue anyone? If they did, that petition would have merit.

Unless Demo was, this feels like a marketing project. And it's working, too, so kudos.

levkk··on The fate of "small" open source
Several issues:

1. Reducing dependencies is a wrong success metric. You just end up doing more work yourself, except you can't be an expert in everything, so your code is often strictly worse.

2. Regenerating the same solutions with a probabilistic machine will produce bugs a certain percentage of the time. Dependencies are always the same code (when versioned).

3. Cognitive overhead for human review is higher with LLM-generated libs, for no additional benefit.

levkk··on How to Obsessively Tune WezTerm
Please show me where. It only works for tabs that have a running process.
levkk··on How to Obsessively Tune WezTerm
The only thing I want from WezTerm is a warning when I'm about to close a tab. The tabs are too small and the x is easy to hit by accident... thousand lines of bash history and temp env vars gone in an instant.

Otherwise, it's awesome and my default terminal everywhere.

levkk··on Google flags Immich sites as dangerous
The Postgres query parameters one is funny. 65k parameters is not enough for you?!
levkk··on Replacing a $3000/mo Heroku bill with a $55/mo server
Yeah, no way. As soon as you hit swap, _most_ apps are going to have a bad, bad time. This is well known, so much so that all EC2 instances in AWS disable it by default. Sure, they want to sell you more RAM, but it's also just true that swap doesn't work for today's expectations.

Maybe back in the 90s, it was okay to wait 2-3 seconds for a button click, but today we just assume the thing is dead and reboot.

levkk··on Incident with Webhooks
Switched to email notifications and disabled them in GitHub, mainly because of this. Huge quality of life improvement.
levkk··on Show HN: I built a web framework in C
That's awesome. With macros, you can go far and most modern web frameworks use whatever complex tools their language allows (like metaprogramming in Rails).

Mad props for building this. It's hard and it's fun!

As to other comments in the thread about the "why": why not. For the love of the craft.

levkk··on Zig feels more practical than Rust for real-world CLI tools
Definitely not. Arc is for immutable (or sync, e.g. atomics, mutexes) data, while borrow checker protects against concurrent mutations. I think you meant Arc<Mutex<T>> everywhere, but that code smells immediately and seasoned Rust devs don't do that.
levkk··on KDE launches its own distribution
Sure except Wayland doesn't work with nvidia so...
levkk··on KDE launches its own distribution
Yup, same, using X, everything mostly works. Wayland - not so much.
levkk··on Two-Phase Commit
It's good to have some guarantee that a write is atomic and won't be left in a half-written state. I think that's strictly worse even if it's only "eventually" consistent. "Eventual" means within a couple milliseconds, really.

> you can redesign your app/system

You generally don't want to do that if you can avoid it. Also, easy to make a mistake there while 2pc is pretty deterministic.

levkk··on Amazon must face US nationwide class action over third-party sales
That can easily be perceived as violating the ToS and get you kicked off the marketplace. Big risk to take if that's where you're making most of your $.
levkk··on Zfsbackrest: Pgbackrest style encrypted backups for ZFS filesystems
Does that work with S3, etc.? I don't remember them allowing partial file uploads.
levkk··on Zfsbackrest: Pgbackrest style encrypted backups for ZFS filesystems
No. Postgres data files are 1Gb each. When you change just one byte in a table, a whole 1Gb file gets updated (just 1 byte change, effectively). Your file-based backup tool now has to upload 1Gb of data to save 1 byte of actual changes.
levkk··on Bear is now source-available
Gotta love these are zero-indexed. Written by an engineer for engineers.
levkk··on Zfsbackrest: Pgbackrest style encrypted backups for ZFS filesystems
Finally! Been looking for this a long time. File-based backups for large Pg databases are not very scalable (even incremental), having this in my toolkit would be amazing.
levkk··on "This question has been retired"
Competition. Go build a better product.
levkk··on PG Slot Notify: Monitor Postgres Slot Growth in Slack
Sounds like we both got hit with this one :)
levkk··on We made Postgres writes faster, but it broke replication
My last big co, we had a team of 10 who's entire job was to sync data from Postgres into Elastic. It would take weeks and fallover regularly due to traffic.

If we could have a DB that could do search and be a store of record, it would be amazing.

levkk··on Visualize and debug Rust programs with a new lens
I want to use this right now, but two issues:

- Bash script from Internet requiring sudo, no way

- VSCode plugin? I don't use VSCode. I'm not switching from Zed (literally built in Rust for Rust development)

Help me out, what can I do to try this?

levkk··on Show HN: PgDog – Shard Postgres without extensions
That's good to know. Don't hesitate to reach out if you ever want to chat about this stuff. My email: lev@pgdog.dev
levkk··on Show HN: PgDog – Shard Postgres without extensions
Thanks!
levkk··on Show HN: PgDog – Shard Postgres without extensions
Interesting. I've been thinking about it as well. I wrote some code to prototype it here: https://github.com/pgdogdev/pgdog/blob/main/pgdog/src/backen...
levkk··on Show HN: PgDog – Shard Postgres without extensions
That's a good question. To echo the comment above/below, we would have to define what a "direct connection" is.

In my interpretation (and as the founder, I think that matters because I would be the only one who would have any enforcement interest) that means TCP. AGPL says "over network" and that means TCP/UDP. To me at least.

So if you're running a SaaS, the TCP connection is broken by the app layer and all the business logic in between. Your code and proprietary products are safe.

And just to restate my previous point: PgDog is a company. If you want to use the product, we'll make it work.

levkk··on Show HN: PgDog – Shard Postgres without extensions
Definitely. I wouldn't be such a big fan of Postgres if this wasn't the case.

That being said, if sharding was natively supported and worked well, they would of done it ages ago.

All massive Pg deployments today are so because we just don't have another option.

But nobody looks at a 10TB table and doesn't worry about what tomorrow will look like.

levkk··on Show HN: PgDog – Shard Postgres without extensions
From an infra management perspective, a multi-TB table is not fun to maintain. If you can comfortably shard before you get there, you'll be happy you did. Caveat being, sharding has to be ergonomic and not get in the way.
← PreviousPage 5 of 10Next →