HNHacker News
TopNewBestAskShowJobs

eatonphil

24,412 karma · joined December 27, 2014

theconsensus.dev
submissionscomments
eatonphil··on A header-only C vector database library
As data stores go go this is basically in memory only. The save and load process is manually triggered by the user and the save process isn't crash safe nor does it do any integrity checks.

I also don't think it has any indexes either? So search performance is a function of the number of entries.

eatonphil··on Canada has one city with good weather and it's too tiny
Expected it to be Vancouver but it's Victoria, a three hour drive south from Vancouver Google tells me.
eatonphil··on Show HN: VillageSQL – A Fork of MySQL with Extensions
It's great to see paths forward for the MySQL community! Exciting to see this become public.

Does the GPL2 license here mean that anyone writing extensions for VillageSQL must license extensions as GPL2? If so that's still quite far from the Postgres ecosystem where plenty of companies are built largely around being proprietary extensions.

eatonphil··on Postgres Postmaster does not scale
That makes sense, thanks.
eatonphil··on Postgres Postmaster does not scale
The article addresses this, sort of. I don't understand how you can run multiple postmasters.

> Most online resources chalk this up to connection churn, citing fork rates and the pid-per-backend yada, yada. This is all true but in my opinion misses the forest from the trees. The real bottleneck is the single-threaded main loop in the postmaster. Every operation requiring postmaster involvement is pulling from a fixed pool, the size of a single CPU core. A rudimentary experiment shows that we can linearly increase connection throughput by adding additional postmasters on the same host.

eatonphil··on Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)
Great educational project! I'm curious why you are using Raft and also 2PC unless you're sharding data and doing cross-shard transactions? Or is Raft only for cluster membership but 2PC is for replicating data? If that's the case it kind of seems like overkill but I'm not sure.

Few distributed filesystems/object stores seem to use Raft (or consensus at all) for replicating data because it's unnecessary overhead. Chain replication is one popular way for replicating data (which uses consensus to manage membership but the data path is outside of consensus).

eatonphil··on Creators of Tailwind laid off 75% of their engineering team
LinkedIn says the company was 2-10 employees. 75% laid off wouldn't have been a lot of people. Tough for them though.
eatonphil··on Operating Systems: Three Easy Pieces Book Club
Hey folks, we're doing the next reading in the Software Internals Book Club. This is a long book so we'll cover it in three chunks with breaks in between. There are over 2,500 members in the club mailing list overall and typically 300-800 join any given book reading. It's all asynchronous over email. Different people kick off discussion for each chapter which makes it more fun and more sustainable to run. Hope to see you there!
eatonphil··on Worst Case Optimal Joins: Graph-Join Correspondence
Part of a series:

- https://finnvolkel.com/wcoj-generic-join

- https://finnvolkel.com/wcoj-datalog-and-genericjoin

- https://finnvolkel.com/wcoj-dbsp-zsets-and-datalog

- https://finnvolkel.com/wcoj-wcoj-meets-dbsp

eatonphil··on The Most Popular Blogs of Hacker News in 2025
brooker.co.za author's name is Marc not Chris. :)

scattered-thoughts.net author is Jamie Brandon. :)

eatonphil··on Biscuit is a specialized PostgreSQL index for fast pattern matching LIKE queries
Noticed Daniel Lemire talking about it and how they use Roaring Bitmaps.

https://x.com/lemire/status/2000944944832504025

eatonphil··on Disks Lie: Building a WAL that actually survives
Seconded.
eatonphil··on Disks Lie: Building a WAL that actually survives
Check out Parity Lost and Parity Regained and Characteristics, Impact, and Tolerance of Partial Disk Failures (which this blog indirectly cites) if you'd like authoritative reading on the topic.

https://www.usenix.org/legacy/event/fast08/tech/full_papers/...

https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...

eatonphil··on Mazda suitcase car, a portable three-wheeled vehicle that fits in the luggage
I think I've seen that Honda one around NYC. Had no clue what it was. It just looked like someone was riding a brief case and that's basically indeed what it is.
eatonphil··on Unexpected things that are people
Looks interesting, thank you!
eatonphil··on Uv is the best thing to happen to the Python ecosystem in a decade
> pyproject.toml

Ah ok I was missing this and this does sound like what I was expecting. Thank you!

eatonphil··on Uv is the best thing to happen to the Python ecosystem in a decade
> 2. You end up needing to use `uv pip` so it's not even a full replacement for pip.

Needing pip and virtualenvs was enough to make me realize uv wasn't what I was looking for. If I still need to manage virtualenvs and call pip I'm just going to do so with both of these directly.

I had been hoping someone would introduce the non-virtualenv package management solution that every single other language has where there's a dependency list and version requirements (including of the language itself) in a manifest file (go.mod, package.json, etc) and everything happens in the context of that directory alone without shell shenanigans.

eatonphil··on Consistent hashing
Wow I didn't know this history about Akamai, thanks for mentioning, interesting as a former Linode guy and a fan of consistent hashing.
eatonphil··on Bringing fully autonomous rides to Nashville, in partnership with Lyft
What is the reason or benefit of them being so secretive about this?
eatonphil··on SQL needed structure
Guessing from the relevant masters thesis linked on his site and his time at materialize and tigerbeetle, Jamie's been working on databases for at least like 15 years?

At least personally when Jamie says something I listen. :)

eatonphil··on Replacing a cache service with a database
Many of these points are not compelling to me when 1) you can filter both rows and columns (in postgres logical replication anyway [0]) and 2) SQL views.

[0] https://www.postgresql.org/docs/current/logical-replication-...

eatonphil··on SQLite's documentation about its durability properties is unclear
You're arguing a strawman and I pointed at a specific example. Sticking with my specific example they could probe for this behavior or this OS version and crash immediately, telling the user to update their OS. Instead it seems they acknowledge this issue exists and they hope it doesn't happen. Which hey everybody does but that's not the claim OP was making.
eatonphil··on SQLite's documentation about its durability properties is unclear
Take a look at Alex Miller's diagrams for what function calls are actually doing on various systems.

https://transactional.blog/how-to-learn/disk-io

eatonphil··on SQLite's documentation about its durability properties is unclear
> Sqlite's test suite simulates just about every kind of failure you can imagine

The page you link even mentions scenarios they know about that do happen and that they still assume won't happen. So even sqlite doesn't make anywhere near as strong a claim as you make.

> SQLite assumes that the operating system will buffer writes and that a write request will return before data has actually been stored in the mass storage device. SQLite further assumes that write operations will be reordered by the operating system. For this reason, SQLite does a "flush" or "fsync" operation at key points. SQLite assumes that the flush or fsync will not return until all pending write operations for the file that is being flushed have completed. We are told that the flush and fsync primitives are broken on some versions of Windows and Linux. This is unfortunate. It opens SQLite up to the possibility of database corruption following a power loss in the middle of a commit. However, there is nothing that SQLite can do to test for or remedy the situation. SQLite assumes that the operating system that it is running on works as advertised. If that is not quite the case, well then hopefully you will not lose power too often.

eatonphil··on Why Aren't People Going to Local and Regional In-Person Events Anymore?
To the contrary, when all the good New York meetups (Papers We Love, Linux User Group) didn't come back, and inspired by the continuously running Munich Database Meetup and TUMuchData, I started the NYC Systems Coffee Club and co-started NYC Systems (talk series) after which came Berlin Systems Group, Bengaluru Systems Meetup, San Francisco Systems Club, Systems from HEL, DC Systems, Vancouver Systems, South Bay Systems, and Seattle Systems.

So I think people are really eager for high quality talks and chances to gather with smart people.

What's more I think there are not enough meetups in almost any major city to satisfy the demand of speakers or attendees. For example, NYC Systems gets hundreds of people asking to speak (we have 12 speakers a year) and gets 2-3x as many attendees wanting to come as we have space for.

eatonphil··on SQLite (with WAL) doesn't do `fsync` on each commit under default settings
Is this sqlite built from source or a distro sqlite? It's possible the defaults differ with build settings.
eatonphil··on The Raft Consensus Algorithm (2015)
Could be, it was a little ambiguously worded. That said, single-decree Paxos is much simpler than Raft but I agree The Part-Time Parliament's analogy is a pain to read. But it's better if you just ignore the beginning chunk of the paper and read like the appendix; A1 The Basic Protocol being simpler to understand.
eatonphil··on The Raft Consensus Algorithm (2015)
Anyone can implement Raft. There are plenty of implementations of them not by Google engineers, including a custom one in the product I work on. And developers in the Software Internals Discord are constantly in there asking questions on the road to implementing Raft or Viewstamped Replication.
eatonphil··on The Raft Consensus Algorithm (2015)
Check out Alex Miller's Data Replication Design Spectrum for what you might use instead of Raft (for replication specifically), or what tweaks you might make to Raft for better throughput or space efficiency (for replication).

https://transactional.blog/blog/2024-data-replication-design...

eatonphil··on Neki – Sharded Postgres by the team behind Vitess
I'm not talking about speaking the protocol. I'm talking about trying as hard as they can to be as indistinguishable from Postgres (to a non-operations user) as they can. And that list is very small.
← PreviousPage 3 of 34Next →