Comment by tptacek | Hacker News Reader

Comment by tptacek | Hacker News Reader

Parent Full thread

If you have a "single source of truth", sqlite/Litestream seems like a pretty-near-optimal way of taking advantage of the relational model while keeping design simplicity. I don't know what the higher-level architecture of Tailscale is, but we have the same problems; we have a complicated "single source of truth" that takes the form of a Consul cluster, but sqlite makes an absolute ton of sense for us, because we can condense the Consul cluster down to a SQL schema and then ship it around our fleet.

(we don't use Litestream right now; we're pushing the complexity up a layer in our design instead)

A lot of people on this thread are looking down their noses at sqlite, but I think they're kind of beclowning themselves; the unreasonable effectiveness of sqlite has been a meme in infra dev for a couple years now. It's not a new idea. Lots of people are doing stuff like this.

The funniest bit on this thread is the person saying they should use RDS, as if their infrastructure was just a big Rails app.

It would be interesting to known why a standard boring RDS setup wouldn’t solve their problem completely. In fact I would be more interested to understand that than the actual details of sqlite tailing.

(I think the reason they gave was vendor lock-in, but apart from that, I didn’t understand why it wouldn’t be adequate)

One obvious reason not to use RDS is wanting disk-local caches of information replicated from a single leader, rather than having every single machine in your fleet calling out to an external service on every read. That's certainly why we're not considering Postgres in our infrastructure, even though managed Postgres is a product we in fact offer.

We use Postgres! It's the backing state for our API, and an important source of truth in our architecture. Postgres is a great way of serving a GraphQL API. It is not necessarily a good way to back an infrastructure service.

Makes sense once you think of Tailscale as an infrastructure-level service and not just an app. Thanks

Sounds like postgres listen/notify could be viable for your high-read-low-write use case? Or it is not scalable enough for the fleet size?

That's still all your infrastructure components calling out to an external service on every read --- and for what advantage?

This isn't an app server; it's an infrastructure component, running (I don't know about Tailscale here, but we use SQLite in similar uses cases) on potentially hundreds or thousands of machines.

No, you don't need to make any network call on every read with listen/notify. It will essentially be a local cache, working the same way as etcd watcher, using one postgres connection per machine.

Reply on news.ycombinator.com