Hacker News Reader

Comment by crawshaw | Hacker News Reader

It is a tricky tradeoff for startups. On the one hand, a startup has very limited resources and so has to focus on the business. On the other hand, a startup has to experiment to find the business. I don't think there's an easy answer.

In our case, the control plane data store really should be as boring as possible. It was real stretch using anything other than MySQL. We tried to lay out the arguments in the post, but the most compelling was we had lots of unit tests that spun up a DB and shut it down quickly. Maybe a hundred tests whose total execution time was 1.5s. The "boring" options made that surprisingly difficult.

(Tailscaler and blog post co-author)

View on HN

For PostgreSQL and Go, here's a package to spin up a temp in-mem PostgreSQL: https://github.com/rubenv/pgtest/

From memory, in Postgres, you could also have a copy of your base database and then copy it in for each test, which is seem to recall being fairly fast. It includes the data too.

    Create database test_db template initial_db;

I'm used to Scala for these things and it seemed fairly easy to do. Docker DB container would be spun up at the start of testing (using a package such as testcontainers-scala). Then the DB would be setup using the migration scripts (using Flyway) so it has a known state. Every test (or set of tests if they're linked) would have a pre-hook which nuked the DB to a clean state. Then the docker container would be shut down at the end of testing. I'm guessing there's even some way to have this run concurrently with multiple DBs (one per thread) but we never did that. Is Java's ecosystem for this type of tooling just that much better?

I wrote about that in the article. Search for "dockertest".

We considered that option but didn't like it. It still wasn't fast enough, and placed onerous (or at least annoying) dependencies on future employees.

> It still wasn't fast enough, and placed onerous (or at least annoying) dependencies on future employees.

Did you configure the Postgres (or MySQL) database to be entirely in memory, e.g. by using a tmpfs Docker volume?

As for being onerous or annoying for new employees, which is worse: having to set up a Docker environment, or using a relatively obscure data store in a way that nobody else does?

Empirically, the former.

We've since hired many employees who just learned about our database today from this blog post but had been happily testing against it on their laptops for months.

3-4 of us know about it, and that's sufficient.

One solution is to use a standard subset of sql so you can use sqlite in unit tests and mysql/postgres in prod. Many languages also have in-memory SQL implementations that are also a more convenient substitute for sqlite.

Of course the benefit of what you did, even if I wouldn't have done it, is that you're _not_ using a different system in dev vs prod.

Exactly. That's why I wrote: "No Docker, no mocks, testing what we’d actually use in production."

I would not go down the road of figuring out what subset of SQL various database understand. You will always be surprised, and you'll be surprised in production because "the thing" you developed against sqlite doesn't work in postgres.

I used to do this and stopped when I noticed that sqlite and postgres treat booleans differently; postgres accepts 't' as true, but SQLite stores a literal 't' in the boolean-typed field. This means you get different results when you read things back out. All in all, not a rabbit hole you want to go down.

Personally, I just create a new database for each test against a postgres server on localhost. The startup time is nearly zero, and the accuracy compared to production is nearly 100%.

Reply on news.ycombinator.com