Testing techniques in Go

It uses two neat strategies:

- testing/quick is used to generate random test cases that try to produce a failure, and;

- when a failure is found, another algorithm kicks in that minimizes the input that caused the failure, removing any superfluous operations.

The former uses a helper function that converts a `[]byte` (supplied by testing/quick) to a list of operations. This has a nice secondary benefit, which is that the same helper function can be used with fuzzing libraries.

The latter is really handy because testing/quick generates rather unusual test cases, and it can be hard to piece together which part of a large input caused the failure.

Very cool!

I liked this blog post and agree with most of it. But I think its section on interfaces is a bit muddled. It links to this article[1] for more advice, and while the article sounds nice in principle from the perspective of someone publishing a library for others to use, I’ve found other techniques to be valuable as well.

One technique I’m particularly fond of is writing a package whose main goal is to provide an abstraction layer between a storage system (e.g., PostgreSQL) and one’s model of data. The utility of the package permits callers to access data without caring too much about the underlying details of the storage itself.

This on its own doesn’t require an interface. It makes sense to define a type that implements various routines that interact with the storage layer and be done with it. But something what’s really useful is to create an in-memory version using the exact same API with the same contracts. It’s useful to use the in-memory version not necessarily just in testing, but potentially also in command line debugging tools. In particular, it lets one debug or test other components of the system without necessarily depending on a running storage service. This doesn’t replace integration tests, of course, but it does permit unit testing in a more fine grained way.

The only real issue here is that you want everyone else that uses this storage system to be able to also use the in-memory implementation as well. The most natural way to do that is to define an interface. Now, you could take the route that is being advocated here and define the interface in each package that wants to use this storage system. I tried that. It got old, real old, fast. Re-defining that interface N times was just silly. And doing it in a granular way was just annoying. As soon as one part of the code wanted to use another aspect of the storage, you need to update any intervening interfaces with an additional method. It was just tedious work that served little to no value from my perspective.

So I switched over to something more like this:

    type Store interface {
        FooA() (*Bar, error)
        FooB() (*Baz, error)
        FooC() (*Quux, error)
        // Potentially many more methods
    }

    type postgresStore struct { ... }
    func NewPostgres(db *sql.DB) Store { ... }

    type memoryStore struct { ... }
    func NewMemory() Store { ... }

The key thing I realized is that Store isn’t meant to be some super modular interface. (In principle, I might implement this using a sum type in a different language.) Other people don’t ever need to implement it. For the most part, the two implementations—one “real” and one in-memory only—are all you ever need. That is, the interface isn’t really being used as a mechanism for extensibility, but rather, as a pragmatic tool to abstract over a small number of implementations of the same API. Namely: if one didn’t care about granular unit tests on this storage API and only ever used integration tests, then there would be no reason for the in-memory implementation and therefore no reason for the interface.

Rob Pike said, “The bigger the interface, the weaker the abstraction.” I’d probably agree with that. And I’m OK with it. The Store interface isn’t meant to be a strong abstraction. It is, in fact, quite weak!

[1] - https://rakyll.org/interface-pollution/

It has several awesome ideas.

And big data glue is best written in Python.

Now, of course, Segment may be big enough to have many large codebases and Go is just one of them.