Test Desiderata 8/12: Tests should be isolated from each other [video]

In my experience a mixed approach can be valuable, but I think whatever approach is taken the real key is to build it in such a way that it's not an individual test's responsibility to clean up either before or after - it's extracted to the test runner/framework so every test gets the appropriate behavior automatically.

As an example of what I mean by a mixed approach: if the thing being tested is software that primarily interacts with, say, a SQL database, then the "clean up before" approach (which might mean running `TRUNCATE foo` on every table) works, but as the schema gets bigger it gets slower, and while it might not be very slow it gets very noticeable if every test runs it (particularly as the test suite is presumably also growing). An effective way to implement the "clean up after" approach might be to setup the test runner so every test runs within a transaction and after the test the transaction is rolled back: but in order for that to be reliable the database must be in a clean state when the test suite is started. So the mixed approach I've used (and is close to the default for Rails & Phoenix apps nowadays, I think) is to do the "clean" op at the start of the entire suite and configure it so every test runs in a transaction.

The transaction/snapshotting features of modern SQL databases make that approach really easy, of course, so if a project's main external interaction is instead filesystem interaction or something it may not be so simple and more of the infrastructure would have to be hand-rolled, but if it has to be done at scale (many tests, many developers), I do think it's worth abstracting into the test runner so that individual tests don't need to remember those details: I've done that for some CLI tools where the test runner sets up a sandbox dir for each test and so the tests just need to use the appropriate patterns for FS interaction (which is easy since all the existing tests do it and people imitate those, and in my case it helped stop the "just write to /tmp" approach that some had taken before that).

The container isolation approach certainly has the advantage that it's difficult to mess up no matter what any particular test does, but if you run each test within a container, in the long term I'd be worried about the overhead causing the same headaches as the "truncate DB tables before every test" approach. And if you're running the entire test suite in a container it seems like you'd still have the potential challenges of each test needing to be responsible for stuff unless some of the other approaches are also taken? Sorry if I didn't understand the container approach you described.