Hacker News Reader

Aren’t you making the point of the author?

Most places don’t have a way to run in development because they haven’t sat down and gotten the application to a running state. Every year it gets ignored, things get harder and more tedious but never impossible

I see both sides of this argument. Often legacy systems are tied to their deployment methodology/dogma and the world moves on from that model. Without a lot of resources poured into maintaining that methodology and upgrading it, the expected environment can age out of current practices far enough that running it locally does take a lot of effort.

I think docker is an obvious enabler of this because people can just spin up an image from 6 years ago in a container and keep running against old libraries/dependencies. Even worse is depending on that container makes it difficult to upgrade dependencies to modern versions of software because the container is so old.

Maybe a corollary to the article is, "without constant maintenance, software becomes un-runnable."

It seems tantamount to declaring that a system can never interact with the outside world.

If I'm creating tax filing software, do I need to spin up a local instance of the IRS to test with?

You can possibly stub out the IRS with realistic test cases that you care about.

> "Run an individual service against mocks" doesn't count.

> "Run most services in a mostly-isolated development environment, calling out to a few hard-to-run external services" doesn't count.

I think they're talking about services that you or your firm develops, or maintains, or ships. In this case the IRS is none of those things.

Yes, obviously this. People who don't understand this either just want to argue, or haven't thought about this at all.

> If I'm creating tax filing software, do I need to spin up a local instance of the IRS to test with?

No, integrate directly with the test endpoints provided by the IRS [1].

[1]: https://www.irs.gov/pub/irs-schema/Portal%20Transition%20Gui...

In the spirit of the article, I think that the approach would be to ask your partners to provide a sandbox/test environment that your sandbox/test environment can interact with, or test accounts in production at least.

> In my 28 years of professional experience working in 9 different organizations, only one of them did fully run the core system in development, and another two of them could have done that with enough time and effort. For the other six, there was no reasonable way to run the full system in development.

Only ~20 years here but I've worked almost exclusively on three or four large systems with test environments despite their complexity. I think it may be the nature of the companies/business that makes the difference.

Ancient (1980's code) hospital ADT+billing system (medseries 4 if anyone cares) on an AS/400. We kept the last generation of hardware around to run the test/demo environment which was end-to-end connected via HL7 to all the other test environments. That included the other thing I managed, Cerner Millennium, a hodgepodge of a medical records system on AIX and Oracle, where again the old hardware got used as the test and build systems. The test environment was critical because even though it was made up of a dozen commercial software packages talking to each other over HL7 and flat files (and serial for lab devices and who knows what else) it was critical to make sure the whole environment worked when doing upgrades or new development. We kept a full copy of prod data for final integration testing. The tooling for (most of) the software supported replicating environments.

The largest example is Photon/Ubiq at Google; end-to-end test environment running with deterministic sampling to save resources at all stages of the pipeline so everything from injection through logjoining and aggregation to the final database updates were continuously running the latest code for testing before a release to prod. Thousands of prod machines devoted to it. That was in addition to the integrated tests and unit tests during the development process.

At FB the statistical counter system I worked on also kept a continuous test pipeline running, again with event sampling to reduce cost, to ensure everything would work properly before a new release.

My takeaway is that where it matters companies are willing to spend the money and people to have full end-to-end testing, often in realtime, and have some leeway to save on resources with subsampling, smaller datasets, etc.

> We kept the last generation of hardware around to run the test/demo environment which was end-to-end connected via HL7 to all the other test environments.

If I understand right (and I may not, both your description and the OP), this may violate the OP's prohibition I've quoted below. What do you think?

> "Run an individual service in a shared stateful development environment running all the other services" doesn't count. A shared development environment will be unreliable as it diverges more and more from the real system.

The OP seems to be awfully strict.

"Run all the services that make up the system in an isolated development environment" was maintained in all the test environments. It'd be a moderate disaster to mix live patient/billing data between prod and test and so the test systems linked together with independent test/demo/prod HL7 gateways.

Right. If I understand it, it does seem to me to be an acceptable and reasonable solution, and I can't think of a better one.

But isn't it the "shared stateful development environment" the OP is saying is unacceptable? Since every developer doesn't get their own "last generation of hardware"?

But I may be misunderstanding the OP?

I am not trying to critisize what you did; I am trying to understand and possibly critique the OP. The OP seems unrealistic to me, despite it's protestations that it really isn't; but I may be misunderstnading the OP.

Shared stateful development environment implies that "hard-to-test" things are not tested.

In that sense, sure; we had one active directory, one physical network, one physical workstation to trigger commands, one office chair, no test version of me, etc.

So if the OP's language is not careful then it could include things in "the development environment" that don't make sense.