Hacker News Reader

Microservices has little to do with tech. It is a way of organizing teams of people, funnelling all communication between teams through well defined specifications instead of ad-hoc meetings. It is not clear where static typing, or lack thereof, comes into play here.

TDD is a method of documenting your application in a way that happens to be self-verifying. You could use a Word document instead, but lose the ability for the machine to verify that the application does what the documentation claims that it should. Static typing does provide some level of documentation as well, but even if you have static typing available static typing isn't sufficient to convey the full intent that your documentation needs to convey to other developers.

# Microservices

I think at the root this idea that Microservices provide this technical solution to a social issue, coordinating teams at scale, hides the driving motivation.

Why can't the teams work on the same codebase and the same database? (again note my edited in disclaimer in the GP post that maybe at certain rare scales it's necessary). What is a well defined specification and why is it only a REST/proto/whatever network API? Why not a defined interface or project/library? Why aren't teams at this scale working in languages that support defining these things in code without adding a complicating, latency and error-inducing, network call?

Statically typed languages are just superior at encoding this "well defined specification". It's a large part of their reason for being. Sure you can still bypass it and do reflection or other naughtiness, but we shouldn't pretend a network call is the only solution.

# TDD

Testing is certainly a way to provide a self-verifying application. TDD is a toxic, hype-driven, mess that has led many good developers astray. Static languages have their own devils here, mocking being chief among them (not to mention the Java clown brigade and their own enthusiastic adoption of -- originally Smalltalk conceived -- TDD).

I added TDD both to annoy people but also to illustrate the general point. These are both concepts derived from the obvious drawbacks of dynamically typed languages that are then treated as generalist answers to complexity in software.

If you'll allow me some pseudo-intellectual waffle, the complexity of a software system C_dynamic + C_software = C_total, that is, complexity of dynamically typed languages and their lack of rigour, plus complexity of software and the domain as a whole gives you total complexity. These approaches primarily target the dynamic complexity. Therefore introducing them and all attendant drawbacks in places where total complexity doesn't include the dynamic complexity because tools and approaches to remove this source of complexity are used actually worsens the entire field.

> Why can't the teams work on the same codebase and the same database?

The same database is tricky because one team might want to change the schema, breaking another team's work. You then need to have meetings to discuss the changes that will work for all parties and you've broken "communicate only by contract", and therefore no longer doing microservices. If you can ensure that there is no way for teams to trample over each other then you could use the same database.

The same codebase is more feasible as long as the team boundary separation is clear. You are likely already doing microservices with other teams when you import third-party libraries. In practice, though, much like the database problem maintaining boundary separation is difficult if you can easily reach into other team's code, so often each team's work will be parted into distinct codebases using IPC between them to ensure that the separation is unbreakable. This certainly isn't a requirement, though, just one way to keep people in line.

> Testing is certainly a way to provide a self-verifying application. TDD is a toxic, hype-driven, mess

Testing is a development tool to help test the functionality of a feature. If you are faced with a dynamic language you very well might write tests to ensure that types are handled correctly. Static types can, indeed, relieve the need for some tests.

TDD, on the other hand, is a documentation tool to relay to other developers what functionality is intended to provide and how it is meant to be used. It is strange to think that documentation is toxic. Before TDD we wrote "Word documents" containing the same information, but often found that the code didn't do what the documentation said it did. What TDD realized is if the documentation is also executable then you can have the machine prove that the application behaves according to spec. It's not exactly magic.

# Microservices

Agreed the same database gets difficult but I guess, where two things need to change dependently there's coordination overhead, that's inescapable. You can in general use e.g. schemas to keep domains fairly separate. Whether it's your HTTP API interface changing or the change needs to be coordinated in the same process I don't think you have a way to avoid these meetings. (or like the 3rd party team I work with you just change your HTTP API and break production without telling us :shrug:).

I guess my point is languages like Java and C# are much maligned for their "boilerplate" and it doesn't let 10x genius programmers go brrrrrr etc. But that boilerplate serves a purpose. Projects (in C#, I can't speak to Java but I believe it has a similar concept) compile to separate .dll's, or .so's. In general a project will publicly expose interfaces, i.e. contracts, and keep its internals encapsulated. Crucially projects all live in the same codebase in the same monolith, they're folder/directory level separation, but on steroids. There are tricks with attributes and reflection that can break this separation but with the additional constraint that dependencies cannot be circular you're forced to make decisions about where exactly things live. This is where the dynamic crowd throw up their hands and say "there are so many layers of pointless abstraction!". I'd agree these languages tend to too many layers but the layers provide the foundation which can support a project growing from 1-2 devs all the way through to 50+.

I'm maybe finally becoming a grumpy old programmer and I'm glad much of the consensus is swinging away from the no-types, no schema, no rules position of the past decade. People forgot Chesterton's fence and decided the layers didn't provide anything, then they added them back, as network boundaries, which was indescribably worse and more of a headache.

# Testing

I guess it's a semantics debate really. I think the red-green-refactor loop of TDD doesn't bring much. I've never seen it applied well on a statically typed codebase and where I've seen advocates try and apply it I've seen them tunnel-vision to suboptimal outcomes in order to fulfil the ceremonial needs of the process.

I think a few tests that take place at the project/dll/so boundary and use as much of the real dependencies as possible are far preferable to high coverage. This is probably a no-true-Scotsman area of debate. Maybe TDD has been done properly and productively by some groups, I've never seen it and no one I've met ever has (check out my confirmation bias!).

> Whether it's your HTTP API interface changing

Interfaces don't change under microservices. Communication by contract enshrines a contract. You must ensure that your API does not break legacy users no matter what changes you want to make going forward. You have committed to behaviour forevermore once you submit the contract to other teams.

This may be another reason why IPC is often preferred over straight function calls as some languages, particularly those with static types, make extending functionality without breakage quite hard. An HTTP API can more easily resort to tricks to return different results to different callers.

> I think the red-green-refactor loop of TDD doesn't bring much.

You find little value in writing a specification for your work or you find little value in automatic validation that the program works according to your spec?

"Red-green-refactor" is a little more specific in that it says that you should write a spec only for the work you know you are going to work on in the short term, whereas TDD in general leaves room for things like writing the entire application's spec before getting down to business. I think this is most practical in the real world, generally speaking. Often you don't know what your entire application should do in the beginning, making writing a full spec unrealistic.

> I've never seen it applied well on a statically typed codebase

Interesting, as I have only ever seen specifications (be it TDD or "Word documents") written for work that is done in statically typed languages. In my experience, the dynamic programming languages tend to attract the cowboy coders who don't understand why other (future) developers need more complete documentation or why specs are important and have no care to see that those things are done.

Apologies for the delay in a reply. I think this has been a constructive discussion certainly for how I think about things.

> Interfaces don't change under microservices. Communication by contract enshrines a contract. You must ensure that your API does not break legacy users no matter what changes you want to make going forward. You have committed to behaviour forevermore once you submit the contract to other teams.

I think this entails a couple of things.

Firstly, it's a trade-off. If we discover some incorrect assumption is baked in to both sides of an interface we're kind of stuck and have a hard time with the migration path going forward. This isn't unsolvable and paths exist to correct it, but they're substantially harder than just updating both parts in the same commit. It moves coordination overhead as a trade-off.

Secondly, what I think actually happens is that most companies using microservices aren't this disciplined and just end up bodging or implicitly coupling services. Probably the pattern I'm talking about is "distributed monolith". Not the good kind where you just scale out your monolith but where you drink the microservices kool-aid and create "microservices" by adding network boundaries at random. This is the real-world application of microservices I have seen where I've seen it. I don't doubt some people can use it correctly with proper rigour. But my theory is the pattern took hold because using "distributed monolith" (appears to help / ) helps tame the complexity of a dynamic monolith. I find it incredibly hard to believe the niche pattern of microservices done correctly would have become so popularized otherwise.

> You find little value in writing a specification for your work or you find little value in automatic validation that the program works according to your spec?

I find little value in providing that specification through the TDD approach or associated ceremony. I prefer as few tests as possible at as high a level as possible with as few mocks as possible. I'll take a single test that takes 15 seconds to run over 1000 tests running in milliseconds.

Again for me this is a question of "what popularized this pattern?".

I think there's a lot of healthy debate to be had about how much testing is needed. I think TDD appeared to provide the kind of guarantees that are really helpful in a dynamic language and gained a lot of advocates that way. When I had to write some (type hint free) Python I naturally defaulted to writing TDD since keeping a large system in your head and error free is basically impossible without an exhaustive suite of tests. It's an approach that works very well in one context that applied to other contexts unquestioningly delivers a lot of pain.

Maybe you're lucky to have only worked with very deliberate and rigorous engineers in your career. Outside the top tier we're working with architecture astronauts and people who jump on whatever hype cycle happens to be passing which explains my anger with these concepts.

> Apologies for the delay in a reply.

Not at all. I'm in no rush.

> Secondly, what I think actually happens is that most companies using microservices aren't this disciplined and just end up bodging or implicitly coupling services.

If an organization is sufficiently large you may have no way to get another team on the line to even try. At that scale the other teams may as well work for other companies and that is what microservices models. If your organization is small, I tend to agree that you won't succeed. You can't beat Conway's Law.

> I find little value in providing that specification through the TDD approach or associated ceremony.

But, ultimately, what's the difference between writing your spec in an executable way or writing it in a Word document beyond the superficial differences in the languages? What is communicated to other developers ends up being the same.

> I prefer as few tests as possible at as high a level as possible with as few mocks as possible. I'll take a single test that takes 15 seconds to run over 1000 tests running in milliseconds.

Seemingly not all that common, interestingly, but Go comes to mind as a language that provides constructs to define separation between specification (TDD) and developer tests. Conceivably you could exclude your TDD specs from execution, only running the tests that help in your development process.

The value of TDD isn't in the execution, but in what is communicated to other developers. That the specs are executable is merely a nice side benefit to help with confirming that the implementation conforms to the spec. You still get 90% of the benefit of TDD even if you never run the executable, albeit granted at that point it is just a fancy Word document.

> I think there's a lot of healthy debate to be had about how much testing is needed.

Perhaps, but TDD isn't about testing. TDD is about providing documentation. I think there is less room for debate there. I'm not sure anyone who has ever inherited a codebase has wished it gave less insight into what the 'business needs' of the program are. Again, a Word document can provide the same documentation, but if you're going to write that Word document anyway why not go the extra mile and gain the additional benefits that come with execution?

I don't think this is true at all.

A compiler can't check that your logic is correct. There may be a bit of overlap in the things verified by tests and a compiler but they don't solve the same problem.

How are static types a requirement for encapsulation? Dynamic languages are perfectly capable of providing encapsulation. Statically typed languages are also perfectly capable of having very poor encapsulation.

The assertion that TDD is a solution to dynamic typing falls apart when you consider that the TDD dogma was born out of Java programmers. I won't argue that Java has the most expressive type system, but it _is_ statically typed.

My understanding is that it's kind of a 3-pronged thing, It originates (or is "rediscovered") by Kent Beck working in, at-the-time, Smalltalk. It has huge adoption in the RonR community. And lastly the usual Java suspects who never met a bad pattern they couldn't massively overadopt, e.g. the so-called "Uncle" Bob who never met a pattern he couldn't jam into code to overcomplicate it.

You know, I knew there was a major association between TDD and XP (wonder why!) and I had always heard of XP being Java related, but looking deeper I'm not sure that's has true as I thought

> It has huge adoption in the RonR community

Is that even true? DHH (RoR creator) hates TDD.

It's been hard to find a timeline or more general history of TDD and I should have probably used the word "had" above. But my understanding is RSpec[0] was a huge driver of the test driven development hype cycle. While DHH[1] has, as of 2014, come out against TDD and was probably never 100% on-board he is not the whole community and that's still a gap of 9 years where, in my understanding, it was broadly popularised as both "the way you should be doing things in Rails" and, as with most things in Rails "the way everyone should be doing things".

[0]: https://www.stevenrbaker.com/tech/history-of-rspec.html

[1]: https://dhh.dk/2014/tdd-is-dead-long-live-testing

Not to mention that before TDD the same documentation was provided as plain text. TDD simply realized that if the documentation was also executable that the implementation could be automatically validated for compliance against the documentation. The idea that you are reconstructing the compiler doesn't even track on the surface.

It is true that one may use testing to stand in for the lack of static typing, which is no dobut what the parent is talking about, but TDD != Testing. Testing long predates TDD.

Dynamic typing + Microservices + Unit Tests just blows everything out of the water on Development Speed / Time to Market.

Most important thing for lots of startups and companies is Time to Market.

Traditional static typing based approachs are just a bad joke (3 times slower on average) in comparsion.

That's why we have the whole microservices and dynamic typing thing going on, because businesses that use it beat up businesses that don't. It's pretty simple really.

> That problem? Not using statically typed languages.

TDD was invented by a Java programmer. How does that fit into your world view?

Java's original type system, while static, was far from being powerful enough to provide strong guarantees of safety.

Saying age is an int is great, but Java doesn't let you say age is an int ranging from 0 to 130.

You can of course create an age object, but the constraints on that object cannot be expressed within the type system.

So you have to add unit tests instead, the unit tests in effect extend the type system out to be as powerful as type systems from the 80s.

Yay, progress! :/

> Java doesn't let you say age is an int ranging from 0 to 130

This sounds good on paper, but good luck passing your custom types to third party libraries. Or even doing something as simple as calculating average age.

You do realize the more powerful the type system is, the more closely it is going to resemble programming language, which means that structures you build on top of it will contain bugs? You're not really solving bugs, you're just pushing them to another layer.

> This sounds good on paper, but good luck passing your custom types to third party libraries. Or even doing something as simple as calculating average age.

Ada solved this in the 80s, it isn't some unresolved field of comp sci. (Java already does this for array bounds!)

> You do realize the more powerful the type system is, the more closely it is going to resemble programming language, which means that structures you build on top of it will contain bugs? You're not really solving bugs, you're just pushing them to another layer.

Unit Tests are no different, except with worse syntax than built in language support.

At least with support in the type system you can turn off range checks for production builds. Meanwhile unit tests only run when invoked, giving less confidence than type checks that undergoing simulated usage as part of daily test builds.

There is of course a gradient, JavaScript exists on one side of that gradient (unit tests are needed for everything, starting with what type of object is even being returned), Java exists in the middle, and you have actual type safe languages farther along.

The point I was originally aiming to make is that Java is statically typed, but its type system is not so powerful as to negate the need for unit tests.

> Ada solved this in the 80s, it isn't some unresolved field of comp sci

Every solution comes at a cost. Where is Ada now?

> Unit Tests are no different, except with worse syntax than built in language support.

And I'm not advocating for unit tests, nor do I treat them as replacement for types. The fact that they occasionally overlap doesn't mean they serve same purposes.

> At least with support in the type system you can turn off range checks for production builds

Sounds like dynamic typing with extra steps.

> And I'm not advocating for unit tests, nor do I treat them as replacement for types. The fact that they occasionally overlap doesn't mean they serve same purposes.

So you don't advocate for unit tests, you don't want a powerful type system, what do you want?

> Every solution comes at a cost. Where is Ada now?

A source of features for newer languages, thus is the circle of programming language life.

> Sounds like dynamic typing with extra steps.

Static assertions based on build flags have been around for a very long time, in all sorts of languages. They are a performance/safety trade off.

> So you don't advocate for unit tests, you don't want a powerful type system, what do you want?

"Write tests. Not too many. Mostly integration."

Leave static typing for performance-critical sections.

> They are a performance/safety trade off.

So is dynamic typing. Only performance in this case means developer performance. Turns out most of our software is not landing airplanes, and developer performance is way more important than occasional non-critical bug that affects 5 users and gets fixed within a day.