Modules, not microservices
blogs.newardassociates.com
blogs.newardassociates.com
There's two technical problems that microservices purport to solve: modularization (separation of concerns, hiding implementation, document interface and all that good stuff) and scalability (being able to increase the amount of compute, memory and IO to the specific modules that need it).
The first problem, modules, can be solved at the language level. Modules can do that job, and that's the point of this blog post.
The second problem, scalability, is harder to solve at the language level in most languages outside those designed to be run in a distributed environment. But most people need it a lot less than they think. Normally the database is your bottleneck and if you keep your application server stateless, you can just run lots of them; the database can eventually be a bottleneck, but you can scale up databases a lot.
The real reason that microservices may make sense is because they keep people honest around module boundaries. They make it much harder to retain access to persistent in-memory state, harder to navigate object graphs to take dependencies on things they shouldn't, harder to create PRs with complex changes on either side of a module boundary without a conversation about designing for change and future proofing. Code ownership by teams is something you need as an organization scales, if only to reduce the amount of context switching that developers need to do if treated as fully fungible; owning a service is more defensible than owning a module, since the team will own release schedules and quality gating.
I'm not so positive on every microservice maintaining its own copy of state, potentially with its own separate data store. I think that usually adds more ongoing complexity in synchronization than it saves by isolating schemas. A better rule is for one service to own writes for a table, and other services can only read that table, and maybe even then not all columns or all non-owned tables. Problems with state synchronization are one of the most common failure modes in distributed applications, where queues get backed up, retries of "bad" events cause blockages and so on.
So does modularity.
"The benefits expected of modular programming are: (1) managerial_development time should be shortened because separate groups would work on each module with little need for communication..."
On the Criteria To Be Used in Decomposing Systems into Modules, D.L. Parnas 1972.
He's right as far as Conway's Law goes, though.
Edit: failed to spell "I" correctly. Ouch.
The point is that Parnas never conceived of doing it because he was writing about a world where the interfaces between modules were known ahead of time and were static.
With microservices, as long as you maintain the contract with caller services, you can deploy whenever you want. If you have some kind of issue, your team is solely responsible. If the caller services do weird things, they are responsible for fixing them.
If you are pushing changes to a module as part of a more monolithic, or wrapper service - if you do a deploy and break the whole big service, you are now responsible for the entirety of any issue, which is now more likely due to integration risk and unrelated changes from others, especially because now there need to be coordination across many teams integrating - hopefully via tests and observability. But this requires a high-degree of maturity for automated quality assurance and site reliability. If you don't have that, the operational risks are very high. So that alternative is having some ops-like function, or other integration team responsible. Or doing more top-down waterfall coordination.
Given the service maturity needed is rare, microservices distributes that operational ownership in a way where there is more accountability.
Infrastructure that someone has to take care of.
In chaotic environments like that, microservices help with defining clear ownership of operational responsibility, though at the cost of clear responsibility for the overall distributed system. This comes at the cost of making it hard to develop, let alone ship, changes that impact multiple microservices (outside the scope of a single team, or closely related sibling teams).
The main point I was making was that, with a more monolithic system, managing that kind to organization disfunction, necessitates top-down, waterfall like control. So moving to microservices enables non-coordinated agility removing one layer of coordination problem (a the cost of other problems).
With modules you need some sort of wrapper application to bring it all together.
With microservices you need some sort of network layer so that the microservices can talk to each other.
You don't restart the network every time you deploy a microservice.
If we're being technical some languages support hot swapping modules so no restart would be needed. Setting that aside, restarting an application isn't anything that needs coordination today. You wouldn't even restart the application. You'd deploy a new version, swap over, and shut down the old version.
>You don't restart the network every time you deploy a microservice.
No, but something changes in the network configuration so that the other microservices are aware of the deployment.
They aren't built in, it's just that the need for them is impossible to ignore. Developers (and management) can't help but recognize and respect modularity in microservices because the separation of services and the APIs between them make the modularity obvious. When the separation between modules only exists in code, and even then only when seen through the eyes of someone who understands the modular architecture of the codebase, it is easily ignored, inevitably forgotten, and might as well not exist at all.
If all modules have to be deployed using the same build even though different build of the modules would have been API compatible - you don't have modules.
IMO a module is defined as something that has a self-contained API, and versioning rules for that API if the module is evolving.
Developers do a much better job with microservices. I think it's easy for them to respect the seriousness of designing and changing the API of a microservice. In contrast, they often don't respect or even realize the seriousness of making a change that affects the modular design of code.
Language-level support for defining and enforcing module interfaces might help developers invest the same level of care for module boundaries in a modular monolith as they do in a microservices architecture, but I've yet to work in a language that achieves this.
Often people are too lazy to issue API version change on a "bugfix", and take down multiple applications.
Funny thing, a modularized monolith may have identified some of those issues.
Modularization allows development to scale.
Microservices allow operations to scale.
What happens if you need to redesign the architecture to meet new needs? That's right; it's not going to happen, because people will fight it tooth and nail.
You also cannot produce any meaningful end-user value without involving several teams.
Microservices is just "backend, frontend, and database team" reincarnated.
My take: do Microservices all you want, but don't organize teams around the services!
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
— Melvin E. Conway
Amazon's warehouses have incredible domain complexity which drives the complexity of the software architecture. It's also one of the oldest parts of the company with a direct lineage going back 20+ years. (for example, our inventory databases still have the same schema and are still using a RDBMS, albeit RDS Postgres instead of Oracle).
About five years ago we started rearchitecting to a centralized orchestration service which is being used to abstract this complexity from process teams. This is to avoid projects where 20+ service teams must be engaged just to launch a new solution.
Long ago I consulted for one company that reconstituted teams for every new project, so people got swapped around every few months, usually ending up in a new chunk of the code. Whatever the theoretical benefits, it meant that nobody felt a sense of ownership for any of the code. Combine that with aggressively short project deadlines and an absolute refusal to ever adjust them and the code rapidly dropped in quality. After all, you'd be on something else soon, so any mess wasn't your long-term problem. And why should you bother given that the code you started with wasn't pristine?
As far as I can tell, the best it gets is organizing around the most durable features of the business environment. E.g., audiences and their real-world needs. Next, long-lived solutions to those needs. And then you find the common and/or large needs of those teams and spin up service teams around them. And in the background, some strong culture around what the priorities really are, so that service teams don't get so self-important that they become the tail wagging the dog.
But I love all the war stories here and would love to hear more, as I think there are no perfect global solutions to organizational problems. I think it's mostly in the details of circumstance, history, and the people involved.
Microsoft, Apple, Amazon I think do a good job at preaching the customer first model and organizing around that.
This is actually the whole point (see: Reverse Conway).
This isn't easily fixable, but I'd like technologists to at least be able to perceive the extent to which the surrounding business culture has permeated into technical culture. I'm probably considered old AF by a lot of you (41) but I'm becoming increasingly interested in tools/methodologies that enable fewer devs to do a lot more, even if it means that there are sharp edges.
As projects get larger, the problems become more organizational in nature.
Horizontal scaling always incurs communication overheads faster, and as we know, developers have high communication overhead to begin with.
Human scaling is its own problem set and companies like to ramp projects up and then down again. Most of those people only think about how to get more people on board, not how to run the ship once attrition starts to condense responsibilities and reduce redundancy, especially for unpleasant tasks.
I’ve been trying to beat the idea of ergonomics into a smart but mule-headed developer (too old not to know better) for years now and it wasn’t until a combination of team shrink and having to work with code written by a couple of people who emulate his development style that all of a sudden he’s repeating things I said four years ago (of course with no self awareness of where those ideas came from). It remains to be seen if it affects how he writes code, or whether he starts practicing Do As I Say, Not As I Do, but I’m a short-timer now so it’s mostly an anthropological study at this point.
Also - talking of redesign should be a major lift for any product/service that has real paying clients. The risk of breaking is huge, and the fact that during that time you won't be delivering incremental value is also going to look bad.
hence, if you want a certain architecture, you likely need to execute a "reverse conway law" org first. To get the org into the target config, the software will follow.
What does not work is splitting ownership of many tings for a long time. It’s too many plates to spin. But some things can be shared by a few people without tipping over into anarchy.
Conway's Law is basically "You don't have a choice." You will cement the design of your app around your organizational design. (At least, beyond a certain org size. If you have only 4 engineers you don't really have the sort of structure in question here at all.) je42 narrowly beats me to the point that if this is a problem you can try to match your organization to your problem, but that takes a fairly agile organization.
"What happens if you need to redesign the architecture to meet new needs? That's right; it's not going to happen, because people will fight it tooth and nail."
Unfortunately, in the real world, this is not so much "a disadvantage to the microservice approach" as simply "an engineering constraint you will have to work around and take account in your design".
Despite what you may think after I've said that, I'm not a microservice maximalist. Microservices are a valuable tool in such a world but far from the only tool. As with everything, the benefits and the costs must be accounted for. While I've not successfully rearchitected an entire organization from my position as engineer, I have had some modest, but very real success in moving little bits and pieces around to the correct team so that we don't have to build stupid microservices just to deal with internal organizational issues. I don't mean this to be interpreted as a defeatist "you're doomed, get ready for microservices everywhere and just deal with it"; there are other options. Or at least, there are other options in relatively healthy organizations.
But you will have code that matches the structure of your organization. You might as well harness that as much as you can for the benefits it can provide because you're stuck with it whether you like it or not. By that I mean, as long as you are going to have teams structured around your services whether you like it or not, go in with eyes open about this fact, and make plans around it to minimize the inevitable costs while maximizing the benefits. Belief that there is another option will inevitably lead to suboptimal outcomes.
You can't engineer at an organizational level while thinking that you have an option of breaking the responsibility and authority over a codebase apart and somehow handing them out to separate teams. That never works long term. A lot of institutional malfunctioning that people correctly complain about on HN is at its core people who made this very mistake and the responsibility & authority for something are mismatched. Far from all of it, there are other major pathologies in organizations, of course. But making sure responsibility & authority are more-or-less in sync is one of the major checklist items in doing organization-level engineering.
The good news is, pretty much everyone has been on at least one side of that balance being broken (that is, having responsibility without authority, or being subject to someone else's authority without responsibility), and a lot of people have been on both, and once you bring it to their attention that they're creating another one there is a certain amount of motivation to avoid it, even if it is a bit more work for a them. You can actually use people's experiences with bad organization to convince them to not create another one, if you are careful and thoughtful.
If your org doesn’t suffer feature creep willingly, I believe that means you can have a lot more flexibility with respect to Conway’s Law. A low-churn project can maintain a common understanding. Not all of them will of course, but it’s at least possible. Feature factories absolutely cannot, and you shouldn’t even try.
What really kills a lot of projects though is an inverted dependency tree. And by inverted I mean that the most volatile code is at the bottom of the call graph, not the top. In this case every build, every deploy, or in the microservices scenario every request, can have different behavior from the previous one because some tertiary dependency changed under you. Now you need all sorts of triggers in your CI/CD pipeline to guarantee that integration tests are rerun constantly to figure out where the regressions are being introduced. Your build pipeline starts to look like one of those server rooms with loose wires everywhere and a bucket under the AC unit. Nothing is stable and everything is on the verge of being on fire at a moment’s notice.
> Any organization that designs a system (defined broadly) will produce _a design whose structure is a copy of the organization's communication structure_.
Organization is the primary method by which we organize communication (indeed, that's its only point). That's why architecture follows organization. It is possible to "break" that boundary by organizing other forms of communication, for example the feature teams of SAFE (not a proponent of safe, just an example).
How often does major reorganization happen? For most companies the answer is never.
The "what if" argument is what leads to all sorts of premature optimizations.
For most companies – regularly.
But the need for them does :)
Yeah, didn't get a lot done...
Exactly. In order to reap the benefits of modular/Microservices architecture, you need teams to be organized around product/feature verticals.
IMO it’s not about the internals of a product/service architecture, but about encapsulating, isolating, and formalizing inter-organizational dependencies.
That will always happen, and I've seen it happen in a monolith too (expressed as module ownership/ownership over part of the codebase).
It's inevitable in almost all organizations.
In the outside world, for an application that may truly need to scale, I'd go MySQL -> Vitess before I'd choose separate data stores for each service with state. But I'd also question if the need to scale that much really exists; you can go a long way even with data heavy applications with a combination of sharding and multitenancy.
Imagine a world where every pip/nuget/cargo package was a k8s service called out-of-process and needed to be independently maintained. We would have potentially hundreds of these things running, independently secured via mTLS, observability and metrics for each, and all calls run out of process. This is the abysmally slow hellscape that some are naively advocating for without realizing it.
It is relatively obvious what should be a library (json, numpy, NLog, etc), and what should be a service (Postgres, Kafka, Redis, NATS, etc.) when dealing with 3rd party components. It is also obvious that team scaling is not determined by whether code exists in a library or service from this example since all 3rd party code is maintained by others.
However, once we are all in the same organization, we lose the ability to correctly make this determination. We create a service when a library would do in order to more strictly enforce ownership. I think an alternate solution to this problem is needed.
Services can be continuously deployed by small owning teams.
Is this correct or are there other aspects that you are considering?
Even if Microservices are better for scale, most companies will never experience the level of scale of 2013 Twitter.
Are Microservices beneficial at much smaller levels of scale? Ex: 1M MAU
I'm not sure at what point the monolith codebase became absolutely enormous, but I would bet that GitHub grew to 1M active users with a monolith just fine.
Micro-services might be necessary for the companies like Github, Twitter, Google that grow to be huge, but monoliths seem to work just fine for the vast majority of other companies.
I fully agree with your argument. Then again, as mentioned elsewhere in this discussion, microservices are often not introduced to solve a scalability problem but an organizational one and there are many organizations that have more engineers than Twitter (had).
Personally, I still don't buy that argument because by solving one organizational problem one risks creating a different one, as this blog post[0] illustrates:
> […] Uber has grown to around 2,200 critical microservices
Unsurprisingly, that same post notes that
> […] in recent years people have begun to decry microservices for their tendency to greatly increase complexity, sometimes making even trivial features difficult to build. […] we experienced these tradeoffs first hand
I'm getting very strong Wingman/Galactus[1] vibes here.
[0]: https://web.archive.org/web/20221105153616/https://www.uber....
This happens a lot. Organisational problems conflated for technical ones. Technical solutions to organisational politics. Etc. It's often easier to admit technical challenges than organisational ones. Also, different people get to declare technical and organisational challenges, and different people that get to design the solutions.
There's also a dynamic where concepts are created by the technical 1% doing vanguard or challenging tasks. The guys responsible for scaling youtube or whatnot. Their methods and ideas become famous and are then applied to less technically demanding tasks, then in less technical organisations entirely.
I think if we can be honest at the actual problem at hand, 80/20 fixes will emerge. IE, the "real" value is not the architecture per se, but the way it lets you divide the responsibilities in the organisation.
Each team gets to distribute libraries over repos (COM, JAR, DLL, whatever), no way around that unless they feel like hacking binaries.
Creating an object-oriented API for a shared library which encapsulates a module to the degree a service boundary would is not trivial and it's very rarely done, never mind done well. Most OO libraries expose an entire universe of objects and methods to support deep integration scenarios. Maintaining that richness of API over time in the face of many consumers is not easy, and it (versioning) is a dying art in the eternal present of online services. The 90s aren't coming back any time soon.
If you own a library which is used internally, and you add a feature which needs a lot more memory or compute, how do you communicate the need to increase resource allocation to the teams who use the library? How do you even ensure that they upgrade? How do you gather metrics on how your library is used in practice? How do you discover and collect metrics around failure modes at the module boundary level? How do you gather logs emitted by your library, in all the places it's used? What if you add dependencies on other services, which need configuring (network addresses, credentials, whatever) - do you offload the configuration effort on to each of the library users, who need to do it separately, and end up with configuration drift over time?
I don't think binary drops work well unless the logic they encapsulate is architecturally self-contained and predictable; no network access, no database access, no unexpected changes in CPU or memory requirements from version to version.
There's plenty of code like this, but it's not usually the level of module that we consider putting inside a service.
For example, an Excel spreadsheet parser might be a library. But the module which takes Excel files uploaded by the user and streams a subset of the contents into the database is probably better off as a service than a library, so that it can be isolated (security risks), can crash safely without taking everything down, can retry, can create nice logs about hard to parse files, can have resource metrics measured and growth estimated over time, and so on.
As for scalability, most orgs aren't Facebook nor Google scale, regardless of how much they want it to be true.
All of those issues can be tackled, when proper architecture design is done, instead of coding on the go and then will see attitude.
The important part of microservices isn't just API boundaries, it's lifecycle management. This CAN be done with a DLL or JAR, but it's MUCH harder today.
Sure if one likes to make things harder than they are supposed to be, there are lots of paths down that road.
It's invented by a software outsourcing firm to milk billable hours from contracts.
This breaks down when the database is essentially a generic graph. The worst solution I've seen to this is to have another service responsible for generic write operations and any service that wants to write data goes through that service -- you're essentially re-introducing the problem you're purporting to solve at a new layer with an added hop and most likely high network usage. The best solution I've seen is to obviously have the monolith. The enterprise solution I've seen, while not good by any means but nearly essential for promoting team breakdown and ownership, is to just let disparate services write as they see fit, supported by abstraction libraries and SOPs to help reinforce care from engineers.
I’ll go one step further and say that you should treat your data stores as services in and of themselves, with their own well-considered interfaces, and perhaps something like PostgREST as an adapter to the frontend if you don’t really need sophisticated service layers. The read/write pattern you recommend is a great fit for this and can be scaled horizontally with read replicas.
If you have a particular piece of the system that needs to be scaled, you can take that module out when it becomes necessary. You can alter your system such that you deploy the entire codebase but certain APIs are routed to certain boxes, and you can use batch processing patterns with a set of code outside the codebase.
You can have an admin application and a user application, and both of them are monoliths. They may or may not communication using the database or events or APIs.
However, you don't make this on the single bounded context guideline.
And I do agree that most people need less of those than they think.
At my last job, there were quite a few times where being able to scale some small "microservice instance" up from 2 -> 4 instances or 4 -> 8 or 8 -> 12 was a lot easier/quicker than investigating the actual issue. It'd stop production outages/hiccups. It was basically encouraged.
Not sure how that can be done with a "it's all modules in a giant monolith".
> A few ways, the easiest being to scale up the whole monolith with more instance
as far as I know, there's no way to granularly scale up a monolith. if the monolith has 20 or 50 or 100 modules and you need 1 or 2 of them scaled, you have to scale up the entire thing which is huge, expensive, etc.
> Another way is run multiple "services" using the same codebase, so you have workload segmentation, either via synchronous network calls, or async worker systems.
this is interesting. a monolith with some form of IPC? why not do microservices at that point? that sounds like microservices-ish?
Yes, yet people still do it that way. This is tradeoff against the costs of microservices. I'm not saying it is worth it, but yes, sometimes you can just inefficiently throw hardware resources at scaling problems.
> this is interesting. a monolith with some form of IPC? why not do microservices at that point? that sounds like microservices-ish?
Because you are incrementally changing an existing monolith, but still getting some of the benefits of scaling distinct workloads independently. If you do this right, it does lend it self to reworking the new "service" to be become its own microservice. Or you can just keep going with the shared codebase.
it brokerages them to subclasses/handlers for CRUD/RPC/plumbing to database/other services
then in a separate process App/main() you have a cron job or a queue of some sort
you bundle these together as a monolith into one main()?
We deploy a few instances of the monolith fronted by a load balancer. There's not much scaling logic here -- when we see p75 request latency reach a certain threshold, we add another monolith instance.
A while later, we realize that the /foo route is getting a ton of spikey traffic, which makes it hard to scale. So, in the load balancer, we set up a new pool that just targets /foo, and in our Maven project we create a new module that will be the deployable for this pool. Now, /foo still requires most of our libs, so the jar we're bundling is still "the monolith" in essence, but maybe it doesn't require a few dependencies, and so is a little slimmer. We deploy that and everything is scaling great!
Then, we discover that in our main monolith, the background threads we were using to send emails are getting really clogged up, which is adversely effecting performance. We decide we want to move this work to an external persistent queue, like Kafka. Now, for our consumer, we create one more Maven module for a new jar. This time, we're lucky. Emails only need 2 of our libraries and so this is actually a pretty small deployable, but it's still being built from the same core set of libraries as the monolith.
my concern would be if you accidentally create bugs by basically creating new mini-monolith flavors as the modules were never expected (in the beginning of their creation) to run in a "partial" context
A cool side-effect is that you can usually run the whole thing in one app for development by just enabling all profiles - as opposed to some microservice architectures where you need dozens of containers and DBs for replicating inter-service bugs.
Microservice scaling is also an inefficient hardware scaling option.
The benefit though is you get a bit of 'free' scaling. Most of the modules don't even have to consider scaling at all, as long as they are growing ~average or slower. So this saves a lot of operational burden for many teams. Conversely with microservices every team has to consider scaling, no matter how simple their service is.
(If you do have 1 module out of 100 that takes up 40% of the resources, then it may be appropriate to split it up. Monolith doesn't literally mean "exactly one service", after all, just a small number of big ones.)
I could be wrong but doesn't monolith usually refer to one really heavy app? As soon as you go from needing 1 instance to 2 (because 1 of the 100 inner "modules" needs to be scaled), I would guess most of the time there's a high expense (lots of memory) in duplicating the entire monolith? Unless you can scale threads or something instead of the entire process... Or if the entire process doesn't initiate 100 modules as soon as it is started (although I imagine it would in most cases)
No. Backend Server clusters are around almost as long as the internet exists.
>I would guess most of the time there's a high expense (lots of memory) in duplicating the entire monolith?
That's right, but there's a gotcha. Everytime you split your monolith into parts, that doesn't mean that each part will consume 1/n of original monolith startup resource. There will be a floor of memory usage. Consider 1 monolith that eats up 1GB to startup vs 4 microservices that uses 512MB each. Right from start you doubled the wasted memory from 1GB to 2GB. That only gets worse the more services are created. Another problem is that microservice/cloud folks loves to create anemic machines. Usually setting up 512MB RAM and half a core cpu to a service that needs 380mb minimum :) Thats 75% overhead and 25% of working memory. It's bonkers.
not necessarily. yes, you do pay for executable bloat and some memory overhead from loading code into memory, but the argument here is that you can still deploy a "Service B" from the same deployable that only services certain kinds of requests in order to scale parts of the application horizontally.
> this is interesting. a monolith with some form of IPC? why not do microservices at that point? that sounds like microservices-ish?
no, because again, you're still deploying the same single codebase, it's just the instance on the other end of the queue is configured just to consume and process messages.
Look, once upon a time managers designed the system that workers implemented. The factory assembly line, or the policy manual.
But now the workers are CPUs and servers. The managers designing the system the workers follow are coders. I say coders are the new managers.
Now this leaves two problems. The first is that there are a lot of "managers" who are now doing the wrong job. (and explains perfectly why Steve Jobs gets involved in emails about hiring coders.) But that's a different post.
For me the second problem is microservices represent the atomic parts of a business. They should not be seen as a technical solution to anything, and because the first problem (managers arent managing workers anymore) there is no need to have the microservices built along Conways Law.
And so if you want to build a microservice it is not enough to consider the technical needs, it is not enough to automate what was there, you need to design an atomic building block of any / your company. They become the way you talk about the company, the way you think about the processes in a company.
An mostly the company becomes programmable. It is also highly likely the company becomes built mostly of interchangable parts.
This is a misunderstanding of Conway's Law. Your code _will_ reflect the structure of your organization. If you use microservices so will their architecture. If you use modules, so will the modules.
If you want a specific architecture, have your organization reflect the modules/microservices defined in that architecture.
The point ebing is that if workers are CPUs and coders are managers, then why worry about how the managers of the coders are arranged. Get rid of that management layer. Conway is not implying the financiers of the organisation affect the architecture.
This basically means that the microservices a company is built out of should more readily align to the realities of the business model. And instead of shuffling organisations around it would behove executives to look at the microservices.
One can more easily measure orders transferred, etc, if the boundaries are clearer.
Plus conway is just a ... neat idea, not a bound fate.
There is a caveat with the architecture reflecting the organisation of the software teams, but that usually follows the other to-be-automated structure as well.
> Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
It doesn't matter WHO defines or does the work (the coders themselves in a team or the manager). It matters how communication is arranged. And communication naturally aligns itself along organizational and team boundaries.
When a group/person/whatever writes an API they are MOSTLY writing a document for other human beings. That is an act of communication. If you meet with someone everyday in a stand-up you're likely to put a less structured API between yourselves than if you meet with someone once a month, or once a year. You're also going to create a finer grained subdivision of the API for them as you're likely working on closely related posts of the problem.
Organization creates communication, and communication creates architecture. Therefore organization creates architecture.
> Plus conway is just a ... neat idea, not a bound fate
It is in fact bound fate. Code is written to communicate with others.
People come together, they decide to solve a problem. They divide up that problem and create functions (an API), objects (an API), modules (an API), and services/microservices (an API). At each later they've written something that communicates with others. The structure of that code reflects the necessary communication to the other parties for their part of the problem. This the code is structured following the lines of communication in the organization.
Organization is how we structure communication (that's it's primary point) and thus organization = architecture + some wiggle room for communication paths defined outside that (say a SAFE feature team).
Just to add it here, how evaluations and goals are set matter too. Those tend to be even more constrained by the formal organization than communication.
I've been a couple levels up the chain at times in my career. When I've seen technical decisions at this level done well is when those people realize that their organizational decisions are inevitably technical decisions because of Conway's law. When the impact of Conway's law is not explicitly considered, things will often look random at the IC level. Too often, organizational leaders don't realize they are dictating technical outcomes with decisions that appear to just be about reporting structure.
An example is when a major new product feature comes along that is big enough to need multiple teams to build/own it for the long term. Where are those teams placed in the organization? Conway's law helps us see that a huge part of the future architecture is being determined by answering that question. Placing those new teams in an existing org mainly responsible for UX will result in a very different architecture than putting the new teams in an org mainly responsible for monetization. Can the teams be split across those orgs? Well, can you envision a reasonable architecture with an enforced divide along those lines? Without recognition of Conway's law, that last question might not be considered and the decision to split or not will be based on other factors.
Unfortunately that requires multidisciplinary thinking (technical architecture, resource allocation, communication structure, people development, ...). Such broad thinking is not always easy. When leadership starts neglecting one or more of these disciplines, more and more of the people doing the work on affected teams start perceiving the decisions as arbitrary and counter productive. Organizational churn (e.g., teams repeatedly changing location within the org over the course of a year or two) is often leadership looking at a different subset of these dimensions over time, yielding different "correct" answers.
But if you chnage the dataflow in a few microservices so that the accounts team no longer deals / works directly with the sales team you have chnaged the organisation.
plus it's way easier to monitor activity etc
https://ardalis.com/conways-law-ddd-and-microservices/#:~:te....
Running code in a single process is MUCH lower overhead because you don't need to transit a network layer and you're generally just passing pointers to data around rather than serializing/deserializing it.
There are definitely some cases where using microservices does make things more CPU/memory efficient, but it's much rarer than people think. An example where you'd actually get efficiency would be something like a geofence service (imagine Uber, Doordash, etc.) where the geofence definitions are probably large and have to be stored in memory. Depending on how often geofence queries happen, it might be more efficient to have a small number of geofence service instances with the geofence definitions loaded in memory rather than having this logic as a module that many workers need to have loaded. But again, cases like this are much less common than the cases where services just lead to massive bloat.
I was working at Uber when they started transitioning from monolith to microservices, and pretty much universally splitting logic into microservices required provisioning many more servers and were disastrous for end-to-end latency times.
Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers.
But this only matters at Facebook scale. I think most devs would be shocked at how much a single beefy server running efficient code can do.
Those machines are made very redundant.
All components in a Parallel Sysplex configuration can operate in redundant mode.
Redundant power. Redundant central processor complex. Redundant multichip module (what would be a chipset on microcomputer platforms). Redundant RAM Memory. Redundant Storage. Redundant internal interconnects. Redundant network adapters.
That's why corporations that require high availability to not get sued for millions/billions use them.
Geographically Dispersed Parallel Sysplex.
I'm not sure about that: you could still put something together within that budget, say, a few different VPSes across Hetzner and Contabo (or different regions within the same provider's offerings), with some circuit breaking and load balancing between those. Probably either a managed database offering, or a cluster of DBs running on similar VPSes.
Of course, this might mean that you have 1-3 instances of a service instead of 10-30, but as long as availability is the goal and not necessarily throughput, that can go pretty far.
Otherwise what you have is a budget that is lower than the possible implications of temporary downtime. That doesn't make sense in the real world.
sed "s/amateur/comparatively poor, from a third world country, without VC money, or have cheap labor/g"
500 USD a month would get you approximately the following resources (taxes vary) on the aforementioned platforms:
Contabo
Nodes: 15 to 83 (depending on configuration)
CPU: 150 to 332 cores
RAM: 664 to 900 GB
SSD: 4150 to 6000 GB
Hetzner
Nodes: 7 to 110 (depending on configuration)
CPU: 86 to 192 cores
RAM: 192 to 384 GB
SSD: 2200 to 4800 GB
(this includes regular VPS packages, not storage optimized ones, or dedicated hosting etc.)
> Otherwise what you have is a budget that is lower than the possible implications of temporary downtime. That doesn't make sense in the real world.
This (depending on the circumstances) does sound like a good point! Maybe "the real world" isn't the best wording, though, and choosing "enterprise settings" or anything along those lines would be more suitable.
> Maybe "the real world" isn't the best wording, though, and choosing "enterprise settings" or anything along those lines would be more suitable.
This is the point - for-profit corporations, by definition, don't want to waste money, and if "high reliability" isn't required (or they don't know about it), they don't waste money. Most of the time, they don't waste the money.
However, if "being cheap" would means billions in lost income (and they know about it), they really want to have reliable, redundant infrastructure and systems around.
But it leads me back to my original statement - extreme requirements for uptime don't come out of nothing.
If you're in a location where IT related labor is extremely cheap - you're just going to have people keep one server up.
I know I used to do exactly that, because the server was more than my annual income. But that didn't last long. After the first 20 minute downtime, the budget for HA solution was allocated. But before a certain point downtime wasn't expensive.
Non-profits would probably be the only reasonable exception, where HA and low budgets could coincide. Otherwise - nah...
When you have a giant monolith with the "load the world" endpoint, it can be tricky to pinpoint the the "load the world" endpoint (or, as is often the case, endpoint*s*) is what's causing issues. Instead, everyone just tends to think of it as "the x app having problems."
When you bust the monolith into the x/y/z, and x and z got the "load the world" endpoints, that starts the fires of "x is constantly killing things and it's only doing this one thing. How do we do that better?"
That allows you to better prioritize fixing those scaling problems.
The silly thing is that something like the JVM (which we use) really wants to host monoliths. It's really the most efficient way to use resources if you can swing it. The problem is when you give the JVM 512GB of ram, it hides the fact that you have a module needlessly loading up 64gb of ram for a few seconds. Something that only comes to a head when that module is ran concurrently.
Split up services where it makes sense and don't over do it. That's how I design my projects.
So if a database service is not available you simply return stale, cached data until the service is back up.
I can see an advantage regarding resource hogging, but the flip side is the extra point of failure of network calls in microservices.
Not saying which is better, but deployment is orthogonal to logical dependence and correctness.
For instance, in a shopping site, why should a crash in the recommendations engine result in a non-functional webpage (rather than a working purchase page with no recommendations)?
Personally I think microservices start to make sense when you have several hundred developers (an environment I'm currently keen to never enter again - $work has 5 devs and might one day have 6).
I agree that microservices are suitable for large organization, where the organization practically has multiple products (which could be purchased from a vendor or sold to another company).
You're going to have to explain what you mean by that a bit more... You surely cant mean it as it is written.
If the recommendations team write code that causes the OOM-killer to end their process, making them run it on separate infrastructure insulates your "main store team" from the bugs they write.
I imagine his logic was something like: "How can OOMs happen less often if you run more processes (possibly on the same machine)?", while your comment actually wants to say: "if a specific service is affected by an OOM, with microservices only that specific microservice goes down, since it's probably running on its own hardware".
Networks could have unexpected delays, routing errors and other glitches. At least with a monolith you can often find a stacktrace for debugging. I have seen startups that have limited traceability and logging when using micro services.
When a small startup has to manage "scalable" K8s infrastructure in the cloud, distributed tracing and monitoring is often not prioritized when you are a team of 5 developers trying to find a product market fit.
I am not against microservices (I work with them daily) but you just trade one type of stability problem with another
I love working with microservices at the scale of $WORK, but we're Big Tech. I can't imagine why a 5 person startup would want k8s and microservices. You don't need that scale until you have more than 2 teams, and you're pushing at the very least 15 engineers at that point and usually the sales and marketing staff to make that investment worth it.
There is a point when it all starts to make sense. But that point is when you go into billions worth business, hundreds of devs etc. And going there has large cost, especially for small/medium systems. And that cost is not one off - it's a day-to-day cost of introducing changes. It's orders of magnitude chaper and faster (head cound wise) to do changes in ie. single versioned monorepo where everything is deployed at once, as single working, tested, migrated version than doing progressive releases for each piece keeping it all backward compatible at micro level. Again - it does make sense at scale (hundreds of devs kind of scale), but saying your 5 devs team moves faster because they can work on 120 micoservices independently is complete nonsense.
In other words micoservices make sense when you don't really have other options, you have to do it, it's not good start-with default at all; and frankly Sam Newman says it in "Building Microservices" and so do people who know what they're talking about. For some reason juniors want to start there and look at anything non-microservice as legacy.
It sort of is.
It's not a perfect world. One issue with monoliths is that, organizations like to take a "if it ain't broke, don't fix it" attitude towards things. Unfortunately, that leads to spotty service and sometimes expensive deployments. Those aren't always seen as being "broken" but just temporary problems that if you pull enough all nighters you can get through regularly.
It takes a skilled dev to be able to really sell a business on improving monoliths with rework/rewrites of old apis. Even if it saves money, time, all nighters. It's simply hard for a manager to see those improvements as being worth it over the status quo. Especially if the fact of running the monolith on big iron masks the resource usage/outages caused by those APIs.
That often is, because new solitions tend to come with their own problems.
It depends on what you are scaling. I think microservices are fundamentally more scalable for deployment, since changes can be rolled out only to the services that changed, rather than everywhere. Unless your language and runtime support hot-loading individual modules at runtime.
Admittedly, it isn't a scalability problem you will run into right away.
But when you need to roll out an emergency fix, there is a big difference between deploying to thousands of servers that all have everything, and ten servers running a single service.
The solution to this is to use the right tool, a build system that supports monorepos like Bazel. Bazel solves this problem wonderfully. It only builds / tests / containerizes / deploys (rules_k8s, rules_docker) what needs to be rebuilt, retested, recontainerized, and redeployed. Builds are much faster, developers have God like visibility to all of an organizations' code, and can easily grep the entire code base and be assured their changes do not break other modules if Bazel test //... passes. It is language agnostic so you can implement your services in whatever language best suits it. It allows you to more easily manage transitive dependencies, manage versions globally across your org's codebase.
Of course Bazel has a steep learning curve so it will be years before it is adopted as widely as Maven, Gradle etc. But in the banks I've worked at it would've saved them tens of millions of dollars.
Also git would need to catch up to large codebases. I think Meta released a new source control tool recently that is similar to git but could handle large monorepos.
We keep object definitions in a separate repo to not duplicate them and generate a maven dependence from that. Pretty simple actually. We mainly run on Java thought. Would be more complicated when there are multiple languages.
Protobuf or Json schema could come to the rescue if needed.
> I disagree, in my opinion micro-services hinder scalability of deployment
…and of anything related to testing:
- Want to fire up the application in your pipeline to run E2E tests? Congratulations, you must now spin up the entire landscape of microservices in k8s. First, however, you need to figure out which versions of all those microservices you want to test against in the first place, since every service is living in a separate repository and thus getting versioned separately.
- Want to provide test data to your application before running your tests? Well, you're looking at 100 stateful services – good luck with getting the state right everywhere.
Not at all. You can run your monolith on multiple nodes just as you can do with microservices. If anything, it scales much better as you reduce network interactions.
200 threads, 12TB of RAM, with a pipe upwards of 200GB/s. This isn't even as big as you can go, this is a reasonable off the shelf item. If your service doesn't need more than this, maybe don't break it up. :)
I believe that this level of service can no longer accurately be described as "micro".
Ruby, Python, PHP and Node.js startups have to figure out how to use the cores while C++, Rust, Erlang and Go have no issues running a single process that maxes out all 64 CPU cores.
So it does not work for FAANG companies but what do I care. I have the rest of the world to play with ;)
As long as hype chasers in middle management don't get in the way after convincing themselves they too must be like FAANG with a few orders of magnitude less of a consumer base.
They want to be both the architect and the manager and anything you say would be over ruled and since they are the boss its hard to ignore them.
This service is a monolith because it has 10K code and it needs to be broken up.The product is at MVP and its rock solid on Java Spring and it hardly crashes. We are never going to lose data based on the design choices we made. None of that matters.
We need zero down time upgrade, when we had zero customers.
Instead, if you build an architecture that can scale the number of corporate users by adding cheap $2k pizza box 1u servers as the company grows, that's much more attractive. Also, you can keep your systems design flexible enough to recompile and take advantage of advancements in hardware tech every 18 months – this gives you better operating margins as your own business starts to grow.
Let alone - AWS Lets you get that machine for less than a junior engineer salary ($7.5 per hour, roughly equivalent to $32 of hourly wage = $67k)
Network latency is real, and some systems really do run on a tight latency budget, but most sane architectures will just do a couple of calls to a database (same performance as monolith) and maybe a transitive service call or something.
Little big concept or mono-micro people like the call it is where it is at. Spending too much time making a complex environment benefits noone. Spending too much time making a complex application benefits noone.
Breaking down monoliths into purposed tasks and then creating pod groups small containers is where it is at. Managing a fleet on kubernetes is easier than managing one in some configuration management stack. Be it puppet or salt. You have too many dependencies. Only your core infrastructure kubernetes should be a product of the configuration management.
I don't think you actually understand what microservices are. You don't put a load balancer to load balance between different services. A load balancer balances trafic between servers of the same service or monolith.
Microservices mean the servers of different services run different code. A load balancer only works together with servers running the same code.
Uh - what?
>A load balancer balances traffic between servers
Correct.
> of the same service or mononlith
Incorrect.
Load balancers used to work solely at Layer 4, in which case you’d be correct that any 80/443 traffic would be farmed across servers that would necessarily need to run the same code base.
But modern load balancers (NGINX et al) especially in a service mesh / Kubernetes context, balance load at endpoint level. A high traffic endpoint might have dozens of pods running on dozens of K8s hosts having traffic routes to them by their ingress controller. A low traffic endpoint’s pod might only be running on few or one (potentially different) hosts.
The load balancer internally makes these decisions.
I hope this clears up your misunderstanding.
The split is inessential. You can just as easily have homogeneous backends & one big load balancing pool. Instances within that pool can even have affinity for or ownership of particular records! The ability to load balance across nodes is not, as you claimed, a particular advantage of microservices.
Are you replying to the right comment? I made no such claim.
The point is, if a client makes a request to a server, the response should always be the same, no matter where the load balancer sends the request to. Which means it should run the same code.
Nginx doesn't even mention route based or endpoint based load balancing in their docs. Maybe they don't consider it load balancing either.
Describes exactly what I’m talking about.
Just because Nginx is doing it, doesn't mean it's load balancing. It's an extra function tacked onto a load balancer.
I've almost never seen situations where a single request would need more resources available, than the entire server has (outside of large GPT models for text, though maybe that's because I couldn't afford beefy machines for that myself).
Instead, if your monolith needs X resources to run (overhead) and Y resources to serve your current load, then in case of increased load you can just setup another instance of your monolith in parallel with another set of X + Y resources (same configuration) and it will generally almost double your capacity.
Now, there can be some issues with this, such as needing to ensure either stateless APIs or sticky sessions, but both are generally regarded as solved problems (with a little bit of work). Monoliths themselves shouldn't be limited to running just a single instance and aren't that different from a scalability perspective than microservices.
Where microservices excel, however, is that you can observe individual services (e.g. systemd services or them running in containers) better and see when a particular service is misbehaving or scale them separately, as well as decrease that X overhead since each service has a smaller codebase when you have lots of instances running. This does come at the expense of increased operational complexity and possibly noisy network chatter, especially if you've drawn your service boundaries wrong.
However, at the same time I've seen actual monoliths that can never have more than one instance running due to problematic architecture, so therefore I propose the following wording (that I've heard elsewhere):
SINGLETONS - a monolithic application that can ever only have a single instance running, for example, when business processes are stored in memory for a bit, or have user sessions or something like that stored locally as well; these will ONLY ever scale VERTICALLY, unless you re-architect them
MONOLITHS - applications that contain all of your project's logic in a single codebase, although multiple instances can be launched in parallel, depending on your needs; can be scaled BOTH VERTICALLY and HORIZONTALLY; they have more overhead though and observability can be a bit challenging
MICROSERVICES - applications that contain a part of the total project's logic, typically across multiple separate codebases, possibly with shared library code, pieces of your project can be scaled separately, BOTH VERTICALLY and HORIZONTALLY; they are operationally more complex, can involve more network chatter and while you can observe how services perform, now you need to deal with distributed tracing
I don't actually expect anyone to use these particular terms, but I dislike when someone claims that monoliths have the issues of these "singleton" applications when in fact that's just because they've primarily worked with bad architectures. Sometimes they wouldn't need to shoot themselves in the foot with microservices if they could just extract their session data into Redis and their task queues into RabbitMQ. Other times, microservices actually make sense.
All of the catalog data was moved to a service which only served catalog data so its cache was optimized for catalog data and the load balancers in front of it could optimize across that cache with consistent hashing.
This was different from the front end web tier which used consistent hashing to pin customers to individual webservers.
For stuff like order history or customer data, those services sat in front of their respective databases and provided consistent hashing along with availability (providing a consistent write-through cache in front of the SQL databases used at the time).
I wouldn't call those areas where it makes things more efficient rare, but actually common, and it comes from letting your data dictate your microservices rather than letting your organiation dictate your microservices (although if teams own data, then they should line up).
But I don't have to now.
Cache locality starts making a difference when you have over ten servers, and a significant fraction of the data is cacheable.
For "transactional" Enterprise CRUD apps, often there is very little that can be cached due to consistency concerns.
I've seen many systems where there are a few hundred to a few thousand users, not hundred of millions or billions of users. There can also be teams of 5-10 developers who manages +20 microservices. I still don't think those projects have the same needs and could have done something else.
I'll agree that a team of developers with 20 microservices sounds obviously wrong (which is really my point that the site should be divided around data and the orgs should probably reflect the needs of the data--not the other way around). And Amazon back then didn't have that problem.
Aren't you still writing out to external data stores? If that's the case then its really not a comparison of in process to a RPC, it just two RPC hops now, no?
So instead of a single DB transaction, often with a single SQL roundtrip, microservices can sometimes force through coordination of separate transactions to several DBs.
It might still be completely viable to rewrite something so that it's only 10% as efficient but you can scale it horizontally easily.
On top of that, the tendency to get complacent with unstructured data in a lot of systems is really creating a very complicated lock-in when systems are developed for unique services on each cloud provider... Bowls of spaghetti.
Hire Solutions Architects for dev projects, make your apps future proof... I warn you. Too much microservice customization leads to vendor lock in and expensive operational costs... This is why a lot of apps get sunsetted early.
Been there: how do you handle schema changes?
One of the advantages that having a separate schema per service provides is that services can communicate only via APIs, which decouples them allowing you to deploy them independently, which is at the heart of microservices (and continuous delivery).
The way I see it today: everyone complains about microservices, 12 factor apps, kubernetes, docker, etc., and I agree they are overengineering for small tools, services, etc., but if done right, they offer an agility that monoliths simply can't provide. And today it's really all about moving fast(er) than yesterday.
Our data is mostly append-only, or if it's being changed, there is a theoretical final "correct" version of it that we should converge to. So to "get" data, you subscribe to messages about some state of things, and then each service is responsible for managing its own copy in its own db. This worked well enough until it didn't, and we had to start doing true-ups from time to time to keep things in sync, which was annoying, but not particularly problematic, as we design to assume everything is async and convergent.
The optimization (or compromise) we decided on, was that all of our services use the same db cluster, and that if the db cluster goes down, it means everything is down. Therefore, if we can assume the db is always up, even if a service is down, we consider it an acceptable constraint to provide a readonly view into other services db. Any writes are still sent async via MQ. This eliminates our syncing drifting problem, while still allowing for performant joins, which http apis are bad at and our system uses a lot of.
So then back to your original question, the way that this contract can break is via schema changes. So for us, since we use postgres, we created database views that we expose for reading. And postgres view updates are constrained that they must always be backwards compatible from a schema perspective. So then now our migration path is:
- service A has some table of data that you like to share
- write a migration to expose a view for service A
- write an update for service B to depend upon that view
- service B now needs some more data in that view
- write a db migration for service A that adds that missing data, but keeping the view fully backwards compatible
I don't think I understand. You need to update (and deploy) service B every time you perform a view update (from service A), although it's backward compatible?
If you don't need the new column, then you don't need to do anything on service B, because you know that existing columns on the view won't get removed and their type won't change. You only need to make changes on service B when you want to take advantage of those additions to the view.
But hey, every team and company has to find their strategy to do things. If this works for you, that's great!
It's just not a microservice by definition.
And yes, that is correct, we agree that once we expose a view, we won't remove columns or change types of columns. Theoretically we could effectively deprecate a column by having it just return an empty value. Our use cases are such that changes to such views happen at an acceptable rate, and backwards incompatible changes also happen at an acceptable rate.
Our views are also often joins across multiple tables, or computed values, so even if it's often quite close to the underlying tables, they are intentionally to be used as an abstraction on top of them. The views are designed first from the perspective of, what form of data do others services need?
And the fact you must keep backward compatibility ( see the OP answer above) at the implementation level shows how fragile this approach is - you will never be able to change a database schema as you wish just because you have consumers that rely on the internal details of your implementation - a table. If you want to change a field from char to int, you can't. How is it important for service B to know that level of detail? An API could still expose a domain model as char, if you want to, and maybe introducing new fields, new methods, whatever way. Or maybe nothing at all, maybe it's not necessary because the database field is never exposed but only used internally (!!).
On the other hand, if you expose a database agnostic API (e.g., http, rpc, ... whatever) you can even swap the underlying database and nobody will notice.
A good rule of thumb is: if I change the implementation, do I need to ask/tell another team? If the answer is yes, that is not a microservice.
something like this
CREATE VIEW table_read_api AS SELECT TO_CHAR(now_int) as was_char FROM the_table;
not sure that the view is an internal structure because it will be exposed via API as it is.
SQL allows to swap the database at least if no specific SQL features are used.
Which proves my point: if I change a field type ( in the table ), I will have to change the view type. I need to change 2 services because one of them changed an implementation detail.
CREATE TABLE the_table (was_char CHAR);
CREATE VIEW table_read_api AS SELECT was_char FROM the_table;
a client consumes data via:
select was_char from table_read_api;
after the change
alter table the_table modify was_char int;
alter table the_table rename column was_char to now_int;
CREATE VIEW table_read_api AS SELECT TO_CHAR(now_int) as was_char FROM the_table;
the client uses the same query
select was_char from table_read_api;
the type of the column has not changed
it is still a char
only "internal implementation" is changed
And any benefit of a microservice owning it's own rDB is still, that schema changes aren't easily reversible. Specially when new, non predefined, data has been flowing in.
Stateless microservices are great, in the sense that you don't have to build multiple versions of APIs... but stateful microservices are just a PITA.
There's a third technical problem that microservices solve, and it's my favorite: isolation. With monoliths, you provide all of your secrets to the whole monolith, and a vulnerability in one module can access any secret available to any other module. Similarly, a bug that takes down one module takes down the whole process (and probably the whole app when you consider cascading failures). In most mainstream languages, every module (even the most unimportant, leaf node on the dependency tree) needs to be scoured for potential security or reliability issues because any module can bring the whole thing down.
This isn't solved at the language level in most mainstream languages. The Erlang family of languages generally address the reliability issue, but most languages punt on it altogether.
> The real reason that microservices may make sense is because they keep people honest around module boundaries.
Agreed. Microservices, like static type systems, are "rails". Most organizations have people who will take shortcuts in the name of expedience, and systems with rails disincentivize these shortcuts (importantly, they don't preclude them).
And for that matter it can (but not necessarily) limit how much damage a bad code release can do.
Of course you don't get those benefits for free just by using microservices, but impleminting those kind if boundaries in a monolith is a lot harder.
Microservices aren't inherently more secure.
Maybe it is possible with some low-level system calls, but at the very least it is a lot more difficult than using separate VMs or containers.
Separate process vaults, HSMs and other techniques of offloading security credentials - are the same for microservice architecture, as "monolith".
The implication that anything other than a microservice architecture must be exclusively an uber executable doing literally everything, is naive.
How do give one module access to the vault/HSM without also giving any other code in the same process access? Even in the event of a security compromise. And that still doesn't address the problem of a vulnerability anywhere in the monolith potentially exposing any sensitive data in processes memory (such as user data including password).
> The implication that anything other than a microservice architecture must be exclusively an uber executable doing literally everything, is naive.
Ok, replace "microservices" with "service oriented architecture". I'm comparing specifically against a monolith where you have an "uber executable doing literally everything".
I doubt that anyone here is advocating for a system that it literally everything in one OS process. I bet even you would say that a process with a RDMBS connection is a monolith, even though it doesn't fit your definition.
As for modules - they can run as a separate process on the same machine, to isolate security critical elements.
Conversely, I doubt that any reasonable microservices architecture has every instance of a microservice in its own subnet with a firewall and strict network access permissions... including one off generated API keys.
Most, at best, use a static API key per microsoervice inside one large "secure" network... which leads me back to my point - they're not exactly easier to secure. Methods may differ a little bit, but the techniques are the same.
Then your example of a buffer overflow is going to be as bad for microservices, as for monolith.
Unless you're going to invest in a variety of systems, programming languages and OSes in your stack - your one buffer overflow, turns into buffer overflow on every other microservice... making your claim of "easier to isolate" a little bit delusional. (Classic example of Log4J bug, where monolith or microservices - once you're breached "game over")
It is about striking a balance. No reason should something be overly compounded or overly broken up.
I gave a talk [1] about scalability of Java systems on Kubernetes, and one of my recommendations is that Java-based systems - or any system on a runtime similar to the JVM, like CLR or even Go - should be scaled diagonally. Efficiency is the word.
While horizontal scaling can easily address most performance issues and load demand, in most cases it is the least efficient way for Java (and again, I risk saying .NET and Go), as these systems struggle with CPU throttling and garbage collectors.
In short, and exemplifying: one container with 2 CPUs and 2GB of RAM will allow the JVM to perform better, in general, than 2 containers with 1 CPU/1GB RAM each. That said, customers shouldn't be scaling horizontally to any amount more than what is adequately reasonable for resiliency, or unless the bottleneck is somewhere like disk access. For performance on the other hand, customers should be scaling vertically.
And Kubernetes VPA is already available. People just need to use it properly and smartly. If a Kubernetes admin believes a particular pod should double in number of replicas, the admin should consider: "would this microservice benefit even more from 1.5x more CPU/Memory than 2x more replicas?" and I bet to say that, in general, yes.
My default recommendation has always been to make instances "as big as possible, but no bigger". You may need 3 instances for redundancy, fault tolerance and graceful updates but past that you should probably scale up to near the size of your machine. There are obviously lots of complications and exceptions (for example maybe using most 3 instances uses 90% of your machines so you can't bin pack other processes there so it is better to use 4 instances at 70% as the machines will be used more efficiently) but bigger by default is generally a good option.
Exactly.
But I am seeing more and more of the opposite: make instances as small as possible, and scale out (not up) if/when needed.
Making data available to a thread on the same core has exceptionally high bandwidth. Just that alone will help with scaling.
If a team wants to try out a different language, or hosting model, or even just framework/tooling, those things can be really hard to do within the same codebase; much easier when your only contract is handling JSON requests. And if your whole organization is totally locked into a single stack, it's hard to keep evolving on some axes
(I'm generally against microservices, but this is one of the more compelling arguments I've heard for them, though it still wouldn't mean you need to eagerly break things up without a specific reason)
Not all engineers need to move around - DS and MLEs are generally useless on the frontend, and vice versa.
Microservices allow your permissions to be clear and precise. Your database passwords are only loaded into the process that uses them. You can reason about "If there's an RCE in this service, here's what it can do". Trying to tie that to monoliths is hard and ugly.
I would like someone to spell this out. It seems to me people are claiming that if a single binary serves some CPU-bound requests and some memory-bound requests, and you give it more memory, then the memory gets "wasted" on the CPU-bound part. Or if you give it more CPU, the CPU gets wasted on the memory-bound part. But this kind of assignment of resources to code paths seems to be a consequence of microservices. In a single computer, single binary situation resources should not get used up unless the workload actually wants to use them. A compute-heavy thread doesn't cost heap. A big heap doesn't slow down a compute-heavy thread. What am I missing?
My IDE should be able to easily traverse the call graph. My development environment should be simple to setup.
I’ve worked on microservices that required an insane amount of boilerplate to do simple things. Like 7 layers of controllers, clients, services, data services, etc, just to fetch a simple piece of data. And the developer experience of running dozens of services in a Kubernetes cluster running on my dev machine was awful.
Does anything like this exist?
I only dabbled many years ago but Erlang/OTP comes to mind.
And tRPC for TypeScript calls in browser and server.
It turns out that "transparent RPC" is basically a contradiction in terms. As soon as you start doing things across process boundaries, and even more so across network boundaries, it requires a very different approach for API design - something that's very cheap locally, like passing objects by reference, becomes expensive and full of footguns.
If you reduce the feature set to the point where it can be transparently mapped to either local or RPC - which is, basically, function calls processing and returning data organized into arrays & trees (but not graphs) - there's still the issue that RPC has so many more failure points that you have to handle that would never light up in local.
This is all still doable; I have my doubts about practical the end result would be, though.
Totally fine to support a limited set of primitives and collections passed by value, and require developers consider the failure modes.
In my experience HTTP+JSON or gRPC are usually lacking in at least one of those respects. If you constrain all the services to be the same language it becomes easier.
> https://github.com/NathanRSmith/lib-courier-js
Interestingly, we're pursuing a monorepo & multi-monolith setup for the next version of our platform. So lib-courier is no longer necessary to stitch it all together. It was fun while it lasted though. Once you understood the routing algo & code patterns, lots of stuff "just worked".
This experience has strongly impacted my view of microservices and for all personal projects I will develop in the future I will stick with a monolith until much later instead of starting with microservices.
How the hell... like... who decided to do Microservices in the first place if they didn't know this? This is such a rookie mistake. It's like somebody right out of high school just read on a blog the new way is "microservices" and then went ahead with it.
And both are way more common than they should be.
Microservices can have both design utility and simultaneously been a major fad.
Lurking reddit and HN you can watch development fashions come and go. It's really really profoundly hard to hold a sober conversation on splitting merits from limitations in the middle of the boom.
tl;dr: gartner_hype_cycle.png
In a start-up, having a team of young people will allow you to move fast, pivot and deliver. What you usually end up with though, is a tower built from spaghetti that's serving actual users. I see this again and again and then people get surprised that the software doesn't scale.
But... yeah. A colleague (25+ years) has to deal with this more than I do, but have hit it some over the years, and it happens more the older I get.
Often he or I may be 'right' but only because of existing constraints in a system. Someone with a different view may even be 'correct' in a nuts-and-bolts technical sense. Yes, A is 8% faster than B. But... we have an existing system in B that meets our current needs, and team who know B, have written it, and maintained it for a couple years. Switching tech stacks to something no one knows for an 8% network speed boost is not remotely helpful.
My experience was also that the young manager was very suspicious of the old fart. I've never had any problem working for managers younger than me; but some young managers don't know what to make of a team member that knows more than they do.
[0] No insult intended; almost everything I learned after the first 10 years, I learned from people younger than me.
Not trying to be harsh here, but not expecting an increase of network call in a system where each component is tied together with... network calls sounds a bit naive.
> We are now attempting to solve this with caches and doing batch requests
So you have built a complex and bottle-necked application for the sake of scalability, then having to add caching and batching just to make it perform? That sounds like working backwards.
Obviously, I have no clue on the scale of the project you are working on, but it sure sounds like you could have built it as a monolith in half of the time with orders of magnitude more performance.
Scalability is a feature, you can always add it in the future.
Yes, if you can't scale fast enough as you need to, it can hurt your business. Not being able to keep up with demand is a (luxury) problem that every business faces, not just in tech. They would often be called 'growing pains' in a business, and though they are bad, they rarely contribute to the failure of a company.
Starting a startup/service/platform with microservices before you even understand the bottlenecks/market fit/customers is usually not a good idea. You can come a very, very long way with a monolith before you hit performance and scalability limits. And once you do, you can always start breaking things up into smaller services for scalabity. Obviously you need to make sure you are scaling on time to keep up with demand.
'Nail it, then scale it', and 'premature optimization is the mother of all f-ups' are popular sayings for a reason.
> I was told ~1200 RPCs independently by several engineers at Twitter, which matches # of microservices. The ex-employee is wrong.
> Same app in US takes ~2 secs to refresh (too long), but ~20 secs in India, due to bad batching/verbose comms.
RPCs are on the server side. Why would they app take longer to refresh in India than in the US?
Some more explanation:https://twitter.com/mjg59/status/1592380440346001408
Let each microservice owner figure out how to achieve their latency+reliability SLA's in every location - whether replicating datastores, caching, being stateless, or proxying requests to a master location.
At my last job we created a whole fleet of microservices instead of a single modular project/repo. Some of them required non-trivial dependencies. Some executed pretty long-running tasks or jobs for which network latency is insignificant and will remain so by design. Some were few pages long, some consisted of similar-purpose modules with shared parts factored out. But there was no or little processes like “ah, I’ll just ask M8 and it will ask M11 and it will check auth and refer to a database. I.e. no calls as trivial as foo(bar(baz())) but done all over the infra.
(Because this is at the same time one of the defining elements of this architecture... and the first one to be opted out when you actually start using it "for real").
Lots of ways to solve a problem, though.
The primary reason for this is that PDFs can contain executable code and the common tools used to process them are full of unpatched CVEs.
Imagine if you are responsible, at runtime, for linking object files (.o) where each object is a just the compilation of a function.
Now why would anyone think this is a good idea (as a general solution)? Because in software organizations, the “linker’s” job is supposed to be done by the (“unnecessary weight”) software architect.
Microservices primarily serve as a patch for teams incapable of designing modular schemas. Because designing schemas is not entry level work and as we “know” s/e are “not as effective” after they pass 30 years of age. :)
> monolith
Unless monolith now means not-microservice, then be aware that there are a range of possible architectures between a monolith and microservices.
For this reason, I've been trying to push for building a monolithic app first, then splitting into components, and introducing libs for common functionality. Only when this is all done, you think about the communication patterns and discuss how to scale the app.
Most microservice shops I've been in have instead done the naïve thing; just come up with random functionally separate things and put them in different micro services; "voting service", "login service", "user service" etc. This can come with a very very high price. Not only in terms of network traffic, but also in debuggability, having a high amount of code duplication, and getting locked into the existing architecture, cementing the design and functionality.
And it is reassuring to hear that you seem to have success in avoiding these issues with a monolithic architecture, as I thought I was oldschool for starting to prefer monoliths again.
The main thing is that regardless of scaling, the app should always be able to run/debug/test locally in a monolithic thing.
Once people scale they seem to abandon the need to debug locally at their peril.
Scaling should just be a process of identifying hot function calls and when a flag is set, to execute a call as a network rpc instead.
[1] https://scholar.harvard.edu/waldo/publications/note-distribu...
I agree with sentiment of the paper in that it can never be fully transparent. When I say "it should just be a process of identifying hot function calls..." this is admittedly a little contrived - there would need to be a more thorough analysis.
The main point seems to be that partial failure is too difficult to handle without explicitly declaring some objects as "remote".
> Merging the models by making local computing follow the model of distributed computing would require major changes in implementation languages (or in how those languages are used) and make local computing far more complex than is otherwise necessary
Given that the paper is from 1994, I'm wondering if there have been language changes that make this more achievable.
What comes to mind is the event loop. The fact that in Node.js, almost every function call is now an asynchronous promise due to the non-blocking paradigm coupled with the convenience of async/await syntax. These event loop runtimes provide a universal way to hook into asynchronous calls.
I would imagine that in 1994, envisioning the code added to the languages of the day needed to handle timeout and asynchronous invocations would have felt overwhelming.
Also thinking about ORMs and databases. It seems TOPLink for SmallTalk was the first, released in 1995 after this paper was. ORMs seem kind of like "transparent remoting", as they usually can run against an in-memory database or db on a local system, or over an TCP connection, which then needs to deal with latency and failure. And for any app talking to a DB with an ORM, means that the failure cases of the ORM must prevade throughout the stack.
> Given that the paper is from 1994, I'm wondering if there have been language changes that make this more achievable.
As the paper argues, it is not possible to make remote calls look like local ones. But it is trivially possible to make local calls look like remote ones! Use call-by-value parameter passing, and make all call asynchronous. The most successful solution I have heard of is Erlang's immutable-message-passing model.
It has been many years since I used it, but I think TOPLink Session and UnitOfWork interfaces threw exceptions signaling remote execution. This is like Erlang - you work with remote APIs; local execution is a special case.
I first wrote a program which ran on a web server about 25 years ago. In that time, computers have experienced about ten doublings of Moore's law, i.e. are now over a thousand times faster. Computers are very fast if you let them be.
Either way, one of my biggest pet peeves is the near-ubiquitous use of HTTP & JSON in microservice architectures. There's always going to be overhead in networked servies, but this is a place where binary protocols (especially ones like Cap'n Proto) really shine.
I wouldn't call that entirely unexpected. :-) It's a rather well-known issue:
Microservices
grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too
seem very confusing to grug
> especially since some of these services are not even in the same data center.
I think you need to answer why? If you can't put all of the services in one data center, then by definition you can't write a monolith either. If the monolith would happily run in one datacenter, then you should have all instances of your microservices in that one datacenter.
It surprised me that you would conclude that this is a problem with microservices. It's like if a particularly architect always punches you in the groin in every meeting, and you've concluded that architects are bad people, rather than this one architect is a bad person.
Why is the other service in another data center? Does it need to be in another data center? If it does, how will a monolith help?
* they force alignment on one language or at least runtime
* they force alignment of dependencies and their versions (yes, you can have different versions e.g. via Java classloaders, but that's getting tricky quickly, you can't share them across module boundaries, etc.)
* they can require lots of RAM if you have many modules with many classes (semi-related fun fact: I remember a situation where we hit the maximum number of class files a JAR could have you loaded into WebLogic)
* they can be slow to start (again, classloading takes time)
* they may be limiting in terms of technology choice (you probably don't want ot have connections to an RDBMS and Neo4j and MongoDB in one process)
* they don't provide resource isolation between components: a busy loop in one module eating up lots of CPU? Bad luck for other modules.
* they take long to rebuild an redeploy, unless you apply a large degree of discipline and engineering excellence to only rebuild changed modules while making sure no API contracts are broken
* they can be hard to test (how does DB set-up of that other team's component work again?)
I am not saying that most of these issues cannot be overcome; to the contrary, I would love to see monoliths being built in a way where these problems don't exist. I've worked on massive monoliths which were extremely well modularized. Those practical issues above were what was killing productivity and developer joy in these contexts.
Let's not pretend large monoliths don't pose specific challenges and folks moved to microservices for the last 15 years without good reason.
How is this possibly a down-side from an org perspective? You don't want to fracture knowledge and make hiring/training more difficult even if there are some technical optimizations possible otherwise.
E.g. if you end up having a requirement to add some machine learning to your application, you might be better off using Tensorflow/PyTorch via Python than trying to deal with it in whatever language the core of the app is written in.
Take an RDBMS as an example, that is a service external to the monolith often written in a very different language to the monolith.
This is the Pendulum Swing all over again. If one language and runtime is limiting, forty is not liberating. If forty languages are anarchy, switching to one is not the answer. This is in my opinion a Rule of Three scenario. At any moment there should be one language or framework that is encouraged for all new work. Existing systems should be migrating onto it. And because someone will always drag their feet, and you can’t limit progress to the slowest team, there is also a point in the migration where one or two teams are experimenting with ideas for the next migration. But once that starts to crystallize any teams that are still legacy are in mortal danger of losing their mandate to another team.
these are not maxims of development, there can be reasons that make these consequences worth it. Furthermore you can still use just a single language with microservices*, nothing is stopping you from doing that if those consequences are far too steep to risk.
*:you can also use several languages with modules by using FFI and ABIs, probably.
You can have modules implemented in different languages and runtimes. For example you can have calls between Python, JVM, Rust, C/C++, Cuda etc. It might not be a good idea in most cases but you can do it.
Lots of desktop apps do this.
A sane thing to do.
>they force alignment of dependencies and their versions
A sane thing to do. Better yet to do it in a global fashion, along with integration tests.
>they can require lots of RAM if you have many modules with many classes
You can't make the same set of features build in a distributed manner comsume _less_ RAM than the monolith counterpart. Given you're now running dozens of copies of the same java vm + common dependencies.
>they can be slow to start
Correct.
>they may be limiting in terms of technology choice
Correct.
>they don't provide resource isolation between components
Correct.
>they take long to rebuild an redeploy, unless you apply a large degree of discipline and engineering excellence to only rebuild changed modules while making sure no API contracts are broken
I think the keyword is the WebLogic Server mentioned before. People don't realise that monolith architecture does't mean legacy technology. Monolith web services can and should be build in Spring Boot, for example. Also, most of the time, comparisons are unfair. In all projects i've worked im yet to see a MS instalation paired feature-wise with his old monolith cousin. Legacy projects tends to be massive, as they're made to solve real world problems while evolving during time. MS projects are run for a year or two and people start to compare around apples to oranges.
>they can be hard to test
If other team's component break integration, the whole building stops. I think Fail-Fast is a good thing. Any necessary setup must be documented in whatever architectural style. It can be worse in a MS scenario, where you are tasked to fix a dusty, forgotten service with an empty README.
If anything, monolithic architecture brings lots of awareness. It's easier to get how things are wired and how they interact together.
imaging your application contains of two pieces - somewhat simple crud, that requires to respond _fast_ and huge batch processing infrastructure, that needs to work as efficient as possible, but doesn't care about single element processing time. And suddenly 'the sane thing to do' is not the best thing anymore. You need different technologies, different runtime settings and sometimes different runtimes. But most importantly they don't need constraints imposed by unrelated (other) part of the system.
>A sane thing to do. Better yet to do it in a global fashion, along with integration tests.
But brutally difficult at scale. If you have hundreds of dependencies, a normal case, what do you do when one part of the monolith needs to update a dependency, but that requires you update it for all consumers of the dependency's API, and another consumer is not compatible with the new version?
On a large project, dependency updates happen daily. Trying to do every dependency update is a non-starter. No one has that bandwidth. The larger your module is, the more dependencies you have to update, and the more different ways they are used, so you are more likely to get update conflicts.
This doesn't say you need microservices, but the larger your module is, the further into dependency hell you will likely end up.
This is incredibly subjective, and contingent on the size and type of engineering org you work in. For a small or firmly mid-sized shop? yea I can 100% see that being a sane thing to do. Honestly a small shop probably shouldn't be doing microservices as a standard pattern outside of specific cases anyway though
As soon as you have highly specialized teams/orgs to solve specific problems, this is no longer sane.
And yet, that is exactly what gets done
or maybe run redundant monolith fail over servers. should work the same as micro services.
Microservices don't make builds radically faster for the majority. People still split systems into larger services.
So it seems like we’re trying to compensate bad design with microservices. It’s orthogonal IMO.
> we’re trying to compensate bad design
No I think we're trying to compensate for developer mistakes and naivety. When you have dozens to hundreds of devs working on an application many of them are juniors and all of them are human and impactful mistakes happen. Just catching the right exceptions and handling them the right way does not protect against devs not catching the right exceptions and not handling them the right way, but microservices does.
Maybe you call that compensating for bad design, which is fair and in that case yes it is! And that compensation helps a large team move faster without perfecting design on every change.
There's a significant overhead to build and run tests at API level, that includes API versioning... and there's less of a need to version API inside a monolith.
And that is if it makes it to production. Here is your fleet of test hosts using production data and being verified against the output of production servers.
You can then have a pretty flexible trade off between the convenience of having email be a rooted library against the trade off of keeping it a lead service (the implication being that leaf services can talk to one another over the network via service stubs, rest, what have you).
This is SOA (Service Oriented Architecture), which should be considered in the midst of the microservice / monolith conversation.
On the RAM front, I am now approaching terabyte levels of services for what would be gigabyte levels of monolith. The reason is that I have to deal with mostly duplicate RAM - the same 200+ MB of framework crud replicated in every process. In fact a lot of microservice advocates insist "RAM is cheap!" until reality hits, especially forgetting the cost is replicated in every development/testing environment.
As for slow startup, a server reboot can be quite excruciating when all these processes are competing to grind & slog through their own copy of that 200+ MB and get situated. In my case, each new & improved microservice alone boots slower than the original legacy monolith, which is just plain dumb, but it's the tech stack I'm stuck with.
You are writing microservices and then running them on the same server??
Isn't that exactly what's required when you're deploying microservices independently of each other? (With the difference of the interface not being an ABI but network calls/RPC/REST.)
You can absolutely call js running in V8 vm from scala running in jvm. No networking needed, hell not even IPC is needed.
And when you deploy this you don't have to deploy all modules' http servers (for external requests into the system) and queue consumers in the same container, only a single module's. So no busy loops affect other modules, unless as a result of direct api call from module to module. If anything it encourages looser coupling as you are incenticised to use indirect communication through the queue over direct api calls.
Uh... what's the trick? I don't see how you can have V8 and the JVM communicate without something that's inter-process.
It's okay to not organize your code. It's okay to have files with 10,000 lines. It's okay not to put "business logic" in a special place. It's okay to make merge conflicts.
The overhead devs spend worrying about code organization may vastly exceed the amount of time floundering with messy programs.
Microservices aren't free, and neither are modules.
[1] Jonathan Blow rant: https://www.youtube.com/watch?v=5Nc68IdNKdg&t=364s
[2] Jon Carmack rant: http://number-none.com/blow/john_carmack_on_inlined_code.htm...
I easily find my way in messy codes with grep. With modules, I need to know where to search to begin with, and in which version.
Fortunately, I have never had the occasion to deal with microservices.
You can just grep / search all the files.
> in which version.
grep / search doesn't search through time whether you're using one file or many modules. You probably want git bisect if you can't find where something is anymore (or 'git log' if you have good commit messages),
But modules can be in different repositories I haven't cloned yet.
(And other tools like symbol search https://www.jetbrains.com/help/idea/searching-everywhere.htm...)
For searching multiple files, ripgrep works really well in neovim :)
Yeah sure, the scale is smaller, but you can't get away with making the user wait 10 seconds to render text and images to a screen either. I think the software world might be a lot better place if more developers thought like game developers.
Sure, but those "needs and requirements" aren't necessarily aligned with things that produce good software, and I think a lot of software development these days is not aligned. Further, I think the evolutionary path of the web in particular has produced a monstrosity that we'd be better off scrapping and starting over with at this point, but that's a tangential discussion.
That you probably don't even need, but there's a paradigm of "every function should be it's own class" that some devs seem to follow that I will never understand.
Their tiny data is rendering millions of polygons with real-time lighting, asset loading in the background, low latency input handling, etc. at 60fps+. If my API responds to the user in 100ms, they’re ecstatic. If their game responds to the user in 100ms (5-10 dropped frames) even a few times, they’re likely to ask for a refund or write a bad review, hurting sales.
The constraints are different, but game programmers do the same kinds of optimization social networks do, just for different problems. They avoid doing the N^2 version of lighting, path finding, physics simulations, etc. Social networks avoid it when searching through their graphs.
I think the web and other big tech companies should try thinking about problems the way game programmers do.
Microservices are the same modules. Though they force-add distributiveness, even where it can be avoided, which is fundamentally worse. And they make integration and many other things lot harder.
distage can do more, like wire your tests, manage component lifecycles, consider configurations while solving dependency graph to alter it in a sound manner.
Ever since my time as a mathematician (I worked at a university) and using LaTeX extensively, I never understood the "divide your documents/code into many small files" mantra. With tools like grep (and its editor equivalents), jumping to definition, ripgrep et al., I have little problem working with files spanning thousands of lines. And yet I keep hearing that I should divide my big files into many smaller ones.
Why, really?
Of course, that probably depends on the nature of the project. Having all backend code of a web app in one file and all the frontend code in another would be very inconvenient. OTOH, I have no problems with having e.g. multiple React components in one file...
I believe that files offer one useful level of compartmentalizing code that makes other people easier to understand what is going on by just looking at the file structure before opening a single file. The other guy can't grep anything before they have opened at least one file and found some interesting function/variable.
LaTeX is an obvious case: why pay to recompile the whole huge project, for each tiny change?
Now that I think about it, LaTeX is a very bad analogy anyway, since LaTeX documents usually have a very linear structure.
There are many other reasons why separation is better for humans (separation is organization) but these arguments about parallelism are valid for androids, humanoids and aliens.
I think the code should be split conceptually thus not just copy/paste part of code from main file to submodules, but split the code around some functional bounderies or concepts so that each file is doing one thing and compose those concepts into more abstract concepts.
So that when I try to debug something I can decide where I want to zoom in and thus be able to quickly identify the needed files.
and when dealing with the code that's well-organized and grouped into logically named files and dirs, you simply can navigate down the path and when you open a file all related code is there in one place without extra 10k lines of misc. code noise.
If some grouping of functions or classes is split out in a separate file where the externally used 'interface' is just a fraction of these functions or classes, and the rest are used only internally within the file, then this segmentation makes the system easier to understand.
That said, Emacs can show different parts of the same file in several buffers, a feature which I use quite a bit (not every day, but still). And of course I can rename them to something mnemonic (buffer name ≠ file name). So I personally don't find this convincing at all.
Also, filenames and directories tree act as metadata to help create a mental map of the application. The filesystem is generally well represented in exploratory tools like file browser and IDE. While the same information can be encoded within the structure of a single file, one needs an editor that can parse and index the format, which may not be installed on every system.
FWIW, sometimes when I worked on a really large file, I put some ^L's ("new page" characters) between logical sections, so that I could utilize Emacs' page movement commands.
Correct. But this is an argument against splitting: once your folder structure reflects your mental model, you should no longer split, no matter how big individual files get. Splitting further will cause you to deviate from your mental model.
Also, it seems like we're arguing against a strawman: saying "big files are okay" is not the same as "you should only have big files". When people mean is that having a big file does not provide enough justification to split it. But it is still a signal.
Generally speaking it's only Enterprise(tm) code that has miniscule (not small, miniscule) files with shit code
As long as it's good code, no one cares. If you aren't working on Enterprise(tm) code odds are every bad code you'll look at has way too many lines in the abstraction unit that is being used
If everything is effectively global to begin with, you're right, it might as well all be in one file. In very large programs the lack of scope control is going to be significant problem either way.
This is where object-oriented programming yields most of its actual value - scope control.
Yeah sure overengineering is a thing, but you're way off the path if you're brushing aside basic modularization.
If you have everything into one .c file, you need to recompile the whole thing every change.
I'm a proponent of keeping things in a monolith as long as possible. Break code into files. Organize files into modules. A time may come when you need to separate out services. Often, people break things out too early, and don't spend enough effort on thinking how to break things up or on the interfaces.
Don't you want determinism and deterministic simultations? If you do, you'll also need stub implementations (mocks, dummies) for your interfaces.
Some notes on that: https://blog.7mind.io/constructive-test-taxonomy.html
> A time may come when you need to separate out services.
Most likely it won't if you organise properly. For example, if each your component is an OSGi module.
You can say the exact same thing about C-headers though.
For instance, in C# a class can span multiple files using "partial" or you can have multiple classes in a single file. It's generally not an issue as long as the code itself is organized. The only downside is the reliance on an IDE, which is pretty standard these days anyway.
Same for business logic - very clear separation can be cumbersome sometimes, but otherwise it becomes messy if you're not careful. And careful people just tend to separate it.
It’s a good idea to break things down to files along logical boundaries. But got reporting isn’t a reason.
edit: "got diff" -> "git diff". DYAC and responding from mobile!
If it's a separate file that is scoped to some specific concern, sure. But its tgat grouping by concern that is key. Not separation into another file. Extracting ra dom bits of code into separate files would be _worse_.
Yes and no.
Git doesn't store lines, it stores files. Git diff knows how to spit out line changes by comparing files.
So to run git blame on a 10k line file you're reading multiple versions of that 10k file and comparing. It's slow. Worse still is that trying to split said file up while trying to preserve history won't make the git blame any faster.
As long as what's there is comprehensible, being able to evolve it over time is a very useful lever.
A 10k lines file is not something that will completely destroy your productivity, but it will have an impact and you'd better look out for it growing further, because completely destroying your productivity is not too far away. It is almost always good to organize your code when it reaches a size like this, and the exceptions are on contexts where you can't, never on contexts where it's worthless.
EDIT: And what editor do you use? I'm wondering if a lot of these differences come down to IDEs haha
This is a fair point but assumes 1 particular use case. It is easier if you are just concerned with a bit of it. If you need to deal with all of it, yeah, good fucking luck. 10k LOC file or 1k 100 LOC files.
I tend to ensure each file serves exactly one purpose (e.g. in C# one file = one class, with a only few exceptions).
I use VS Code, but in every IDE with a file opening palette it's actually really fast: you want to look for the code to, let's say, generate an invoice, just search for "invoice" in the list of files and you'll find it immediatly.
(Also modules have their own problem, I was mainly talking in a general way since that's what the parent comment was talking about.)
I also find that updating code to take advantage of new/better language features / coding styles, etc. is impossible to do on a large code base at once. However, sprinkling these kind of things randomly leads to too much inconsistency. A reasonable sweet spot is to make each file self-consistent in this regard.
My experience stems from larger 500+ person-year projects with millions of lines of code.
If a particular domain gets big enough it probably means it contains sub-domains and can benefit from being divided too. But you cannot make that decision based on size alone.
The point is that a numeric upper bound on LoC is inherently subjective and pointless. Instead of measuring the right thing (concepts) you're measuring what's easy to measure (lines).
In fact, it usually makes things worse. I've seen it over and over: you have N tightly coupled classes in a single file which exceeds your LoC preference.
Instead of breaking the coupling you just move those classes into separate files. Boom, problem solved. Previously you had a mess, now you have a neat mess. Great success!
Yes java developers can't do anything without their IDE. It helps them mask the 30000 nested directories they've created to "organize" the code.
Also, the question of inlined code mostly applies to programming in the small, while modules are about programming in the large, so I don't think there's much relationship between the two.
Most systems don't have to be maintained by hundreds of people. And yet they are: maybe because people don't listen to folks like Carmack?
We like stories about huge teams managing huge codebases. But what we should really be interested in is practices that small teams employ to make big impact.
Most of the time when I split out a method what I want is a closure that is clearly separated from the rest of the code. If I can have such closures, where input and output is clearly scoped, without defining a new method, that might be preferable. Lambdas could be one way to do it but they still inherit the surrounding closure so it's typically not as isolated as a method.
For sure, I have absolutely no idea how your comment relates to what I wrote.
It’s easy to miss the point of what the OP is saying here and get distracted by the fact this file is ridiculously huge. This file used to be a paltry 1k file, a 10k file, a 20k SLOC file… but it is where it is today because of the OP suggested approach.
<rant> You will (for most statements) both be able to find a best and worst case. That's the catch with most generalized statements/ principles, e.g., DRY. The challenge is to find a good "enough" solution because perfection is usually either unfeasibly expensive or impossible (different viewpoints, ...) </rant>
Though it's kinda hilarious that the source code of a MS project is not natively viewable on a MS platform.
With a trace from a profiler tool, which will tell you which line number is the hot spot. If run from any modern IDE, you can jump to the line with a mouse-click.
In essence, the file boundaries ought not to make any difference to this process.
I find this the strongest counter argument. File boundaries shouldn’t matter - but in practice they do.
The ide will give up doing some of its analysis because this is not an optimised use case, the IDE vendors don’t optimise for it. Some IDEs will even give up just simple syntax highlighting when faced with a large file, never mind the fancy line by line annotations.
Bitbucket is the biggest offender for me. It starts to disable syntax highlighting and context once the file hits about 500KB, which should be trivial to deal with.
Answer: the same place - with a profiler. Logical code organization matters a lot more than “physical” break to specific files.
I have inherited a Java class spaghetti project in the past, with hunderds (perhaps thousands) of short classes, each which doesn’t do much but sits in its own file - and I would much prefer to work on an SQLite style codebase, even if I have to start with the amalgamation file.
By the way, JavaScript parser in esbuild is a 16 thousand lines of code module too:
https://github.com/evanw/esbuild/blob/0db0b46399de81fb29f6fc...
The problem is when it's OK and for how long. If you have a team of people working with a codebase with all those "okays", then they have to be really good developers and know the code inside out. They have to agree when to refactor a business login out instead of adding a hacky "if" condition nested in another hacky "if" condition that depends on another argument and/or state.
I guess what I'm trying to say that if those "okays" are in place, then there's a whole bunch of unwritten rules that come in place.
But I agree that microservices certainly aren't free (I'd say they are crazy expensive) and modules aren't free either. But all those "okays" can end up costing you your codebase also.
If you're a single dev making a game, by all means, do what you want.
If you work with me in a team, I expect a certain level of quality in the code you write that will get shipped as a part of the project I'm responsible for.
It should be structured, tested, malleable, navigable and understandable.
One suggested alternative is to split reusable code into a local lambda first and lift it into a separate code piece only once we need that code elsewhere. It seems to me that such approach would limit the complexity of the code graph that you need to keep in your head. (Then again, when I think about it maybe the idea isn't really that novel.)
Great comment!
I personally find that most codebases are overstructured and undertested :)
In my experience, module boundaries tend to make code less malleable, less navigable, and less understandable.
People have different thresholds for when their code reaches these states though, especially "understandable".
You can meaningfully define these (and test for them) on a small scale, in a team, but talking about all developers everywhere, these are all very loosely defined.
Games are highly stateful, with a game loop that iterates over the same global in-memory data structure as fast as it can. You have (especially in Carmack era games) a single thread performing all the game state updates in sequence. So shared global state makes a ton of sense and simplifies things.
Most web applications are highly stateless with request-oriented operations that access random pieces of permanently-stored data. You have multiple (usually distributed) threads updating data simultaneously, so shared global state complicates things.
That game devs gravitate towards different patterns for their code than web service devs should not be a surprise.
---------- style C:
void MajorFunction( void ) {
// MinorFunction1
// MinorFunction2
// MinorFunction3
}
> At this point, I think there are some definite advantages to "style C", but they are development process oriented, rather than discrete, quantifiable things, and they run counter to a fair amount of accepted conventional wisdom, so I am going to try and make a clear case for it. There isn't any dogma here, but considering exactly where it is and isn't appropriate is worthwhile.
> In no way, shape, or form am I making a case that avoiding function calls alone directly helps performance.
Now your MinorFunction3 code can access all the local variables used by MinorFunction1 and MinorFunction2, and there's no easy list of things that it might access which makes it harder to read.
Separate functions do have a nice list of things they might access - their arguments!
Of course this technically only applies to pure functions so maybe that's why it doesn't matter to Carmack - he's used to using global variables willy nilly.
Also sometimes the list of things a function might need to access gets unwieldy, which is when you can reach for classes. So no hard and fast rule but I think increased scope is the issue with it.
Using A or B results in self-documenting code and I think DOES (or at least, CAN) improve readability. It also can help reduce excessive nesting of code.
Couldn't disagree more. As usual, it's a tradeoff. You could spend an infinite amount of time refactoring already fine programs. But complex code decreases developers productivity by orders of magnitude. Maybe it's not always worth refactoring legacy code, but you're always much better off if your code is modular with good separation of concerns.
It's not. This is the thing where you start thinking "YAGNI", yadayada, but you inevitably end up needing it. Layering with at least a service and a database/repositories is a no brainer for any non-toy app considering the benefits it brings.
> It's okay to have files with 10,000 lines
10.000 lines is a LOT. I consider files to become hard to understand at 1.000 lines. I just wc'd the code base I work on, we have like 5 files with more than 1.000 lines and I know all of them (I cringed reading the names), because they're the ones we have the most problems with.
It gets worse when the people who made the mess quit or move on, leaving the new hires to deal with it. I've seen this pattern enough times to wonder if it gets repeated with most companies or projects.
I do agree that microservices and/or modules aren't magical solutions that should be universally applied. But they can be useful tools, depending on the situation, to organize or re-organize a system for particular purposes.
Anecdotally, I've noticed that smart people who aren't good programmers tend to be able to write code quickly that can scale to a particular point, like 10k-100k lines of code. Past that point, productivity falls rapidly. I do believe that part of being a skilled developer is being able to both design a system that scales to millions of lines of code across an organization, and to operate well on one designed by someone else.
You have to understand everything that calls that function before you change it.
This is not always obvious. It takes time to figure out where it's called. IDEs make this easier but not bullet proof. Getting it wrong can cause major, unexpected problems.
If you split up MegaFunction(){} to Func1(){} Func2(){}, etc, but it never makes sense to call Func2 except after Func1, then you haven't actually created two functions, you've just created one function in two places.
Refactoring should be about logical separation not about just dicing a steak because it's prettier that way.
MegaFunction()
{
Func1();
Func2();
...
FuncN();
}
By chopping up MegaFunction like this, you've not actually separated Func1 and Func2 if they aren't really independent, they're just MegaFunction in disguise but now split across two places making it more difficult, not easier, to reason about.
If you need the state (implied or explicitly passed in and out) from having executed Func1 to run Func2 then you're just creating spaghetti code. You're taking a big ball of mud and smearing it around instead of actually tackling the abstraction.
This is how you end up with Func(a, b, c, d, &e, &f) which ends up changing your MegaFunction state.
Or just as bad, Func1,2,N are all private functions, never called anywhere else outside the class MegaFunction is in, so logically (and from the point of view of testability) it's no different to having them all inline.
If you're creating a function that's only ever called once, then instead of a function call you're probably better off with a comment to "name" that block and explain the process instead.
Frankly It Depends(tm). Sometimes you can do this and not pass too much state, sometimes you can not. Sometimes your state is in a class, or global, and you just pass the class around.
I have somewhere a 3000 line state machine with the app state in a class - i just pulled out logically connected states in auxiliary files when it went over 1k lines. In my case it's easy to comprehend because groups of states are kinda separated logically.
Unfortunately most engineers don't like nuance, they want one-size-fits-all solutions.
I find there are practical negative consequences to having a 10,000 line file (well, okay; we don't have those at work, we have one 30k line file). It slows both the IDE and git blame/history when doing stuff in that file (w.r.t. I'll look at the history of the code when making some decisions). These might not be a factor depending on your circumstances (e.g. a young company where git blame is less likely to be used or something not IDE driven). But they can actually hurt apart from pure code concerns.
>It's okay to not organize your code. It's okay to have files with 10,000 lines. It's okay not to put "business logic" in a special place. It's okay to make merge conflicts.
I absolutely disagree. It's "okay" if you're struggling to survive as a business and worrying about the future 5+ years out is pointless since you don't even know if you'll make it the next 6 months. This mentality of there being no need for discipline or craftsmanship leads to an unmanageable codebase that nobody knows how to maintain, everybody is afraid to touch, and which can never be upgraded.
You don't see the overhead of throwing discipline out the window because it's all being accrued as technical debt that you only encounter years down the road.
https://github.com/tinspin/rupy/wiki/Process
All input I have is you want your code to run on many machines, in fact you want it to run the same on all machines you need to deliver and preferably more. Vertically and horizontally at the same time, so your services only call localhost but in many separate places.
This in turn mandates a distributed database. And later you discover it has to be capable of async-to-async = no blocking ever anywhere in the whole solution.
The way I do this is I hot-deploy my applications async. to all servers in the cluster, this is what a cluster node looks like in practice (the name next to Host: is the node): http://host.rupy.se if you click "api & metrics" you'll see the services.
With this not only do you get scalability, but also redundancy and development is maintained at live coding levels.
This is the async. JSON over HTTP distributed database: http://root.rupy.se (2000 lines hot-deployable and I can replace all databases I needed up until now)
I.e. the idea that each microservice has direct access to its own, dedicated, maybe duplicated storage schema/instance (or if it needs to know, for example, the country name for ISO code "UK" it is supposed to ... invoke another microservice that will provide the answer for this).
I always worked in pretty boring stuff like managing reservations for cruise ships, or planning production for the next six weeks in an automotive plant.
The idea of having a federation of services/modules constantly cross-calling each other in order to just write "Your cruise departs from Genova (Italy) at 12:15 on May 24th, 2023" is not really a good fit for this kind of problems.
Maybe it is time to accept that not everyone has to work on the next version of Instagram and that Microservices are probably a good strategy... for a not really big subset of the problems we use computers for?
No, you don't use separate microservices for writing out that text message.
The idea is pretty simple instead of writing one big program, you write many smaller programs.
It's useful around the 3 to 4 separate developers mark.
It avoids you having to recompile for any minor change, allows you run the tests for just the part you changed and allows you to test the microservices in isolation.
If you a production issue, the exception will be in a log file that corresponds to a single microservice or part of your codebase.
Microservices are a hard form of encapsulation and gives a lot of benefits when the underlying language lacks that encapsulation. e.g. Python.
But in order to find out that Genua is the name of the port from where the cruise is departing, the appropriate time (converted to the timezone of the port, or the timezone of the client who will see this message, depending on what business rule you want to apply) and that Genua is in IT=Italy... how many microservices do I have to query, considering that port data, timezones, ISO country codes and dep.date/time of the cruise are presumably managed on at least four different "data stores"?
e.g.: https://www.baeldung.com/cs/microservices-db-design
"2.1. Fundamentals By definition, microservices should be loosely coupled, scalable, and independent in terms of development and deployment. Therefore, the database per service is a preferred approach as it perfectly meets those requirements. Let’s see how it looks:"
Please understand that I have worked only on monoliths and will probably retire while still working on monoliths. This kind of absurd positions only come up when someone comes to my office with some grand plan to convert the application I work on to something "more microservice oriented".
Let's suppose we are Acme Cruise Lines running a Cruiseliner:
Microservice - National Coastguard Ship Arrival System Feed Handler
Database - Logs incoming messages on the feed
Microservice - Asian and Australian Joint Ship Monitoring System
Database - Logs incoming messages on the feed
Microservice - Cruiser Arrival and Departure Times
Database - Cruiser Arrival and Departures Times in a Standard Format
Microservice - Customer Bookings and Payments
Database - Customer Bookings and Payments
Microservice - Fuel Management System
Database - Ship Fuel Levels & Costs of fuel at Various Ports.
It's that high level of split up.
(AWS Lambdas aren't quite the same thing as microservices.)
The things you listed are ... 4-5 different applications, mostly running directly on the ship(s) and what is conspicuously missing are the parts that are managed shoreside, like:
Itinerary planning (your product is one or more cruises: therefore you need to preplan the itineraries, which ships to use, when you will enter each port and when you will leave it, and so on... try to imagine this like a mix between hotel management and flight company management) Inventory management (i.e. Reservation).
You mention "bookings" like a microservice. This could work for ferry line, where you are basically selling a ticket for a single trip, most of the time with no personal accommodation (except maybe for the car).
A Cruise booking usually comes with a ton of ancillary services (e.g. I live in Berlin and I want to take a cruise in the Caribbeans... therefore I need a flight to get there and back. This could be provided by a charter flight or a normal airline ... in either cases the ticket will be part of the booking itself) - take in account that cruise customers are mostly middle-aged or older and relatively affluent, therefore the last thing they would like to do is to create their own itinerary by booking services on 4-5 different websites.
(But I suppose it is better to stop there, we are really OT now)
When you book a cruise, your expectation is to get everything (including hotel stays before or after the cruise, along with flights to and from it) as a single package provided by the cruise vendor (or a travel agency). So the "reservation system" must take care of all of that.
When I said "4-5 different websites" I was trying to explain the point that a 60 yo high-income guy is usually not interested in getting the cruise itself on Carnival.com, then go look for excursions on CarribeansExcursion.net and to book flights on Lufthansa.de or AirFrance.fr.
It was a remark on the way the Cruise business works, and why it is so, not about architecture.
(But, once again, I really believe that we are way off topic... personally I do not really feel like creating a "Ask HN: are microservices a good choice for the cruise industry?" but if someone feels like submitting one I will try to contribute).
The thing is that the framing of "the problems we use computers for" misses the entire domain of problems that microservices solve. They solve organisational problems, not computational ones.
And this is not because "my stuff is complicated and your stuff is a toy", either. It's more like "ERP or Banking Systems" were deployed decades ago, started as monoliths and nobody can really afford to rewrite these from scratch to leverage Microservices (or whatever the next fad will be). (I am also not sure it is a good idea in general for transactions that have to handle/persist lots of state, but this could be debated).
The problem, in fact, is that "new guys" think that Microservices will magically solve that problem, too, want to use these because they are cool and popular (this year), and waste (and make me waste) lots of time before admitting something that was clear from day 1: Microservices are not a good fit for these scenarios.
(I still think that "these scenarios" are prevalent in any company which existed before the 90s, but I might be wrong or biased on this).
> I have exactly the opposite problem though: the kind of problems I have worked on so far would not be "solved" by leveraging "large number of developers that can independently work on an equally large number of small, modular programs with a very well defined, concise interface".
If you don't have multiple teams working on it, and you don't have specific chunks of functionality that you need to scale orthogonally, then you don't have the problems microservices solve. So don't use them. That seems uncontroversial to me.
> I still think that "these scenarios" are prevalent in any company which existed before the 90s, but I might be wrong or biased on this
This is survivorship bias, in a sense. Microservices only make sense where access to CPU and network is cheap and on-demand (largely). That's only started to be an assumption you could really make since Amazon's EC2 convinced the world's CIOs that fungible tin was better than their own racks.
That means you don't see microservice architectures in older companies, and you don't see problems being solved with them even where it might have made sense retrospectively because IT Ops would never have been set up to enable it.
Today you don't need a big rewrite to justify introducing microservices where they're needed. That's a straw man. All you need is a team that will be able to move faster if their deployment is decoupled from everyone else's.
But fundamentally if your problem area smells like "bunch of business rules and a database" then, again: if you don't have the problem that the architecture solves, don't use it.
cough Aging directors in Health and Finance cough
A micro service implementation of that would be CQRS, where the services to write updates, backend process (eg, notifying the crew to buy appropriate food), and query existing records are separated.
You might even have it further divided, eg the “prepare cruise” backend calls out to several APIs, eg one related to filing the sailing plans, one related to ensuring maintenance signs off, and one related to logistics.
One strategy I've used is to designate one system as a source of truth (usually an ERP system) and periodically query its database directly to reload a cache. Every system works off their own periodically refreshed cache. Ideally, having all the apps query a read replica would prevent mistakes from taking down the source of truth.
I haven't done this, but I think using Postgres with a combination of read-only foreign tables and materialized views could neatly solve this problem without writing any extra code.
I don't know how far this would scale. I do know that coordination and procedures for using/changing the source of truth will fall apart long before technical limitations.
Does anyone have a microservice "map" of Instagram? I feel that would be helpful here.
Or you can think of it as trunk/branch architecture. One main "trunk" service and other branch services augmenting it, which is a simpler thing to reason about.
Now imagine a small shop of 20 devs deciding to build something more complicated.
Binaries are deployed and scaled independently as thrift services.
Tons of rpc.
And the virtue of microservices is that they create hard boundaries no matter your skill and seniority level. Any unsupervised junior will probably dissolve the module boundaries. But they can't simply dissolve the hard boundary of having a service in another location.
The best place for such boundaries is at the team/division/org level, not team-internal or single dev-internal like microservices implies with its name. Embrace Conway's law in your architecture, and don't subdivide within a service.
However this is muddied by the fact that I almost never see microservices, I see a lot of distributed monoliths tho.
This is the road to bullshit. Of course no manager or CEO will admit that their team/company is that. Admitting technical non-excellence is nearly impossible. Organisation inadequacy... impossible.
So the best course of action is to pretend your organisational methods are really just software architecture... and back to square one.
When he helped set the tone for the Zen architecture he took AMDs existing people, set them on a more aggressive path and one of the cardinal tenants was you could not cheat on the interfaces. This is one of the nuggets you can pull from his interviews.
It's possible. It happens. And the end results can be industry changing.
If you are a dumpster-fire S&P500 company CTO and there is a new shiny thing that would actually improve things, you are probably more capable of redefining that new term into the horse-shit you are currently doing; than actually do it.
If done right microservices is a way to transform part of your organization challenge into a technical one, which for many organizations is the right move.
The biggest issue is if you aren't large enough to have challenging organizational issues it's much easier to just solve the very solvable organizational issues than implement and use microservices.
Famous last words, if done right... Or you just multiply your organizational issue with a technical one.
But implementing microservices is not an unsolvable problem. It's a problem that 1000s of organizations have solved.
When the "new shiny" effect wears off you usually find a turd that smells worse than what came before. Which is why we see this thread ever month or two and will until the tooling catches up and companies stop creating the distributed turds or the method falls out of grace because people finally realize you can scale simple systems pretty far vertically.
I've seen people doing that a few times already. They start changing due to some uninformed kind of "convenience", and then you look at the services API and it makes no sense at all.
Look up a thing called "distributed monolith".
The rigor of maintaining resilient, backward-compatible, versioned internal APIs is too resource and time consuming to do well. All I see is hack and slash, and tons of technical debt.
It seems like in the last couple of years it started sinking in, that distributed systems are hard.
"Hey our developers can't make modules correctly! Let's make the API boundaries using HTTP calls instead, then they'll suddenly know what to do!"
And that unsupervised junior? At one place I joined, that unsupervised junior just started passing data he "needed" via query string params in ginormous arrays from microservice to microservice.
And it wasn't a quick fix because instead of using the lovely type system that TELLS you where the stupid method has been used, you've got to go hunting for http calls scattered over multiple projects.
All you've done is make everything even more complicated, if you can't supervise your juniors, your code's going to go sideways whatever.
Microservices don't solve that at all and it's pure circular logic to claim otherwise. If your team can't make good classes, they can't make good APIs either. And worse still, suddenly everything's locked in because changing APIs is much harder than changing classes.
Microservices are a solution to a problem. TDD is a solution to a problem, the same problem. Both are solutions that themselves create more, and worse, problems. Thanks to the hype driven nature of software development the blast radius of these 'solutions' and their associated problems expands far beyond the people afflicted by the original problem.
That problem? Not using statically typed languages.
TDD attempts to reconstruct a compiler, poorly. And Microservices tries to reconstruct encapsulation and code structure, poorly. Maybe if you don't use a language which gets hard to reason about beyond a few hundred lines you won't need to keep teams and codebases below a certain size. Maybe if you can't do all kinds of dynamic nonsense with no guardrails you don't have to worry so much about all that code being in the same place. The emperor has no clothes, he never had.
Edit: to reduce the flame bait nature of the above a bit. Not in all cases, I'm sure there are a very few places and scales where microservices make sense. If you have one of those and you used this pattern correctly, that's great.
And splitting out components as services is not always bad and can make a lot of sense. It's the "micro" part of "microservices" that marks out this dreadful hype/trend pattern I object to. It's clearly a horrible hack to paper over the way dynamically typed codebases become much harder to reason about and maintain at scale. Adding a bunch of much harder problems (distributed transactions, networks, retries, distributed state, etc) in order to preserve teams sanity instead of just using tooling that can enforce some sense of order.
Is that even true? DHH (RoR creator) hates TDD.
[0]: https://www.stevenrbaker.com/tech/history-of-rspec.html
It is true that one may use testing to stand in for the lack of static typing, which is no dobut what the parent is talking about, but TDD != Testing. Testing long predates TDD.
A compiler can't check that your logic is correct. There may be a bit of overlap in the things verified by tests and a compiler but they don't solve the same problem.
How are static types a requirement for encapsulation? Dynamic languages are perfectly capable of providing encapsulation. Statically typed languages are also perfectly capable of having very poor encapsulation.
TDD was invented by a Java programmer. How does that fit into your world view?
Saying age is an int is great, but Java doesn't let you say age is an int ranging from 0 to 130.
You can of course create an age object, but the constraints on that object cannot be expressed within the type system.
So you have to add unit tests instead, the unit tests in effect extend the type system out to be as powerful as type systems from the 80s.
Yay, progress! :/
This sounds good on paper, but good luck passing your custom types to third party libraries. Or even doing something as simple as calculating average age.
You do realize the more powerful the type system is, the more closely it is going to resemble programming language, which means that structures you build on top of it will contain bugs? You're not really solving bugs, you're just pushing them to another layer.
Ada solved this in the 80s, it isn't some unresolved field of comp sci. (Java already does this for array bounds!)
> You do realize the more powerful the type system is, the more closely it is going to resemble programming language, which means that structures you build on top of it will contain bugs? You're not really solving bugs, you're just pushing them to another layer.
Unit Tests are no different, except with worse syntax than built in language support.
At least with support in the type system you can turn off range checks for production builds. Meanwhile unit tests only run when invoked, giving less confidence than type checks that undergoing simulated usage as part of daily test builds.
There is of course a gradient, JavaScript exists on one side of that gradient (unit tests are needed for everything, starting with what type of object is even being returned), Java exists in the middle, and you have actual type safe languages farther along.
The point I was originally aiming to make is that Java is statically typed, but its type system is not so powerful as to negate the need for unit tests.
Every solution comes at a cost. Where is Ada now?
> Unit Tests are no different, except with worse syntax than built in language support.
And I'm not advocating for unit tests, nor do I treat them as replacement for types. The fact that they occasionally overlap doesn't mean they serve same purposes.
> At least with support in the type system you can turn off range checks for production builds
Sounds like dynamic typing with extra steps.
So you don't advocate for unit tests, you don't want a powerful type system, what do you want?
> Every solution comes at a cost. Where is Ada now?
A source of features for newer languages, thus is the circle of programming language life.
> Sounds like dynamic typing with extra steps.
Static assertions based on build flags have been around for a very long time, in all sorts of languages. They are a performance/safety trade off.
"Write tests. Not too many. Mostly integration."
Leave static typing for performance-critical sections.
> They are a performance/safety trade off.
So is dynamic typing. Only performance in this case means developer performance. Turns out most of our software is not landing airplanes, and developer performance is way more important than occasional non-critical bug that affects 5 users and gets fixed within a day.
TDD is a method of documenting your application in a way that happens to be self-verifying. You could use a Word document instead, but lose the ability for the machine to verify that the application does what the documentation claims that it should. Static typing does provide some level of documentation as well, but even if you have static typing available static typing isn't sufficient to convey the full intent that your documentation needs to convey to other developers.
I think at the root this idea that Microservices provide this technical solution to a social issue, coordinating teams at scale, hides the driving motivation.
Why can't the teams work on the same codebase and the same database? (again note my edited in disclaimer in the GP post that maybe at certain rare scales it's necessary). What is a well defined specification and why is it only a REST/proto/whatever network API? Why not a defined interface or project/library? Why aren't teams at this scale working in languages that support defining these things in code without adding a complicating, latency and error-inducing, network call?
Statically typed languages are just superior at encoding this "well defined specification". It's a large part of their reason for being. Sure you can still bypass it and do reflection or other naughtiness, but we shouldn't pretend a network call is the only solution.
# TDD
Testing is certainly a way to provide a self-verifying application. TDD is a toxic, hype-driven, mess that has led many good developers astray. Static languages have their own devils here, mocking being chief among them (not to mention the Java clown brigade and their own enthusiastic adoption of -- originally Smalltalk conceived -- TDD).
I added TDD both to annoy people but also to illustrate the general point. These are both concepts derived from the obvious drawbacks of dynamically typed languages that are then treated as generalist answers to complexity in software.
If you'll allow me some pseudo-intellectual waffle, the complexity of a software system C_dynamic + C_software = C_total, that is, complexity of dynamically typed languages and their lack of rigour, plus complexity of software and the domain as a whole gives you total complexity. These approaches primarily target the dynamic complexity. Therefore introducing them and all attendant drawbacks in places where total complexity doesn't include the dynamic complexity because tools and approaches to remove this source of complexity are used actually worsens the entire field.
The same database is tricky because one team might want to change the schema, breaking another team's work. You then need to have meetings to discuss the changes that will work for all parties and you've broken "communicate only by contract", and therefore no longer doing microservices. If you can ensure that there is no way for teams to trample over each other then you could use the same database.
The same codebase is more feasible as long as the team boundary separation is clear. You are likely already doing microservices with other teams when you import third-party libraries. In practice, though, much like the database problem maintaining boundary separation is difficult if you can easily reach into other team's code, so often each team's work will be parted into distinct codebases using IPC between them to ensure that the separation is unbreakable. This certainly isn't a requirement, though, just one way to keep people in line.
> Testing is certainly a way to provide a self-verifying application. TDD is a toxic, hype-driven, mess
Testing is a development tool to help test the functionality of a feature. If you are faced with a dynamic language you very well might write tests to ensure that types are handled correctly. Static types can, indeed, relieve the need for some tests.
TDD, on the other hand, is a documentation tool to relay to other developers what functionality is intended to provide and how it is meant to be used. It is strange to think that documentation is toxic. Before TDD we wrote "Word documents" containing the same information, but often found that the code didn't do what the documentation said it did. What TDD realized is if the documentation is also executable then you can have the machine prove that the application behaves according to spec. It's not exactly magic.
Agreed the same database gets difficult but I guess, where two things need to change dependently there's coordination overhead, that's inescapable. You can in general use e.g. schemas to keep domains fairly separate. Whether it's your HTTP API interface changing or the change needs to be coordinated in the same process I don't think you have a way to avoid these meetings. (or like the 3rd party team I work with you just change your HTTP API and break production without telling us :shrug:).
I guess my point is languages like Java and C# are much maligned for their "boilerplate" and it doesn't let 10x genius programmers go brrrrrr etc. But that boilerplate serves a purpose. Projects (in C#, I can't speak to Java but I believe it has a similar concept) compile to separate .dll's, or .so's. In general a project will publicly expose interfaces, i.e. contracts, and keep its internals encapsulated. Crucially projects all live in the same codebase in the same monolith, they're folder/directory level separation, but on steroids. There are tricks with attributes and reflection that can break this separation but with the additional constraint that dependencies cannot be circular you're forced to make decisions about where exactly things live. This is where the dynamic crowd throw up their hands and say "there are so many layers of pointless abstraction!". I'd agree these languages tend to too many layers but the layers provide the foundation which can support a project growing from 1-2 devs all the way through to 50+.
I'm maybe finally becoming a grumpy old programmer and I'm glad much of the consensus is swinging away from the no-types, no schema, no rules position of the past decade. People forgot Chesterton's fence and decided the layers didn't provide anything, then they added them back, as network boundaries, which was indescribably worse and more of a headache.
# Testing
I guess it's a semantics debate really. I think the red-green-refactor loop of TDD doesn't bring much. I've never seen it applied well on a statically typed codebase and where I've seen advocates try and apply it I've seen them tunnel-vision to suboptimal outcomes in order to fulfil the ceremonial needs of the process.
I think a few tests that take place at the project/dll/so boundary and use as much of the real dependencies as possible are far preferable to high coverage. This is probably a no-true-Scotsman area of debate. Maybe TDD has been done properly and productively by some groups, I've never seen it and no one I've met ever has (check out my confirmation bias!).
Interfaces don't change under microservices. Communication by contract enshrines a contract. You must ensure that your API does not break legacy users no matter what changes you want to make going forward. You have committed to behaviour forevermore once you submit the contract to other teams.
This may be another reason why IPC is often preferred over straight function calls as some languages, particularly those with static types, make extending functionality without breakage quite hard. An HTTP API can more easily resort to tricks to return different results to different callers.
> I think the red-green-refactor loop of TDD doesn't bring much.
You find little value in writing a specification for your work or you find little value in automatic validation that the program works according to your spec?
"Red-green-refactor" is a little more specific in that it says that you should write a spec only for the work you know you are going to work on in the short term, whereas TDD in general leaves room for things like writing the entire application's spec before getting down to business. I think this is most practical in the real world, generally speaking. Often you don't know what your entire application should do in the beginning, making writing a full spec unrealistic.
> I've never seen it applied well on a statically typed codebase
Interesting, as I have only ever seen specifications (be it TDD or "Word documents") written for work that is done in statically typed languages. In my experience, the dynamic programming languages tend to attract the cowboy coders who don't understand why other (future) developers need more complete documentation or why specs are important and have no care to see that those things are done.
> Interfaces don't change under microservices. Communication by contract enshrines a contract. You must ensure that your API does not break legacy users no matter what changes you want to make going forward. You have committed to behaviour forevermore once you submit the contract to other teams.
I think this entails a couple of things.
Firstly, it's a trade-off. If we discover some incorrect assumption is baked in to both sides of an interface we're kind of stuck and have a hard time with the migration path going forward. This isn't unsolvable and paths exist to correct it, but they're substantially harder than just updating both parts in the same commit. It moves coordination overhead as a trade-off.
Secondly, what I think actually happens is that most companies using microservices aren't this disciplined and just end up bodging or implicitly coupling services. Probably the pattern I'm talking about is "distributed monolith". Not the good kind where you just scale out your monolith but where you drink the microservices kool-aid and create "microservices" by adding network boundaries at random. This is the real-world application of microservices I have seen where I've seen it. I don't doubt some people can use it correctly with proper rigour. But my theory is the pattern took hold because using "distributed monolith" (appears to help / ) helps tame the complexity of a dynamic monolith. I find it incredibly hard to believe the niche pattern of microservices done correctly would have become so popularized otherwise.
> You find little value in writing a specification for your work or you find little value in automatic validation that the program works according to your spec?
I find little value in providing that specification through the TDD approach or associated ceremony. I prefer as few tests as possible at as high a level as possible with as few mocks as possible. I'll take a single test that takes 15 seconds to run over 1000 tests running in milliseconds.
Again for me this is a question of "what popularized this pattern?".
I think there's a lot of healthy debate to be had about how much testing is needed. I think TDD appeared to provide the kind of guarantees that are really helpful in a dynamic language and gained a lot of advocates that way. When I had to write some (type hint free) Python I naturally defaulted to writing TDD since keeping a large system in your head and error free is basically impossible without an exhaustive suite of tests. It's an approach that works very well in one context that applied to other contexts unquestioningly delivers a lot of pain.
Maybe you're lucky to have only worked with very deliberate and rigorous engineers in your career. Outside the top tier we're working with architecture astronauts and people who jump on whatever hype cycle happens to be passing which explains my anger with these concepts.
Not at all. I'm in no rush.
> Secondly, what I think actually happens is that most companies using microservices aren't this disciplined and just end up bodging or implicitly coupling services.
If an organization is sufficiently large you may have no way to get another team on the line to even try. At that scale the other teams may as well work for other companies and that is what microservices models. If your organization is small, I tend to agree that you won't succeed. You can't beat Conway's Law.
> I find little value in providing that specification through the TDD approach or associated ceremony.
But, ultimately, what's the difference between writing your spec in an executable way or writing it in a Word document beyond the superficial differences in the languages? What is communicated to other developers ends up being the same.
> I prefer as few tests as possible at as high a level as possible with as few mocks as possible. I'll take a single test that takes 15 seconds to run over 1000 tests running in milliseconds.
Seemingly not all that common, interestingly, but Go comes to mind as a language that provides constructs to define separation between specification (TDD) and developer tests. Conceivably you could exclude your TDD specs from execution, only running the tests that help in your development process.
The value of TDD isn't in the execution, but in what is communicated to other developers. That the specs are executable is merely a nice side benefit to help with confirming that the implementation conforms to the spec. You still get 90% of the benefit of TDD even if you never run the executable, albeit granted at that point it is just a fancy Word document.
> I think there's a lot of healthy debate to be had about how much testing is needed.
Perhaps, but TDD isn't about testing. TDD is about providing documentation. I think there is less room for debate there. I'm not sure anyone who has ever inherited a codebase has wished it gave less insight into what the 'business needs' of the program are. Again, a Word document can provide the same documentation, but if you're going to write that Word document anyway why not go the extra mile and gain the additional benefits that come with execution?
Most important thing for lots of startups and companies is Time to Market.
Traditional static typing based approachs are just a bad joke (3 times slower on average) in comparsion.
That's why we have the whole microservices and dynamic typing thing going on, because businesses that use it beat up businesses that don't. It's pretty simple really.
Good LUCK getting that property with a monolithic or modular system. QE can never be certain (and let's be honest, they should be skeptical) that something modified in the same codebase as something else does not directly break another unrelated feature entirely. It makes their life very difficult when they can't safely draw lines.
Two different "modules" sharing even a database when they have disparate concerns is just waiting to break.
There's a lot of articles lately dumping on microservices and they're all antiquated. News flash: there is no universal pattern that wins all the time.
Sometimes a monolith is better than modules is better than microservices. If you can't tell which is better or you are convinced one is always better, the problem is with you, not the pattern.
Microservices net you a lot of advantages at the expense of way higher operational complexity. If you don't think that trade off is worth it (totally fair), don't use them.
Since we are talking about middleground, one I'd like to see one day is a deploy that puts all services in one "pod", so they all talk over Unix socket and remove the network boundary. This allows you to have one deploy config, specify each version of each service separately and therefore deploy whenever you want. It doesn't have the scalability part as much, but you could add the network boundary later.
1. Deployment. Being able to deploy code rapidly and independently is lost when everything ships as a monolith.
2. Isolation. My process is GC-spiraling. Which team's code change is responsible for it? Since the process is shared across teams, a perf bug from one team now impacts many teams.
3. Operational complexity. People working on the system have to deal with the the fact that many teams' modules are running in the same service. Debugging and troubleshooting gets harder. Logging and telemetry also tends to get more complicated.
4. Dependency coupling. Everyone has to use the exact same versions of everything and everyone has to upgrade in lockstep. You can work around this with module systems that allow dependency isolation, but IMO this tends to lead to its own complexity issues that make it not worthwhile.
5. Module API boundaries. In my experience, developers have an easier time handling service APIs than library APIs. The API surface area is smaller, and it's more obvious that you need to handle backwards compatibility and how. There is also less opportunity to "cheat", or break encapsulation, with service boundaries compared to library boundaries.
In practice, for dividing up code, libraries and modules are the less popular solution for server-side programming compared to services for good reasons. The downsides are not worth the upsides in most cases!
Why do we care about network latency, when we JUST established that microservices are about scaling large development teams? I have no problem with hackers ranting about slow, bloated and messy software architecture...but this is not the focus of discussion as presented in the article.
And then this conclusion:
> The key is to establish that common architectural backplane with well-understood integration and communication conventions, whatever you want or need it to be.
...so, like gRPC over HTTP? Last time I checked, gRPC is pretty well understood from an integration perspective. Much better than Enterprise Java Beans from the last century. Isn't this ironic? And where are the performance considerations for this backplane? Didn't we criticize microservices before because they have substandard performance?
> common architectural backplane with well-understood integration and communication conventions, whatever you want or need it to be
Regardless of the tech used to implement, this paradigm needs to be solved to have a good system. That backplane is not an implementation, but a set of understood guiderails for inter-module communication.
Even with gRPC, both sides need to know what to call and what to provide and expect in response. That's the "conventions" part. Having consistency is more important than the underlying tech. Just simple ReST over HTTP works just as well as gRPC.
Then why is the author even discussing microservices in the first place? Following your logic, they are an implementation detail just like modules.
Take for example Gartner's definition:
> A microservice is a service-oriented application component that is tightly scoped, strongly encapsulated, loosely coupled, independently deployable and independently scalable.
That's not too controversial. But... as a team why and when would you want to implement something like this? Again, let's ask Gartner. Here are excerpts from "Should your Team be using Microservice Architectures?":
> In fact, if you aren’t trying to implement a continuous delivery practice, you are better off using a more coarse-grained architectural model — what Gartner calls “Mesh App and Service Architecture” and “miniservices.”
> If your software engineering team has already adopted miniservices and agile DevOps and continuous delivery practices, but you still aren’t able to achieve your software engineering cadence goals, then it may be time to adopt a microservices architecture.
For Gartner, the strength of Microservice Architecture lies in delivery cadence (and it shouldn't even be the first thing you look at to achieve this). For another institution it could be something else. My point is that when people talk about things like Microservices they are often at cross-purposes.
Another way to put it is that teams that share parts of the same codebase introduce low level bugs that affect each other, and most organizations are clueless about preventing it and in some cases do not even detect it.
Part of the reason is that anything that leaves the application package increases the error potential exponentially.
Also, modules "bottleneck" functionality, and allow me to concentrate work into one portion of the codebase.
I'm in the middle of "modularizing" the app I've been developing for some time.
I found that a great deal of functionality was spread throughout the app, as it had been added "incrementally," as we encountered issues and limitations.
The new module refines all that functionality into one discrete codebase. This allows us to be super-flexible with the actual UI (the module is basically the app "engine").
We have a designer, proposing a UX, and I found myself saying "no" too often. These "nos" came from the limitations of the app structure.
I don't like saying "no," but I won't make promises that I can't keep.
BTW: The module encompasses interactions with two different servers. It's just that I wrote those servers.
Most likely the question is not well defined in the first place.
It's like we're stuck assembling applications by stacking pre-existing Lego sets together with home-made, purpose-built, humongously-sized Lego bricks made of cardboard acting as glue.
I've ranted a bit about that here: https://news.ycombinator.com/item?id=34234840
This is generally what I think the best part about microservices are; an easy way to improve upon the MVP in any dimension. Re-writing a whole monolith can take forever and probably will introduce a ton of bugs. But incrementally re-writing microservices won't stop feature development and is easy to A/B test for correctness.
> ... the Fallacies of Distributed Computing.
I feel like I’m taking crazy pills, but at least I’m not the only one. I think the only reason this fallacy has survived so long this cycle is because we currently have a generation of network cards that is so fast that processes can’t keep up with them. Which is an architectural problem, possibly at the application layer, the kernel layer, or the motherboard design. Or maybe all three. When that gets fixed there will be a million consultants to show the way to migrate off of microservices because of the 8 Fallacies.
When a critical vulnerability comes out for whatever language you are using, you now have to patch, test and deploy X apps/repos vs much fewer if they are consolidated repositories written modularly. Same can be said for library/framework upgrades, breaking changes in versions, deprecated features, taking advantage of new features, etc.
Keeping the definition of runtimes as modular as the code can be instrumental in keeping a bunch of related modules/features in one application/repository. One way is with k8s deployments and init params where the app starts specific modules which then lends itself to be scaled differently. I'm sure there are home-grown ways to do this too without k8s.
I'm glad that this post was written so that we can look at widely accepted ideas a little more critically.
Monoliths are way slower to deploy than microservices, and when you have hundreds or thousands of changes going out every day, this means lots of changes being bundled together in the same deployment, and as a consequence, lots of bugs. Having to stop a deployment and roll it back every time a defect is sent to production would just make the whoe thing completely undeployable.
Microservices have some additional operational overhead, but they do allow much faster deployments and rollbacks without affecting the whole org.
Maybe I am biased, but I would love an explanation from the monolith-evangelist crowd on how to keep a monolith able to deploy multiple times a day and capable of rolling back changes when people are pushing hundreds of PRs every day.
Are those thousands of developers working on a single product? If so, then I'd argue that you have way too many developers. At that point, you'd need so many layers of management that the overall vision of what the product is gets lost.
You're likely right.
I think what most detractors of microservices are pointing to is that most companies don't reach thousands of devs in size. Or even hundreds.
People just read up on whatever seems to be the newest, coolest thing. The issue is that MS articles are usually coming from FAANG/ex-FANNG. These companies are solving problems that 99% of others do not.
As engineers we should be looking for the most effective solutions to a given business problem. Sadly, I see engineers with senior/staff titles just throwing cool tech terms/libs around. Boring tech club ftw
As a CTO, I couldn't agree more. For our internal product, we use 100% boring technologies. The most "modern" you'll find is a React SPA.
I sigh when clients want to go the microservices route for a team of just a few developers. When you want to use NextJs for their tables&forms app. When they choose to use Kubernates instead of a couple EC2 instances.
Don't get me wrong, these technologies are great for us because we can charge more for the wasted human time developing these overengineered solutions. But I always, for my peace of mind, try to talk them out of them. Sometimes works, sometimes doesn't, at the end of the day it's their money.
The article gets it right in ny opinion.
1. It has a lot to do with organisational constraints.
2. It has a lot to do with service boundaries. If services are chatty they should be coupled.
3. What a service does must be specified in regards to which data it takes in and what data it outputs. This data can and should be events.
4. Services should rely and work together based on messaging in terms of queues, topics, streams etc.
5. Services are often data enrichment services where one service enrich some data based on an event/data.
6. You never test more than one service at a time.
7. Services should not share code which is vibrant or short lived in terms of being updated frequently.
8. Conquer and divide. Start by developing a small monolith for what you expect could be multiple service. Then divide the code. And divide it so each coming service own its own implementation as per not sharing code between them.
9. IaaS is important. You should be able to push deploy and a service is setup with all of its infrastructure dependencies.
10. Domain boundaries are important. Structure teams around them based in a certain capability. E.g. Customers, Bookings, Invoicing. Each team owns a capability and its underlying services.
11. Make it possible for other teams to read all your data. They might need it for something they are solving.
12. Don't use kubernetes or any other orchestra unless you can't it what you want with cloud provider paas. Kubernetes is a beast and will put you to the test.
13. Services will not solve your problems if you do not understand how things communicate, fail and recovers.
14. Everything is eventually consistent. The mindset around that will take time to cope with.
A lot more...
Well, in our +20 product teams with all serving different workflows for 3 different user types, the separate micro services are doing wonders for us for exactly the things you've listed.
My comment should just stop here to be honest.
Microservice and modularity is orthogonal, it's not the same.
Modularity is related to business concept, microservice is related to infrastructure concept.
For example, i could have a module A which is deployed into microservide A1 and A2. In this case, A is almost abstract concept.
And of course, i could deploy all modules A, B, C using 1 big service (monothlic).
Moreover, i could share one microservice X for all modules.
All confusion from microservice, is made from the misconception that microservice = module.
Worse, most of "expert advice" which i've learnt actually relate Domain Driven Design to Microservice. They're not related, again.
Microservice to me, is to scale. Scale the infrastructure. Scale the team (management concept).
Then you can have a mono repo that deploys to multiple micro services / cloud functions / lambdas as needed depending on code changes and programmers don’t have to worry about RPC or json when communicating between modules and can just call the damn function normally.
Though there were many attempts to do it, I would just mention Erlang and Akka.
The answer to your question is close to the answer for "what's wrong with Akka"
Solving this in the general case for every function call is difficult. Pure functions are idempotent, so you can retry everything until nothing fails.
But once you add side effects and distributed state, we don’t know how to solve this in a completely generalized and performant way.
But author clearly avoided the real reasons why you actually need to split stuff into separate services: 1. Some processes shouldn't be mixed in the same runtime. Simple example batch/streaming vs 'realtime'. Or important and not important. 2. Some things need different stack, runtimes, frameworks. And is much easier to separate them instead of trying to make them coexist.
And regarding 'it was already in Simpsons' argument, I don't think it should even be considered as argument. If you are old enough to remember EJB, you don't need to be explained why it was a bad idea from the start. Why services built on EJB were never scalable or maintainable. So even if EJB claimed to cover the same features as microservices right now, I'm pretty sure EJB won't be a framework of choice for anybody now.
Obviously considering microservices as the only _right_ solution is stupid. But same goes for pretty much any technology out there.
If I run a monolith and one least-used module leaks memory real hard, the entire process crashes even though the most-used/more important modules were fine.
Of course it's possible to run modularised code such that they're sandboxed/resources are controlled - but at that point it's like...isn't this all the same concept? Managed monolith with modules vs microservices on something like k8s.
I feel like rather than microservices or modules or whatever we need a concept for a sliding context, from one function->one group of functions->one feature->one service->dependent services->all services->etc.
With an architecture like that it would surely be possible to run each higher tier of context in any way we wanted; as a monolith, containerised, as lambdas. And working on it would be a matter of isolating yourself to the context required to get your current task done.
It’s old. The examples are in PL/I. But his framework for identifying “Functional Strength” and Data Coupling is something every developer should have. Keep in mind this is before functional programming and OOP.
I personally think it could be updated around his concepts of data homogeneity. Interfaces and first class functions are structures he didn't have available to him, but don’t require any new categories on his end, which is to say his critique still seems solid.
Overall, most Best Practices stuff all seem either derivative of or superfluous to this guy just actually classifying modules by their boundaries and data.
I should note, I haven't audited his design methodologies, which I'm sure are quite dated. His taxonomy concerning modules was enough for me.
"The Art of Software Testing" is another of his. I picked up his whole corpus concerning software on thriftbooks for like $20.
Having said that, if you want to eke out another 3x throughput improvement, then by all means, grab your OpenSwoole or ReactPHP or AMPHP and go to town. But PHP already has Fibers, while OpenSwoole still has coroutines. Oh yeah, and the try/catch in OpenSwoole is broken so good luck catching stuff.
Queues are awesome, just use queues almost all the time and then either build micro services or don’t.
It is about where the shoe fits. If you become too heavily dependent on modules you risk module incompatibility due to version changes. If you are not the maintainer of your dependent module you hold a lot of risk. You don't get that with microservices.
If you focus too much on microservices you introduce virtualized bloat that adds too much complexity and complexities are bad.
Modules are like someone saying it is great to be monolithic. Noone should upright justify an overly complicated application or a monolithic one.
The solution is to build common modules that are maintainable. You follow that up with multi-container pods have them talk low level between each other.
Stricking that exact balance is what is needed not striking odd justifications for failed models. It is about, "What does my application do?" and answering with which design benefits it the most.
The way I usually describe my preferred heuristic to decide between modules and microservices is:
If you need to deploy the different parts of your software individually, and there’s a cost of opportunity in simply adopting a release train approach, go for microservices.
Otherwise, isolated modules are enough in the vast majority of cases.
A failed lookup of a function is greeted by "Do you want to import X so you can call foo()?". Having a battery of architectural unit tests or linters ensuring at module foo doesn't use module bar feels like a crutch.
Now, it might seem like making microservices just to accomplish modularization seems like a massive overkill and sa huge overhead for what should be accomlished at the language level - and you'd be right.
But that leads to the second largest appeal, which is closely related. The one thing that kills software is the big ball of mud where you can't really change that dependency, move to the next platform version or switch a database provider. Even in well-modularized code, you still share dependencies. You build all of it on react, or all the data is using Entity Framework or postgres. Because why not? Why would you want multiple hassles when one hassle is enough? But this really also means that when something is a poor fit for a new module, you shoehorn that module to use whatever all the other modules use (Postgres, Entity Framework, React...). With proper microservices, at least in theory you should be able to use multiple versions of the same frameworks, or different frameworks all together.
It should also be said that "modules vs microservices" is also a dichotomy that mostly applies in one niche of software development: Web/SaaS development. Everywhere else, they blur into one and the same, but sometimes surfacing e.g. in a desktop app offloading some task to separate processes for stability or resource usage (like a language server in an IDE).
The typical example is that when you have to explain something is the job of X and Y. Usually this means that X and Y are breaking those boundaries. Just make a semi-private (or even public) API only used for that thing and you have a broken boundary again. Or push it on a message queue, etc.
I think, it certainly helps, but then again it doesn't prevent it. Having modules you have the same effect.
So in the end you get more spots where things can get wrong and more operational complexity. If you stick to using them right, I think you can also stick to using modules right with less complexities, better performance, easier debugability, fewer moving parts.
Hackers will find a way to do hacky things everywhere. ;)
Also this whole discussion reminds me of Linus discussing how he thinks micro kernels add complexity a very long time ago. Not sure if they should be considered modules or microservices though.
Sharing global state and so on while maybe it shouldn't be done lightly, without thinking about it can and does make sense. And in most modern environments it's not like the most quoted issues can happen too easily.
Also I strongly agree with pointing out that this is actually a niche topic. It's mostly big because that niche is where probably the majority of HN and "startup" people spend their time.
But in general,just write some damn code. Presumably you have to write code, because building a software engineering department is one of the most difficult things you can do in order to solve a business problem. Even with the smartest engineers in the world (which you don't have), whatever you ship is inevitably going to end up an overly complex, expensive, bug-riddled maintenance nightmare, no matter what you do. Once you've made the decision to write code, just write the damn code, and plan to replace it every 3-5 years, because it probably will be anyway.
Someday I will find a way to untangle this wishlist enough to turn into a design.
1) A remote call – inherently – fail. However, some local calls never fail. Either because they are designed to never fail or because you have done the required checks before executing the call. A remote call can fail because the network is unreliable and there is no way around that (?).
2) A remote call – inherently – can be very slow. Again because of the unpredictable network. A local call may be slow as well but usually either because everything is slow or because it just takes a while.
So if you have a call that may or may not be local you still have to treat it like a remote call. Right?
I think having a certain set of calls that may or may not be executed locally is not that bad. Usually it will just be a handful of methods/functions that should get this "function x(executeLocally = false|true)" treatment - which is prob. an acceptable tradeoff.
Those modern computers are a network mesh of small D/G/CPUs, and it's coming back to bite us sometimes.
RPC vs local calls is trivial in comparison and you can get that level of transparency out of the box with functional programming - it's just data in data out.
Using hindsight, most of the systems that pretend that the network part of invoking RPCs is "easy" and "simple" and "local" end up being very complex, slow and error prone themselves.
See DCOM, DCE, CORBA and EJBs RMI.
You need instead a efficient RPC protocol that doesn't hide the fact it is a RPC protocol - like Cap'n Proto (it's "time-traveling RPC" feature is very interesting).
Push the monolith to production, monitoring performance, and if and when performance spikes in an unpleasant way, "offload" the performance intensive work to a separate job server that's vertically scaled (or a series of vertically scaled job servers that reference a synced work queue).
It's simple, predictable, and insanely easy to maintain. Zero dependency on third party nightmare stacks, crazy configs, etc. Works well for 1 developer or several developers.
A quote I heard recently that I absolutely love (from a DIY construction guy, Jeff Thorman): "everybody wants to solve a $100 problem with a $1000 solution."
That said, server bottlenecks are not the only thing (micro)services are trying to address.
Yes, and that's the point.
> server bottlenecks are not the only thing (micro)services are trying to address.
The only other real advantage is scale of network throughput and isolation from other processes (in case some service is particularly volatile or prone to errors). Even those are a stretch as they're both solved/solvable by modern infra and just isolating code into its own job server (technically a "microservice").
Ideally we could flexibly deploy services/components in the same way as WebLogic EJB. Discovery of where components live could be handled by the container and if services/components were deployed locally to one another, calls would be done locally without hitting the TCP/IP stack. I gather that systems like Kubernetes offer a lot of this kind of deployment flexibility/discovery, but I'd like to see it driven down into the languages/frameworks for maximum payoff.
Also, the right way to do microservices is for services to "own" all their own data and not call downstream services to get what they need. No n+1 problem allowed! This requires "inverting the arrows"/"don't call me, I'll call you" and few organizations have architectures that work that way - hence the fallacies of networked computing reference. Again, the services language/framework needs to prescribe ways of working that seamlessly establish (*and* can periodically/on-demand rebroadcast) data feeds that our upstreams need so they don't need to call us n+1-style.
Microservices are great to see, even with all the problems, they DO solve organizational scaling problems and let teams that hate each other work together productively. But, we have an industry immaturity problem with the architectures and software that is not in any big players' interest in solving because they like renting moar computers on the internet.
I have no actual solutions to offer, and there is no money in tools unless you are lucky and hellbent on succeeding like JetBrains.
One thing I've observed in a microservice-heavy shop before was that there was the Preferred Language and the Preferred Best Practices and they were the same or very similar across the multiple teams responsible for different things. It lead to a curious phenomenon, where despite the architectural modularity, the overall SAAS solution built upon these services felt very monolithic. It seemed counter-productive, because it weakened the motivation to keep separation across boundaries.
If I want to calculate the price of a stock option, that's an excellent candidate to package into a module rather than to expose as a microservice. Even if I have to support different runtimes as presented in the article, it's trivial.
A different class of problem that doesn't modularize well, in shared library terms, is something with a significant timing component, or something with transient states. Perhaps I need to ingest some data, wait for some time, a process the data, and then continue. This is would likely benefit from being an isolated service, unless all of the other system components have similar infrastructure capabilities for time management and ephemeral storage.
I'm sure I could gain many of the advantages of microservices through an OSGi monolith, however, an OSGi monolith is not the hot thing of the day, I'm likely to be poorly supported if I go down this route.
Ideally I also want some of my developers to be able to write their server on the node ecosystem - if they so choose, and don't want updating the language my modules run on (in this case the JVM) to be the biggest pain of the century.
Besides, once my MAU is in the hundreds of thousands, I probably want to scale the different parts of my system independently anyway - so different concerns come in to play.
Yes, the same often happens in microservices, but the extra complexity of a distributed system provides a slightly stronger nudge to decouple that means some teams at the margin do achieve something modular.
I’m something of a skeptic on modular monoliths until we as an industry adopt practices that encourage decoupling more than we currently do.
Yes, in theory they’re the same as microservices, without the distributed complexity, but in practice, microservices provide slightly better friction/incentives to decouple.
I've done monoliths and microservices. I've worked in startups, SMEs and at FAANGS. As usual, nothing in this article demonstrates that the person has significant experience of running either in production. They may have experience of failure.
In my experience, microservices are simply one possible scaling model for modules. Another way to scale a monolith is to just make the server bigger: the One Giant Server model.
If you have a fairly well defined product, that needs a small (<30) number of engineers, then the One Giant Server model might be best for you. If you have a wide feature set, that requires >50 engineers, then microservices is probably the way to go.
There is no noticeable transition from a well implemented monolith with a small team into a well implemented Giant Server with a small team. Possibly some engineers are worrying about cold start times for 1TB of RAM, but that's something that can happen well ahead of time, and hardware qualification is something one needs to do for microservices too. Some of the best examples of Giant Servers are developed by small teams of very well paid developers.
The transition from a monolith to a set of microservices, however is a very different affair. Unfortunately, the kind of projects that need to go microservices are often in a very poor state. Many such adventures one reads about are having to go to microservices because they've already gone to One Giant Server and those are now unable to handle the load. Usually these stories are accompanied by a history of blog posts about moving fast and breaking things, yolo, or whatever is cool. The transition to microservices is difficult because there are no meaningful modules: no modules, or modules that all use each other's classes and functions.
I don't believe that, once a particular scale is reached, microservices are a choice. They are either required or they are not. Either you can scale successfully with One Giant Server, or you can't.
The problem is that below a certain scale, microservices are a drag. And without microservices, it's very easy for inexperienced teams to fail to keep module discipline.
I like that architecture - the services are abstracted with clearly defined boundaries and they are easy to navigate / discover. Not sure if Java modules satisfy the concerns of the author or other HN users, but I liked it.
Going back to systems thinking, flow control (concurrency and rate limiting) and API scheduling (weighted fair queuing) are needed to make these architectures work at any scale. Open source projects such as Aperture[0] can help tackle some these issues.
Monolith is better in almost every way. The only thing that still bothers us is the build/iteration time. We could resolve this by breaking our gigantic dll into smaller ones that can be built more incrementally. This is on my list for 2023.
I have worked on a monolith that solved the exact same problem. And it was straightforward to maintain and upgrade.
I feel sorry for future developers who will have to take over and maintain micro-services systems created today. Teams that can’t create maintainable, well designed monoliths, will create an even bigger cluster f** using micro-services.
You only need to agree on an API between the modules and you're good to go!
Microservices suck dick and I hate the IT industry for jumping on this bandwagon (hype) without thoroughly discussing the benefits and drawbacks of the method. Debugging in itself is a pain with Microservices. Developing is a pain since you need n binaries running in your development environment.
For our services, there is never a single point of failure. We use Resilience4j for retries, caching inside the dynamic proxy handler (using annotations to describe what is cacheable), and annotations to describe what can be done asynchronously.
In the cases where we use these, the caller is aware (via the interface documentation) that this is a distributed system where sometimes invocations happen really fast and reliably.
> In the cases where we use these, the caller is aware (via the interface documentation) that this is a distributed system where sometimes invocations happen really fast and reliably.
This is at odds with
> I have systems where the decision to put the module in the same process or to call it via RPC is done at runtime.
Either it's in the caller's choice (based on the documentation), or it's at runtime, but it can't be both and be equally reliable.
I did not say it is the caller's choice nor that it is equally reliable. It is a system composition decision. It is a system that is sometimes distributed and treated as such from a reliability point of view. Sometimes the actual implementation is more reliable and faster.
We delivered many talks on that subject and implemented an ultimate tool for that: https://github.com/7mind/izumi (the links to the talks are in the readme).
The library is for Scala, though all the principles may be reused in virtually any environment.
One of the notable mainstream (but dated) approaches is OSGi.
https://michaelfeathers.silvrback.com/microservices-and-the-...
I've often noticed that these boundaries are not considered when carving out microservices.
Subsequently, workarounds are put in place that tend to be complicated as they attempt to implement two phase commits.
It's on the same page as "yes, we could have written this in assembler better" or "this could simply be a daemon, why is it a container?"
As if an agile, gitops based, rootlessly built, microservice oriented, worldwide clustered app will magially solve all your problems :D
If i learned anything it's to expect problems and build a stack that is dynamic enough to react. And that any modern stack includes the people managing it just as much as the code.
But yes, back when ASP.NET MVC came out i too wanted to rebuild the world using c# modules.
Mastery is substantially about figuring out what rules of thumb and aphorisms are in place to keep beginners and idiots from hurting themselves or each other, and which ones are universal (including some about not hurting yourself, eg gun safety, sharps safety, nuclear safety).
And this is how I work all the time.
In Fuchsia, the device driver stack can be roughly split into three layers:
* Drivers, which are components (~ libraries with added metadata) that both ingest and expose capabilities,
* A capability-oriented IPC layer that works both inside and across processes,
* Driver hosts, which are processes that host driver instances.
The system then has the mechanism to realize a device driver graph by creating device driver instances and connecting them together through the IPC layer. What is interesting however is that there's also a policy that describes how the system should create boundaries between device driver instances [2].
For example, the system could have a policy where everything is instantiated inside the same driver host to maximize performance by eliminating inter-process communication and context switches, or where every device driver is instantiated into its own dedicated driver host to increase security through process isolation, or some middle ground compromise depending on security vs performance concerns.
For me, it feels like the Docker-like containerization paradigm is essentially an extension of good old user processes and IPC that stops at the process boundary, without any concern about what's going on inside it. It's like stacking premade Lego sets together into an application. What if we could start from raw Lego bricks instead and let an external policy dictate how to assemble them at run-time into a monolithic application running on a single server, micro-services distributed across the world or whatever hybrid architecture we want, with these bricks being none the wiser?
Heck, if we decomposed operating systems into those bricks, we could even imagine policies that would also enable composing from the ground up, with applications sitting on top of unikernels, microkernels or whatever hybrid we desire...
[1] https://fuchsia.dev/fuchsia-src/development/drivers/concepts...
[2] https://fuchsia.dev/fuchsia-src/development/drivers/concepts...
with a good branching strategy
In cases when a rollback is necessay, you'll often find yourself in a cycle where a breakage goes out, the deployment is rolled back, a patch is applied, a new deployment goes out and a new breakage happens, the deployment is rolled back, and so on. I am exaggerating how often this happens, but it does happen when you have thousands of engineers, and it happened often enough at my workplace that developers were unhappy and management pushed for breaking up the monolith.
Just because you can, doesn't mean you should.