Micro-monolith anti-pattern
chi.pl
chi.pl
The only problem is that microservices add overhead and require more tuning and control than monoliths. This is especially true if you give teams carte blanche to build things however they want with no shared standards.
Being an early stage startup guy I am probably biased, but I would argue it's best to default to a monolith and try to keep things modular with an eye towards later extracting microservices as A) you find incongruent workloads and B) your team is growing and you need to create some isolation so not everyone has to know everything.
Only once you understand the problem domain and workloads are you fit to draw service boundaries.
Recognize which time you're in and make the best choice for now (and some of your expected future).
If everyone designed everything with the expectation of Facebook levels of traffic, nothing would ever get done (your team isn't big enough), and that overhead would be wasted in 99.9% of cases.
Eh, I don't see this as an antipattern. Building disparate services but using the same frameworks/languages has tons of advantages and relatively few(none?) disadvantages.
A few:
- Engineers can move between teams and features more seamlessly
- Infrastructure and common functionality can be shared
- You will inevitably end up debugging someone else's code when it comes to on-call, that's gonna suck if it looks completely different/foreign to you than what you use day to day.
As often is the case, the best place is normally between those extrema. You probably want to restrict languages and frameworks, but you probably do not want to dictate any single one. Or, if you want to dictate a single one, you'll probably want a monolith.
-> depends
A -> B
B -> C
C -> nothing
D -> A
If A has changed, then on merging A, we should run D with new A. If C changed, then A, B, C, D are need to be tested. A flag or a tag for "non-backward compatible" should be added so on merging we can notify the developers (both producer of the library and consumer of the library) aware of breaking changes in their review queue. If breaking changes keep popping up, time for the tech leads and team managers of both sides to meet and understand how to avoid breaking changes so often.
I don't know, someone working with large tech stack should comment on their experience. But we can't just use a text file and an email hoping someone would follow up or realize.
The language runtimes and dependencies already add a lot of boilerplate, even for the simplest of services. You will also need databases and data sources in the same box.
Laptop have little memory. MacBook are no exceptions.
To put things in perspective: There are in fact dozens of microservices running on every laptop out there. They're called "system processes" or "daemons".
Say you boot up the recipes service that needs your auth service. In this scenario mock the auth service either directly in code (you should be able to do this if you use DI for the actual service repo) or by static responses for your work scenario into a proxy/stub server like SimpleHTTPServer from Python [1]. When you're recipe service calls out, it's hitting the localhost:8002 for the python service which returns the same general application/json-v1-auth data. You can even go a bit further and make the Python code adaptive based on input.
You might benefit from splitting the system apart with events too. For example, duplicate some of the authentication or authorization data across the micro services. Let's say your recipe service needs to know who can read a given recipe. Use JWT or some other tokening feature to say who the current caller is (in a secured way). In the recipe's service, keep track of that user's authorizations locally. JWT tells you they are authenticated, the local DB can tell you their authorization. Now you don't need the auth service since you can probably create valid tokens. When the user's authorization changes, an even tells the recipe service. While there is a small period of time in which the person might be able to access something they shouldn't, in a functioning distributed system this isn't too bad. If you can't handle such failures, you might now want a micro service architecture.
Caching other microservice's data in general just sounds like a bad idea.
The same is true if you put groups in a JWT. Caching is caching. You always have to weigh the benefits against the cons.
In general this is a problem with distributed systems. CAP and all that. What do you do if your network is partitioned such that the recipes service can service requests while the auth service can't notify the recipe service of changes? You can bring the whole system down. You can continue to allow operations based on the cached rights. You could limit operations if you haven't gotten a heart beat from the auth service in while.
Back to the example, say group changes occur rarely. Say over that, network partitions (including message system failures) occur rarely too. Chances of both happing is P(partition, msg_failure). Probably small chance. If you can live with that, you're life might be better. If you have hard requirements to always get the auth, you might need to couple. Even then you're not really guaranteed that the latest data from the auth service is the latest data concurrently.
Finally, even if you have coupling, you can still design the system to use profiles to pick the service. When you're running locally, don' go to the real auth system, just go to SimpleHTTPServer. Don't attempt to run the auth service locally. SimpleHTTPServer will probably use fewer resources than your micro service using some complex library.
It can be tempting because an engineer thinks "I can optimize this and reduce the network overhead using this creative protocol." But now they have put themselves in a situation where if anything in that custom protocol changes they will have to update every single microservice in their backend ecosystem.
This is just API versioning, that needs to happen regardless of the protocol you choose.
edit I see you mean the protocol itself (like if gRPC goes from version 1 to 1.2 or something). Yes, that's a thing... most protocol developers are careful about backwards compatibility to allow for some services to be behind a few minors to account for this. Also, HTTP still isn't invulnerable to this...HTTP/2 is very much a thing.
Isn't that just called a business requirement?
> you have to change and deploy every last one of your services.
How is that different to code duplication as the article recommends? When business requirements change all affected code points needs to change.. you can't just wish that away.
-
The real problem here is mismanaging seperation of concerns.
Cqs mostly solves the hard problems. But having shared read objects will, at times, require multiple systems to make use of the same domain logic. Whether you're better off with code duplication or a hard shared dep is a trade off up to you to figure out. I tend to go with a mix that makes sense based on expected frequency of change and how breaking the changes are likely to be.
Another microservices is also one answer but can be an unworkable bottleneck for a lot of these types of situations.
Do you? As long an appropriate versioning strategy is in place, it seems to me that you would only need to update the service that prompted the code change, leaving the other services to use the older version. If the other services need to have their code changed as well, then that'd be the case whether or not you used a shared library, except you'd have to update the code in 12 places instead of just one.
would the alternative be changing every one of your services, then deploying?
This is actually quite important from both licensing and security point of view, and having such list available encourages discussions. I'd like to write one of my services in Go, but does my company have the resource to maintain Go application after I depart? What are the strong technical reasons that the service must be written in Go rather than in Python or in Java? If Go is approved, then I need to make sure management offers training / materials for developers to learn Go, and that I am available to train others (have good documentation, brown bag sessions and what not).
Read the post I replied to. It advocates requiring a single framework.
Yes I am worse off because the framework sucks but there is no incentive for it to improve because everyone is forced to use it. I would love an approved list as you suggest because at least that would allow some choice, but even better than that would be working at a place that trusted me as an engineer to make those decisions myself.
Question about this: if we're to take the above to heart, doesn't that imply that we may implement some common functionality multiple times? It seems there's a real tension here between "code re-use" as a generally laudable goal and the need to avoid sychronization of testing/deployments.
I'm curious how others out there have dealt with this particular issue.
But really, it's a constant struggle. Micro-service teams are semi-autonomous and are going to recreate functionality at will. That's a communication problem mostly though.
[1] Disclaimer: I've only been programming for ~10 years, and professionally for ~3 years
[2] Personally, I'm currently using Sonatype Nexus
[3] I'm currently grappling with trying to automate building a multi-targeted assembly .Net assembly, and the co-dependency between VS, msbuild, and Nuget gives me agita
You're barking up a tree that was cut down a few years ago.
I also think that functionally, there's a very small difference between a library with a public API and a microservice with a public API - the only main advantage is that microservices can be written in any language, but that may not be important for many businesses.
I think you should never "take it to heart", at least not in the meaning that you should follow advice slavically. It's just one more tradeoff you should be thinking of to avoid cargo culting.
Nit, but you probably meant "slavishly" (even given that the author is Polish)
The other microservices all included this shared library, and it benefited devs by providing them with an easy way to create a new microservice quickly without having to recreate a bunch of boilerplate. It also provided an easy way to centrally manage a few things like the version of express (the node.js web server we chose) and a few other things. By updating that central lib the rest of the microservices would all get upgraded automatically the next time we rebuilt them.
Combine that with a reasonable deprecation schedule on old versions of the library and let teams upgrade when they're ready.
Now you have the best of both worlds - a library that keeps moving forward with a known window (per the deprecation schedule) when they can completely drop old functionality, all while other teams are happily using the library.
Ofcourse, a number of things needed to pull this off:
* independent deployments so services don't automatically share the same library versions
* communication tools so library owners can publish a deprecation schedule and the right people are notified effectively
* verification mechanisms so library owners can make sure everybody has upgraded before pushing out breaking changes
That isn't feasible in a lot of real world contexts.
- what if the same service is used by web and mobile applications?
- the backend services usually have access to data that shouldn't be exposed in the DMZ. By keeping all of the services that communicate with backend systems behind the firewall and having the front end services exposed in the DMZ, you have another layer of protection with the only port being opened between the DMZ and the internal network is port 80/443
Lots of apps use S3 as a common store. Is S3 an antipattern? The same goes for things like SQS, Google Pub/Sub, Google Datastore, etc. In my opinion, the idea that microservices must be isolated silos is misguided.
There are many things that are top-down (high-level control plane, many small individual, stateless parts controlled by it), but the opposite of this is the bottom-up "shared substrate". Things like data storages and buses that have the responsibility of storing state or shuffling data between actors.
In fact, I consider microservice that's a silo — that is, has its own private data store — to be an antipattern in many cases. To borrow an old phrase, data wants to be free. Today's backend patterns are very API-centric, with APIs acting as awkward gatekeepers to small amounts of data. It's a much better idea to invert this, Haskell-style: Let the data be first class, and orient your APIs to work on your data.
This seems to me the same principle as "no libraries, instead keep a spec and re-implement the domain logic in each service" discussed elsewhere in this thread.
You still have a common dependency of all services (the spec/design doc) and you'll still have to modify them all if the dependency changes. Except now you also have heaps of duplicate code.
This is a great opportunity to have shared libraries or at least schemas across different microservices. In fact having each service write their own parsers or serializers would be a mistake for me.
Wat
What?