gRPC for Microservices Communication
techdozo.dev
techdozo.dev
You can hear this nonsense from some poor quality bloggers. People who are serious about it say things like "for most startups microservices is bad idea", "monolith should be starting point in most cases", "there is a price that comes with microservices", "modularised monoliths are often pefect fit" etc. this is coming not from random internet dudes but from Sam Newman author of "Building Microservices" and "Monolith to Microservices", Martin Fowler etc.
Microservices/services/monoliths are one of design kinds, they best fit into different setups.
References:
* https://www.theregister.com/2020/03/04/microservices_last_re...
* https://www.youtube.com/watch?v=GBTdnfD6s5Q
etc.
Also, I don't like how the discussion about "microservices" is drowning the discussion about plain services. Services are a natural consequence of Conway's law when your organization gets bigger. You can see parallels with the first wave of OO programming for software projects, where the goal of having a public interface and private internals was to make projects scale when 10/100/1000 times more people are working on it. This is exactly what's happening with services: a team maintain a public API, and private internals (and the developpers from the other team depend on the internals anyways because reality isn't that easy). Then it's easy to see what microservices are: ravioli code.
In a way, services and microservices are OO programming in the Alan Kay sense: late binding, message passing, local retention and hiding of state.
Yeah I think this really sets back the conversation when, in my mind, microservices are just services with the acknowledgement that minimum size should not factor into breaking off a new service.
> This is exactly what's happening with services: a team maintain a public API, and private internals (and the developpers from the other team depend on the internals anyways because reality isn't that easy). Then it's easy to see what microservices are: ravioli code.
This is a moot point. Following up on you OO example, there is not reason you can have separate public and private implementations within a (modularized) monolith.
The biggest downsides of (modularized) monoliths lie in 1) scaling - e.g., you can independently scale and load balancer each service, 2) releases - you can have independent releases (at the risk of maintaining version compatibility).
I feel this line of argument is disingenuous and is based on a mix of strong personal opinions and very limited to non-existent insight onto which problems other orgs experience.
For starters, the main selling point of microscopes is not performance. It is organizational advantages. It is a way to draw clear and crisp lines regarding ownership and ops. Small teams are able to design and deploy and run and troubleshoot small bits of a large machine without wasting time in back-and-forths with other teams, and they can put in place safeguards so that if other teams screw up then their blast radius is limited.
But regarding performance, let's not fool ourselves into believing that there's always a fatter network pipe and a beefier box to deploy to. Often there is, but often there simply isn't. Once you are forced to deploy your monolith into multiple instances, most of the criticism directed at microservices is rendered moot. Also, beefier boxes are far more expensive to operate, so let's not pretend there are no costs.
Also, there's the reliability aspect. If you want to have a reliable service then you need redundancy, which means multiple deployments. Once you have to deal with load balancers serving traffic to multiple instances of your monolith, there is no longer a significant architectural and ops difference in whether you operate a monolith or shave a service or two out of it. Thus most of the criticism directed at microservices is rendered moot.
Lastly, some orgs are ok with serving users in a very limited geographical region. That's perfectly fine. Other orgs might have users in different parts of the globe. Do you expect them to tolerate latencies in the 200-500ms range routinely, or do you find acceptable to reel in those latencies back to the 50ms territory by deploying a part of the service closer?
I get the Monolith first principle, but you're fooling yourself if you believe that only FAANG-level companies can benefit from it.
That's true, but at this point we should clarify what we mean by microservices. By "microservices", I mean "you end up with more services than engineers". If you've got a team for each service, then I would just call them services. I also don't agree that back and forth with other teams is always a waste of time, especially since just after this you talk about limiting their blast radius if they screw up. That's probably necessary if your org doubles every year or already has thousands of engineers, but that's not for everyone.
> But regarding performance, let's not fool ourselves into believing that there's always a fatter network pipe and a beefier box to deploy to. Often there is, but often there simply isn't.
I don't think we are talking about the same thing here. My point was that if you don't have network effect (like Twitter or Facebook), there's no need to put all of your users into the same box. You could have the extreme opposite, one instance for each users. So you only need to be able to handle your biggest client.
> Once you are forced to deploy your monolith into multiple instances, most of the criticism directed at microservices is rendered moot.
I don't think that's true. Having multiple instance of a monolith is not the same as having multiple different services.
> Also, there's the reliability aspect. If you want to have a reliable service then you need redundancy, which means multiple deployments. Once you have to deal with load balancers serving traffic to multiple instances of your monolith, there is no longer a significant architectural and ops difference in whether you operate a monolith or shave a service or two out of it. Thus most of the criticism directed at microservices is rendered moot.
> Lastly, some orgs are ok with serving users in a very limited geographical region. That's perfectly fine. Other orgs might have users in different parts of the globe. Do you expect them to tolerate latencies in the 200-500ms range routinely, or do you find acceptable to reel in those latencies back to the 50ms territory by deploying a part of the service closer?
Again, I don't think that's true. You can have one/multiple monolith instances in each region. You may argue that it's not strictly a monolith if there is a routing and load-balancing service in front of it, and that's true, but it's also not a microservice architecture.
That's not what "microservices" is.
If you are not familiar with the concept then you should refrain from post personal assertions on guidelines or best practices.
That's how it sometimes ends up. You said:
> For starters, the main selling point of microscopes is not performance. It is organizational advantages. It is a way to draw clear and crisp lines regarding ownership and ops.
But that's just regular old services. The "micro" in "microservices" usually implies smaller services, and one extreme of that is having more services than engineers. That's something I saw happen, heard about and read about. Microservices can be abused by making them too small and making too many of them. Uber is an example of that.
> microservices are just services with the acknowledgement that minimum size should not factor into breaking off a new service
Thanks to you both for the perspective.
How do you fix that ?
You run away as fast as you can is what you do.
One perspective is that we were negligent and caused a bug, but another possible perspective is that maybe the user service team should have caching instead of every other team building their own caching for the user service.
Payload is binary, it's fast -> Anybody can read the payload but it's structured -> Structure is redundant, look how lightweight is this -> Payload is binary, it's fast
The problem in software world is, there's no silver bullet for any particular problem but there are technology fashions claiming otherwise.
CORBA (as was DCOM) provided an object model where the client would make method calls to object by identity. This provided "locality transparency" — an object looks identical in the client as in the server. The client object is just a "stub" in CORBA parlance (or "proxy" in DCOM, if I remember correctly), and method calls are transparently translated to RPC calls. The consequence of this is that a client can hold onto entire object graphs, and gets very tightly coupled with the server. For example, a single client can keep tons of objects "open" inside the server just by virtue of having references to them. Very careful tracking of object references would be needed. Both CORBA ans DCOM use explicit reference counting; if a client gets this wrong, it can cause memory leaks inside the server. Not great for scalability, and a big weakness in terms of security.
gRPC is more like DCE RPC or Sun RPC, while leveraging a bunch of the web stack which didn't exist back then. There's no denying that technology is cyclical in some ways, of course. But we're not exactly coming full circle. We bring back some things and integrate it into the new stuff. The tooling around gRPC is miles ahead of what we had back then, while being based more or less on the same principles.
Interestingly, there's a similar (but maybe more depressing) cyclical story with dynamic vs. static typing in programming language. Python, Ruby, PHP, etc. allowed more people to shift away from commercial tech stacks (especially the horrific Java app server stuff in the early 2000s) while building out super-productive web frameworks. JavaScript allowed us to write web apps that were capable of competing with desktop apps. But in the process we gave up static typing. Millions of person hours have been spent in blogs and discussion boards defending, and extolling the superiority of, dynamic typing — yet this was found (in my view) to be a fallacy, and something of a shared delusion among developers drunk on web frameworks like Rails.
Bingo. I learned C++ and then Java first, so I've always been biased towards static typing. When the whole MEAN stack (including non-relational databases i.e. just shoving tons of JSON back and forth) become the cool thing, I refused to buy into the hype and continued using Java, even if it didn't make me popular with the newer developers.
And now I make good money cleaning up Node / Python monstrosities that grew too big for dynamic typing to handle. The worst thing about these is that somewhere in the last 4-5 years, the dynamic advocates began to realize that static types really did make sense, and so now they've added all sorts of hacks like Typescript or Python typing libs (Pydantic) and annotations that try to bridge the gap.
But it doesn't work. The few actual benefits of "simple" languages like Python and Javascript (e.g. One Way to Do Things and quick prototyping) are a thing of the past. Instead, a typical Python or Express web app now has just as many unintelligible annotations and verbose "interfaces" or model objects as an "enterprise" Java / C# app had 10 years ago.
And the pseudo-types aren't even all that helpful. The IDEs can sometimes understand things, but usually not 100% like they would with a Java app. Rarely does auto-complete or large refactoring work. Typing errors aren't always found at "compile" time - still plenty of surprises discovered only at runtime.
Readability is just as bad as the typical Java-Spring app with all sorts of magic going on underneath the covers at runtime. There is no way to just glance at a simple controller and understand what's going on without knowledge of increasingly bloated frameworks.
It's just a mess. At least I'll have plenty of work for the next decade.
Recently, I was working on building the backend for our Speech Recognition engine and it seems like the message size limit came in the way of being able to perform batch/offline inference. I might be lacking some of the intuition behind designing and deploying such services, and am curious if you guys have run into this issue. The platform doesn't seem to accept the channel options from the client.
That and the issue of no easy way to do gRPC calls from a browser yet, and the resulting architecture then needing to have been built the ground up with considerations for an extra piece - a gateway, or something like Envoy to make it more accessible over the web. I feel like that would be okay if I was working on a project with k8 ingress set up already and already utilizing microservices. But for getting running it seems like added complexity. Not hating on it or anything, just sharing my experiences and thoughts, hoping that some of you might've had experience reasoning and dealing with such designs and can offer some insight.
Disclaimer: I am one of the creators of Kreya
You still have to unmarshal json into some sort of object/struct.
With gRPC you also have a standardized middleware API that is implemented for "all" languages. The concepts cleanly map across multiple languages and types are mostly solved for you.
Adding to that you can easily define some conventions for a proto and make amazing libraries for your team. At a previous job I made this: https://github.com/CaperAi/pronto/
Made it super easy to prototype multiple services as if you mock a service backed by memory we could plop it into a DB with zero effort.
I think this "gRPC vs X" method of thinking isn't appropriate here because protos are more like a Object.prototype in JavaScript. They're a template for what you're sending. If you have the Message you want to send you can serialize that to JSON or read from JSON or XML or another propriety format and automatically get a host of cool features (pretty printing, serialization to text/binary, sending over the network, etc).
Second this. I think its also really important to consider the "trap" of going in on gRPC, but using something like grpc-gateway to also spit out JSON as a "backup". We did this for a project I was on, and the JSON API was the thing everyone else used (because up until then, everything there was a JSON API, naturally). As a result, unless a consuming team was willing/able to get involved in the protobuf definition internals, most of our consumers didn't reap any benefits from our typed protobuf API.
I know it really isn't hard to do, but lowest friction denominator wins far too often in a feature-focused environment :-\
You can automate a lot of this which essentially makes the normal level of friction easier.
The way you write your deployments can also know about this convention and automatically generate the environment variables you need (didn't get to this point). I hope one day I get a chance to work on building some tooling like this, open sourcing it, and making it easy for others to adopt a similar mentality.
Amiga got right what Unix got wrong: A well-structured, open binary protocol is almost always preferable to a "plain text" protocol because the parsing overhead really adds up.
That said, I'd like to see better support for C# code generation from proto files. For example, the support for nullable types works ok but is a bit cumbersome.
(Please, someone tell me I'm wrong. Please.)
protoforce.io has better modeling and implementation design, supports also 2-way communication, while remaining sane to read and debug.
/disclaimer: one of the authors
And is it open source? (apparently not, and is only free for non-commercial uses)
Kinda hard to compete when grpc and cap'n'proto are fully FOSS