Comment by bru | Hacker News Reader

Comment by bru | Hacker News Reader

Some of the weaknesses can be tempered by not using HTTP to communicate between the microservices:

- "slowdowns on the order of 1000%"

- " bunch of code necessary to marshal/unmarshal data [...] there are always dragons in there.

And also problems of versioning, data integrity, etc.

I've had those problems in a microservices architecture. That's things that are solved by protobuf[0]. Your servers exchange small, efficient structured data and you get tons of other benefits ({un,}marshaling for free, integrity, versioning, ...).

Potential downside: a language you want to use having no protobuf API.

Finally, I see another downside to the microservices architecture: it may be decided that the smaller, decoupled code bases should be stored in multiple CVS repos. Which turns into a nightmare: a single bugfix may span across multiple repos and there is no clean built-in way to links commits across them, you still should sync the interfaces (e.g. with git submodules), etc. This is a thing I've witnessed firsthand, and proposals to merge the repos were dismissed since "We [were] using a microservices architecture". Yes, it's a mistaken implementation of the microservices paradigm, but it still happens.

edit: I recommend protobuf not by preference over other equivalent solutions, but because it's the only one I know and have used. Alternatives are evoked below.

0: https://developers.google.com/protocol-buffers/

A network protocol is only a small component of a microservice in terms of affecting performance, by far the biggest difference is an in-process call vs network/IPC call. The latter is at least hundreds time slower due to how computers work [0].

I'm talking about a function call overhead only so if the actual processing takes more than a few milliseconds it stops being important.

[0] http://www.eecs.berkeley.edu/~rcs/research/interactive_laten...

I really recommend against using protobuffs. There are long standing bugs that Google just refuses to fix in the public version. I can't remember what they are off the top of my head, but I know a semi-prominent YC company that uses them and they pull their hair out all the time.

Just use zerorpc. It's more reliable than zeromq + protobuffs and it comes with a bunch of freebies, like built in heartbeats, streamed responses, etc.

How about Cap'n Proto? Their marketing is great, but I don't recall seeing many experiences with it, so I'm asking for feedbacks here.

First I'm hearing of it. Looks really promising, but without any experience with it I cannot really say.

They wrote it specifically for sandstorm.io, so they're dog-fooding it.

Take a look at GRPC (http://grpc.io). It addresses a lot of the long-standing complaints about using Protobuf as an RPC solution.

Having used protobufs, they are not magic. Sure they're faster to iterate and use than HTTP but not significantly so in many use cases.

They still marshall/unmarshall but they do a better job hiding it from you.

The network is still unreliable, insecure, slow, etc. and so will still have many of the same failure modes as HTTP. A better Corba is still Corba... Whether you call it XMLRPC, RESTful, protobufs, Thrift, Hessian or what have you.

And goddamn I hate the optionality of all fields that is "best practice" with protobufs. It makes processing any somewhat complicated data structure a PITA.

Yep, when evaluating whether to go with more microservices we looked into RabbitMQ for the transport and protobufs for serialization. In the end, we decided to roll the existing microservices into the monolith, which was by far the better decision, as we didn't need the scalability.

When you were evaluating AMQP, did you consider a lightweight RPC system like Nameko ?

http://lucumr.pocoo.org/2015/4/8/microservices-with-nameko/

No, we didn't get that far, but this looks very interesting, thanks for the link!

Instead of protobuf, we went with Thrift ( https://thrift.apache.org ) and have been happy with it so far. Our GoLang, Java, and Python components all use it to talk to each other.

I have a feeling even protobug is a lot slower than a monolith that can just pass data in memory. You're still opening network connections, probably to other servers, and allocating memory on the receiving end etc.

Reply on news.ycombinator.com