Hacker News Reader

Comment by nsonha | Hacker News Reader

To be fair this "worse is better" thing permiates all aspects of tech: c++, java, python, wasteful text based protocols...

View on HN

> wasteful text based protocols…

Until you’re the unfortunate person trying to debug an API deployment where the team behind the API decided to use ‘much more efficient protobuffs’ making them entirely unreadable and impossible to troubleshoot. Modern processors and networks are fast enough to allow us to spend a few extra bytes making protocols easy to work with as well as ‘efficient’, and I’d argue that at scale, those ‘efficient’ protocols waste probably 100x more developer time troubleshooting than their text based counterparts. YMMV.

This sounds like a tools problem though. Bytes and hex aren't readable until you decide then to characters and use a text renderer to display the.

I can't imagine this isnt possible with protobuf and the right debugging tools (though it's not my area of expertise)

Sometimes you just want to dump the bytes that are going over the wire. Sure you could pipe the output to a protobuf parser, but then you'd need to make sure the output of tcpdump or ngrep or whatever is what the parser expects. As an example, I needed to track down why some service traffic was getting misrouted, and I found the problem by tcpdumping the traffic and realizing one service was silently corrupting a header value - I could have added more debug logs to either service, but that would have taken a lot longer to implement and deploy.

Personally I'd rather stick with the simpler protocol until someone runs the numbers and finds something more efficient would improve performance or save money by a significant enough amount.

> until someone runs the numbers and finds something more efficient would improve performance or save money by a significant enough amount

This argument must be the ultimate defense in code review. The thing is, we have the field of theoretical computer science and things like big O so that we can reason about code without having to do benchmarks about every little thing, because that's impossible in practice.

Regarding protocols, I'd argue that the most important thing for them is to be efficient, at least when IO is still the biggest bottleneck of computing. The other comment already points out debug being a tooling problem.

One of the biggest culture shock moments when I came back to the web world after spending time in high performance environments was how many resources web applications were willing to waste on parsing and sending terribly inefficient wire formats.

I’ll never understand why we’d make the trade off of making crazy inefficient human readable protocols for talking between computers instead of learning wire shark.

They're not just inefficient, they're also harder to work with! Binary formats are rigid and less corner-casey. Human readable formats are a total false economy.

I think you’re correct, it is a tooling problem, but that’s just as valid a problem class as any other. Part of the issue with solving tooling to deal with binary formats is the lack of self documentation. Reading JSON off the wire, its structure is immediately usable and understandable, whereas in most cases for a complex API, a binary protocol requires some foreknowledge of the structure of the data coming back. In the future, if we can create an efficient combination of binary protocol and self-documenting data structure, I’ll gladly change my mind entirely.

> I’d argue that at scale, those ‘efficient’ protocols waste probably 100x more developer time troubleshooting

In another comment you said there should be numbers to back this sort of claim

Anyway, the problem with our software stack is that there are so many layers, so if we get performance wrong at every one of them, it'll compound terribly at the top. We should have some sort of strategy to decide priorities at each layers, say, lower layers like OS, container, programming language, network protocols, should be about performance, then we can be wasteful at higher layers, the domain code, if that gives us better abstractions and maintainability.

in the case of python: "better is better" ;)

As in "very slow though flexible is better".

I get why you are saying an interpreted language is slow but given that people use python to orchestrate most compute intense (AI) and data-heavy (data science) work, it obviously does a pretty excellent job of being pretty 'fast' in practice - ie offloading the 'slow' bits to dedicated code.

as it stands right now, python is more complex and has worse type system comparing to typescript. It has all of javascript warts and more (those confusing underscore methods, GIL, pyproject.toml is only now standardized).

The good things about python have nothing to do with the language, it's the ecosystem and standard library (which is still not as good as, say, java's). So I'm not quite sure why people say javascript THE LANGUAGE is bad, but python the language is "better".

Reply on news.ycombinator.com