Hacker News Reader

Comment by jasonhansel | Hacker News Reader

> Structured logs are non-negotiable

I disagree with this. Logs should only be used by human operators; if you want something machine-readable, you should expose metrics via some other channel or record errors in a dedicated service.

Sure, some tools will let you read structured logs quite cleanly, but it will never be as nicely formatted as something designed, first and foremost, for humans. Of course, you may need to (say) tack on a request ID to avoid logs from different threads/goroutines/etc from getting interleaved, but that doesn't require everything to be in JSON.

View on HN

There's some logs which you want machine readable that are not necessarily metrics.

Let's say you have 10 services in serial, and you want to get precise information on how that request was performed - because your customer reported it performed super bad. To do that you might need to get logs from all those 10 services and align them. If they all are human readable and you can filter by a common request-id, that's easy to do. And you can even have tooling on top of it that automatically determines what might have happened by looking at other fields of the log entries.

Metrics are mostly just pushed to a metrics system, and that moment you don't care about them or the relation to an end-to-end request anymore. So while all services might have emitted a metric for the request, looking at the overall metrics won't tell you if outliers in the services refer to the same end-to-end request or different ones.

Structured logs can still be human readable.

Ex:

[2022-01... INFO] trace=abcd123 <msg>

There is a structure which can be parsed.

Yeah, I'm referring to the "log everything as JSON" format used by (say) MongoDB.

I don't care if it's in json, fmt or your own hacky format, just chose on and structure all you output so that it's predictable where I have to look.

> Logs should only be used by human operators

I suspect that you will find a lot of resistance to the idea. At least I have. Machines should be consuming metrics, not logs, unless there aren't any alternatives. That statement never seems to be controversial, but people... just don't do it, for whatever reason.

Also, the author seems to want to do a poor man's version of distributed tracing.

Reply on news.ycombinator.com