Comment by packetlost | Hacker News Reader

Comment by packetlost | Hacker News Reader

Parent Full thread

I started developing a tracing/span library that does just this: log messages are "global" (to a system/org) hierarchical "paths" + timestamp + a tagged union. The tagged union method allows you to have zero or more internal parameters that can be injected into a printf (or similar style) format string when printing, but the message itself is only a few bytes.

The benefits to this approach is it's dramatically easier to index and cheaper to store at any scale.

One thing I think people don't appreciate about logging efficiency is it enables you to log and store more and I think many don't appreciate how much even modest amounts of text logs can bog down systems. You can't read anything, but you filters easy and powerful and you can't filter something that doesn't exist.

Another thing people won’t appreciate is ANY amount of friction when they “just want to log something real quick”. Which has merit, you’re debugging some garbage, and need to log something out in production because it’s dumb, harmless, quick, and will tell you exactly what you need. That’s why I think you need a sort of fallback as well, for something like this to capture enough mindshare.

How did your solution work out in terms of adoption by others? Was it a large team using it? What did those people say? Really curious!

It doesn't really replace something like print-line debugging, but the type of system that benefits/can use print-line debugging would see no benefit from a structured logging approach either. The systems I'm targeting are producing logs that get fed into multi-petabyte Elasticsearch clusters.

To answer your question: the prototype was never finished, but the concepts were adapted to a production version and is used for structured events in a semi-embedded system at my work.

There are logging libraries that do this. The text template is logged alongside a binary encoding of the arguments. It saves both space and cpu.

Yup, I'm aware. My focus was more on scaling it out to large aggregation systems.

Reply on news.ycombinator.com