Hacker News Reader

Comment by geophile | Hacker News Reader

You are very attached to this "voluminous" point. What do you mean by it?

As I said, responding to another comment of yours, a distributed system I worked on produced a few GB a day. The logs were rotated daily. They were never transmitted anywhere, during normal operation. When things go wrong, sure, we look at them, and generate even more logging. But that was rare. I cannot stress enough how much of a non-issue log volume was in practice.

So I ask you to quantify: What counts (to you) as voluminous, as in daily log file sizes, and how many times they are sent over the network?

View on HN

> You are very attached to this "voluminous" point. What do you mean by it?

I mean "a lot" or more specifically; "a whole lot."

Here is an exercise which illustrates this. For the purposes here, assume ASCII characters are used for log entries to make the math a bit easier.

Suppose the following:

  Each log statement is 100 characters.
  Each service invocation emits 50 log statements.
  Average transactions per second during high usage is 200tps.
  High usage is on average 2 hours per day.

  100 x 50 x 200 x 60 x 60 x 2 = 7_200_000_000 = 7.2GB / day

> So I ask you to quantify: What counts (to you) as voluminous, as in daily log file sizes, and how many times they are sent over the network?

The quantification is above and regarding log entries being sent over a network - in many production systems, log entries are unconditionally sent to a log aggregator and never stored in a local file system.

7.2G/day doesn't sound terrible. And I'd reduce it by a factor of 25, since in normal operation (i.e., not looking into a problem) I would have either 2 log statements per call (entry, exit), or none at all. It might be more than 2, if I needed detailed logging.

But in normal usage, even in the scenario you describe your argument about log volume is not convincing.

Reply on news.ycombinator.com