Hacker News Reader

Comment by hinkley | Hacker News Reader

From what I understand, some time sequence databases use some sophisticated compression techniques. Are you there already or is that on the roadmap?

I think it would be kinda cool in a way to go back to dual-machine debugging but I’m not sure everyone else would agree with me.

View on HN

There's fair bit of compression already, but actually I haven't done that much industrial spying into time-series databases, good idea, thanks.

Also, I've tested dual machine debugging with BugJail, as part of spike towards CI/QA/SIT/UAT/Pre-Prod use. It works okey-ish but the (crappy 1GB) network became bottleneck and it was slower than single box.

If 1GB wired doesn't work well, WiFi will be worthless. Compression is a really good idea

My "Fair bit of compression" was massively simplified: there isn't one true canonical "compression", but dozen different places and ways where and how to compress.

BugJail does compression around storage layer, but nothing before the network hop from capturing agent to processing server.

And fully agreed of course; when BugJail graduates to CI/QA/SIT/UAT/PreProd use, it will need compression before the network hop. But that is some time away. Currently the agent and server run in same box, and compression between them has negative net effect (compression uses more CPU than what shifting around less data saves).

statsd might be a good example there. Local coalescing before reporting upstream.

Basically they didn’t make the client smart, they made a proxy and spam loopback.

Reply on news.ycombinator.com