Building low-overhead metrics collection for high-performance systems | Hacker News Reader

Building low-overhead metrics collection for high-performance systems

blog.wallaroolabs.com

71 pointsspooneybarger10 comments

Hi, I'm the author of this post and I'll be happy to answer any questions here.

Wallaroo seems like a well-thought-out system, but it’s hard to find any case studies or mentions of real-world enterprise/startup use of the product, and therefore it’s tough to justify trying it out. Are any of those stories coming down the pipeline as well?

Thanks for the encouraging comment, we've spent a lot of time on Wallaroo's design so we're happy to see people appreciate it.

We have a few clients who will be running Wallaroo in production in the upcoming months. We're still a fairly new product, hoping to have one of those stories out in the not so distant future.

Is there anything you'd like to gain from one of those stories? Would be helpful to us to have a target audience in mind.

Thanks for the write-up. Why did you DIY instead of using an existing system like e.g. Prometheus?

We didn't set out to replace Prometheus, our Metrics UI was designed to give Wallaroo users insight into several parts of the system to spot bottlenecks while developing applications.

What we did do from scratch was write the instrumenting of Wallaroo in order to collect the metrics that could be used by our Metrics UI or Prometheus.

We thought it'd create a better user experience to provide a tool that provides the metrics information in a meaningful way as opposed to letting our users pick from Prometheus, Grafana, etc off the bat and having them set it up themselves.

In the end, if a user wants to use a tool like Prometheus they can write an adapter to convert our Metrics Protocol into a format suitable for Prometheus.

I see, thanks!

For those who don't know (including me), Wallaroo appears to be a competitor to and/or similar to Kafka: high-throughput stream processing. If I'm incorrect, please let me know!

Hi Mike,

Yes. The core of Wallaroo is for high-throughput, low-latency stream processing.

We currently have a Python 2.7 API released and Go API preview release out.

I don't think of it as a competitor to Kafka although Kafka streams does provide a means to do stream processing.

There's a number of JVM based solutions out there (Apache has several). We are looking to be the solution for folks who want to use Python, JavaScript, Go and other non-JVM languages.

Pardon a newbie here. I was wondering how, or if it's even possible, to use wallaroo with c# and/or java apps?

Hi VP of Engineering at Wallaroo Labs here.

At this time, we support Python 2.7 and Go (preview release is out).

Python 3 and JavaScript are on our roadmap.

Java/C# would be things we'll be considering for the future.

Reply on news.ycombinator.com